584683 - Make UA string "Gecko/2.1" or "Gecko/2.1 (Linux i686) Firefox/5.0"

Status: RESOLVED WONTFIX

(Core :: Networking: HTTP, enhancement)

Description

[Ben Bucksch (:BenB)]

In the last months, we have a number of bugs that cut unnecessary parts of the User Agent string. A lot of that is inspired by <https://panopticlick.eff.org>.

We already removed:
* Crypto strength ("; U") - bug 572668
* Locale ("; en-US") - bug 572656

We have proposals to remove:
* Gecko build date, replace with major version - bug 572661 and bug 572659
  It doesn't make sense, a 3.6.12 minor release may have a later date than
  the 3.7.0 major release, and exposes which precise security bugs you're
  vulnerable to, and due to the multitude of builds (e.g. from linux
  distros or custom complications) allow easier identification in
  panopticlick.
* "Firefox/.." - bug 572665
  For the site, it should be irrelevant whether it's Firefox, Minefield,
  Seamonkey, Camino or KMeleon. All that matters is the Gecko rendering
  engine and its major version.
* OS and CPU "Linux i686" or "Windows NT 6.1" - bug 57555 comment 79
  That's only needed for software downloads, not for websites.

Furthermore, the following is entirely redundant:
* "Mozilla/5.0 "
  All desktop browsers have that
* Window system ("Windows; " at the start)
  It's pretty much implied by the OS, and not relevant for the site anyways.

I realize many of these are, each by itself, controversial. Please argue them in their respective bugs or the newsgroups.
What I'd like to propose here is a summary view and tracking. If we do all of the above, the user-agent string could be just:

Gecko/1.9.2 (Linux)

That's all. Nice, slim, and as the RFC intended it originally. And, most importantly, sufficient.

Downsides, if we were to do that:
- Software downloads (of executables) cannot automatically offer the
  right binary version. We either accept that or create another mechanism
  for the site to query that on demand, possibly only with user permission.
- Sites have stats about market of Firefox vs. Seamonkey vs. Camino. All
  they know is the market share of all Gecko browsers. I claim that's all
  they need to know. Upside is that they cannot look for "Firefox" and
  break Minefield, as several sites currently do.
If both of these were considered serious, and kept in the UA string, it would look like:

Gecko/1.9.2 (Linux i686) Firefox/4.0

That's still much nicer than the current:
Mozilla/5.0 (X11; Linux i686; rv:2.0b2pre) Gecko/20100708 Firefox/3.6
or the former:
Mozilla/5.0 (X11; Linux i686; U; en-US; rv:2.0b2pre) Gecko/20100708 Firefox/3.6

Comment 1

[Ben Bucksch (:BenB)]

> - Software downloads (of executables) cannot automatically offer the
>   right binary version. We either accept that or create another mechanism
>   for the site to query that on demand, possibly only with user permission.

Note that a site could still query the JS navigator.oscpu property. That would allow us to put a user confirmation in between, and avoid useless bytes in every single request.

Comment 2

[Dave Garrett]

Personally I'd prefer this ordering:
Gecko/2.0.x Firefox/4.0.x (Linux i686)

If we're willing to break most sniffers this would be great. Do we actually have an idea of which sniffer scripts are the most common and how the general masses of poorly written sniffers would react?

To state it here as well as the other bugs it has been mentioned: there's also a small performance gain from slimming down the UA string by removing extra useless bytes we're always sending out with every HTTP request.

Comment 3

[Ryan Jones-Ward [:sciguyryan]]

Wouldn't it make thing easier if all browsers sat down and finally set out a standard for this?

Were it all standardized I'm sure it would then be worth the effort for people to go off and change the sniffers that are around the place. All these changes are just making matters worse otherwise since there are multiple variations that now identify the same browser.

Would anyone be willing to set own a standard for this?

Comment 4

[Karsten Düsterloh]

The User-Agent format is already standardized, see RfC 2616 section 14.43.

Comment 5

[Ryan Jones-Ward [:sciguyryan]]

(In reply to comment #4)
> The User-Agent format is already standardized, see RfC 2616 section 14.43.

Even better. Then would it not be good to get everyone to try and follow that format?

Comment 6

[Karsten Düsterloh]

We do and always did for the last couple of years, as do/did others. The problem is the actual content, not the format. The problem is that almost everyone has Mozilla/5.0 in its UA, or random maybe important info added as a comment...

If there is an inclination to strip down the UA, we should do a real clean cut and get rid of all the noise - we're breaking crappy sniffers anyway...

Comment 7

[Ben Bucksch (:BenB)]

Removing Mozilla/5.0 is bug 527886

Comment 8

[[e-mail]]

We're not doing this. There's some good discussion here, but it needs to be broken out into more granular bugs. See bug 586165 for a pretty decent list.

Comment 9

[Karsten Düsterloh]

(In reply to comment #8)
> We're not doing this.

Care to elaborate on this, especially who and most important why exactly?

Comment 10

[[e-mail]]

Where this bug was filed -- Gecko. There *are* some bits you can presently customize at runtime (prefs etc), but for the most part all the UA stuff is burnt in at build time. People rolling their own builds with the ability to patch the source (distributions, for instance) can change it.

The reason is that it's just too minimal. There are some good uses for things like OS, platform, and product (Firefox) in the string. The various other bugs have some discussion on those. Hiding fundamental properties like that doesn't help the fingerprinting cause, since there are myriad other ways to determine them.

Comment 11

[Ben Bucksch (:BenB)]

> Where this bug was filed -- Gecko

hm? This is filed against the HTTP lib, and it's RFC 2616 which defines the User Agent string, and nsHttpHandler.cpp which implements it. It's the right component. I didn't mean to imply anything about runtime vs. build time config.

> There are some good uses for things
> like OS, platform, and product (Firefox) in the string.

All of these are in:
Gecko/1.9.2 (Linux i686) Firefox/4.0
which comment 0 mentioned as possible outcome of this bug.

> The various other bugs have some discussion on those.

Yes, and this was a tracker.
Anyway, you are working on it, it doesn't really matter much if bug 586165 or this bug fixes it, as long as it gets fixed.

Comment 12

[[e-mail]]

(In reply to comment #11)
> hm? This is filed against the HTTP lib

Yep, which is part of Gecko. We could make the whole UA string customizable at build time, but that's a bunch of work that would only serve to fragment the consistency across various products that embed Gecko. That's what the 'comment' and 'extra' fields in the UA string are for.

> Yes, and this was a tracker.

Ah, OK -- sorry. (The title implied otherwise, perhaps we could've changed that.) We have enough other trackers though, so if you have specific changes in mind I'd suggest you file bugs on them. :)

Comment 13

[Ben Bucksch (:BenB)]

Given bug 572661 comment 94, and the postpost to Firefox 5, and the reduction of bug 586165, which this bug was closed in favor of, I am reopening.

This bug shows where we *could* be, and that it's realistic.

We should gather some data how much of the web it breaks, by automated tools and Firefox 5 alpha/early beta build feedback.

Comment 14

[rsx11m]

By which logic? This bug still doesn't seem to be justified on its own...

(In reply to comment #8)
> We're not doing this. There's some good discussion here, but it needs to be
> broken out into more granular bugs. See bug 586165 for a pretty decent list.

This still applies, and bug 572650 remains open as well for tracking. You are proposing a minimal string which is essentially the sum of all other bugs which were filed (and some of which were won't-fixed rather than just deferred).

Comment 15

[Ben Bucksch (:BenB)]

> By which logic?

Being closer to what the standard intended.
<http://asg.web.cmu.edu/rfc/rfc2616.html#sec-14.43>
User-Agent: CERN-LineMode/2.15 libwww/2.17b3

> You are proposing a minimal string which is essentially the sum of all
> other bugs which were filed

Exactly.

> (and some of which were won't-fixed rather than just deferred).

I still have to see reasons on that, and I think they were closed in light of FF4 and without data. See comment 13:
"We should gather some data how much of the web it breaks, by automated tools
and Firefox 5 alpha/early beta build feedback."
This tracking bug is useful for that as well.

Comment 16

[rsx11m]

(In reply to comment #15)
> Being closer to what the standard intended.
> <http://asg.web.cmu.edu/rfc/rfc2616.html#sec-14.43>

That section states an example and defines the syntax, it doesn't say anything about which details to which extent the UA string is supposed to provide.

> [sum of all other bugs] Exactly.

Thanks for agreeing, so this bug is technically invalid as it doesn't have a single purpose and isn't a tracking bug either, and on top of that replicating what's said in other bugs anyway. The question being, what do you want here?

Comment 17

[Ben Bucksch (:BenB)]

It's a tracking bug.

Also, as I said, it helps testing. We can test all changes together instead of each individual one, and see how much breaks.
This bug is not hurting anyone, and it already has people voting for it.

Comment 18

[j.j.]

Such a change should not ignore the Version-10 Problem
 http://my.opera.com/hallvors/blog/2008/12/19/10-is-the-one

We need somthing like this ;)
 Gecko/00002.1 Firefox/00005.0 (Linux i686)

Comment 19

[Markus Popp]

(In reply to comment #2)
> Personally I'd prefer this ordering:
> Gecko/2.0.x Firefox/4.0.x (Linux i686)

I like this format.

And if I interpret the standard correctly, this would totally be in line with it.

Also, please get rid of the Mozilla/5.0. I don't see how this part makes sense these days, and somebody should begin to stop using it. Why not Mozilla?

Comment 20

[Ben Bucksch (:BenB)]

> Also, please get rid of the Mozilla/5.0.

That's bug 527886, see blockers. Please argue there.