384721 - Put reference to Firefox in User Agent (ua) string

Status: RESOLVED FIXED

(Camino Graveyard :: General, defect, P3)

Description

[froodian (Ian Leue)]

We've been talking at the meet-up about including something like "like Firefox x.x.x" or even "not Firefox x.x.x" (with the Firefox release that corresponds with the relevant Camino release).  This could significantly improve compatibility with websites that're sniffing "Firefox."  On the other hand, there are "a non-zero number of sites" (pink's quote) that are currently specifically sniffing for Camino to feed us specialness, which doing this could break.  Additionally, there are potential legal implications, so we should definitely talk to MoCo before doing this.

Comment 1

[Samuel Sidler (old account; do not CC)]

Frank or Gerv, are there any trademark/legal concerns with us doing this? This would only be an entry in the user agent and wouldn't be seen (except by servers). I think this falls under the realm of the "Mozilla" in the front of 90% of UAs now, where it's not a trademark violation for that to happen.

Sample UA based on proposal and on what other browsers use:

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en; rv:1.8.1.5) Gecko/20070614 (like Firefox) Camino/1.6

Comment 2

[Frank Hecker]

I don't see any trademark implications to this at all. You are simply stating (in the UA string) that Camino is like Firefox or compatible with Firefox in some sense; you are not representing that Camino *is* Firefox. So I think you can stop worrying about the legal implications and just consider the technical issues.

Comment 3

[Samuel Sidler (old account; do not CC)]

Thanks Frank, sounds good. Now we need a patch. ;)

Comment 4

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

One other thing to keep in mind is that browser detection on some sites breaks when there are things in parens following the Gecko version.

Comment 5

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: sample patch

If you're not married to "(like a fox)" appearing before "Camino", this sample patch will work, and it follows the recommended method of adding extra identifiers to the UA.  (This particular patch won't apply on the branch for obvious reasons.)

You'd get a UA like this if we applied it to 1.5 ML:

Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en; rv:1.8.1.4) Gecko/20070509 Camino/1.5 (MultiLang) (like Firefox)

Comment 6

[Gervase Markham [:gerv]]

There are no trademark/legal concerns, but have you consulted with e.g. dbaron or brendan? We've spent ages evangelising people to check the Gecko version (and so support all Gecko browsers) rather than Firefox specifically. Doing this would be basically conceding defeat, and be a bad thing for web compatibility in the long run (because people would be sniffing Firefox versions rather than rendering engines, and getting worse sniffing).

We do have a standard which is supposed to cover how we do this, I think:
http://www.mozilla.org/build/revised-user-agent-strings.html

Gerv

Comment 7

[Mike Pinkerton (not reading bugmail)]

Comment on attachment 268718 [details] [diff] [review]
sample patch

rs=pink

Comment 8

[Håkan Waara]

Comment on attachment 268718 [details] [diff] [review]
sample patch

Looks good to me.

Comment 9

[Stuart Morgan]

(In reply to comment #6)
> We've spent ages evangelising people to check the Gecko version
> (and so support all Gecko browsers) rather than Firefox specifically. Doing
> this would be basically conceding defeat, and be a bad thing for web
> compatibility in the long run (because people would be sniffing Firefox
> versions rather than rendering engines, and getting worse sniffing).

Yes, obviously evangelism is the ideal approach, and it's why we haven't ever done this before. Frankly though, we are tired of getting feedback about very high-profile sites that are broken, have been broken for a very long time despite many attempts to get them to change, and have in many cases explicitly stated that they have no intention of changing in the future (we have many cases where we've emailed site admins to explain that Camino would work perfectly if they changed their sniffing, with pointers to relevant documentation, and the official response is "We don't support your browser"). While I understand why you think it will hurt compatibility, the bottom line is that although there are a lot of people who understand and to the right thing, for all the people who refuse to do the right thing having a 0.04% market-share browser continue to stand their ground is clearly not going to be the deciding factor.

I agree that it's conceding defeat, but after a certain amount of time fighting a battle without making noticeable progress and no expectation that it will change, conceding is better than continuing to hurt all of our users.

Comment 10

[Stuart Morgan]

(In reply to comment #5)
> Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en; rv:1.8.1.4) Gecko/20070509
> Camino/1.5 (MultiLang) (like Firefox)

We talked about maybe doing "like Firefox/2.0.0.x" (or whatever) to try to handle sites that do version checking too; we should watch for sites that might now actively block us because they think we are Firefox 1.5.

Comment 11

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

I'm opposed to this, for basically the reasons given in comment 6, and I'm surprised to see this significant a change happening in a bug that doesn't have any evidence of broken sites.  (For what it's worth, those sites would also be broken in Firefox alphas/betas.)  Do you have references to bug reports about such sites?

Comment 12

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(And, for what it's worth, for trademark/legal issues about the Firefox trademark, I think you should ask Mozilla *Corporation*.)

Comment 13

[Samuel Sidler (old account; do not CC)]

(In reply to comment #11)
> Do you have references to bug reports about such sites?

All the dependencies of bug 334967. One of the most visible ones is bug 330575.

Comment 14

[Samuel Sidler (old account; do not CC)]

(In reply to comment #12)
> (And, for what it's worth, for trademark/legal issues about the Firefox
> trademark, I think you should ask Mozilla *Corporation*.)
 
As I understand it, MoFo owns the trademark and licenses it to MoCo, which is the primary protector of the trademark. However, MoFo can also make licensing and trademark calls. I could be wrong about this point, but that's my understanding.

Comment 15

[Stuart Morgan]

(In reply to comment #11)
> I'm opposed to this, for basically the reasons given in comment 6

Do you have a solution for the hundreds of thousands of users who are getting a bad experience with Camino currently that's better than "hope that somehow the last several years of evangelism suddenly start working much better"? If so, we'd be more than happy to consider it as an alternative.

Comment 16

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Sometimes it's worth standing up for something even though it doesn't provide the best user experience right now.  If we didn't believe that we'd have given up on Gecko and shipped a Windows-only browser that wraps Trident.

From a technical perspective, this is clearly the wrong solution.  It adds yet more bytes to the user-agent header that all Gecko apps must ship; that has a real cost in bandwidth (especially at the point where the headers end up needing an extra packet).

And giving up here makes it more likely we'll give up the next time this comes around, whether that's because of the browser that takes over from Firefox or because Firefox has to send "really Firefox".  I'm not convinced that the list of 22 bugs in https://bugzilla.mozilla.org/showdependencytree.cgi?id=334967&hide_resolved=1 is enough to make us want to do that.

Comment 17

[Stuart Morgan]

> Sometimes it's worth standing up for something even though it doesn't provide
> the best user experience right now.

Agreed, but we aren't talking about "right now". We are talking about years of the same thing, and, again, explicit statements from some of the large sites that they have no plans to change their behavior with regards to Camino.

What users are doing right now is downloading a third-party preference pane that wraps the useragent hidden prefs and spoofing as Firefox because it's the only way to get to those sites. I fail to see how that's better.

> From a technical perspective, this is clearly the wrong solution.

So what is the correct technical solution here?

> It adds yet more bytes to the user-agent header that all Gecko apps must ship

How so? Camino doesn't set the policy for all Gecko apps.

Comment 18

[Stuart Morgan]

And just to reiterate an important point that I think is being overlooked here: Camino is a Mac-only browser. Our theoretical maximum share of the internet is in the 4-5% range, so if we had 0.5% that would be huge for us. 0.5% isn't even a blip in terms of leverage against site authors.

It's great that Firefox exists, and that it has the kind of leverage that can make things change in the way things are done on the internet, but Camino is not and never will be able to fill that role.

Comment 19

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(In reply to comment #17)
> > It adds yet more bytes to the user-agent header that all Gecko apps must ship
> 
> How so? Camino doesn't set the policy for all Gecko apps.

Why should this decision be made separately for Camino than for other non-Firefox Gecko apps?  (And even if it should, Camino giving in increases the pressure on other non-Firefox Gecko apps to give in too.)

Comment 20

[Stuart Morgan]

(In reply to comment #19)
> Why should this decision be made separately for Camino than for other
> non-Firefox Gecko apps?

For the same reason that we make, for example, different UI decisions than SeaMonkey: we are a different product, made by a different group, with different goals.

Comment 21

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Why should *this* decision be made differently?  What differences would support reaching a different conclusion?

Comment 22

[Samuel Sidler (old account; do not CC)]

For the record, this decision has been made by other browsers already. They claim they are Firefox, at least we add the "like" (although maybe adding a version number is smart as Stuart mentioned in comment 10)...

Epiphany under Ubuntu 6.10: Mozilla/5.0 (X11; U; Linux i686; en; rv:1.8.1) Gecko/20060601 Epiphany/2.16 Firefox/2.0 (Ubuntu-feisty)

Epiphany under XUbuntu 7.04: Mozilla/5.0 (X11; U; Linux i686; en; rv:1.8.1.3) Gecko/20061201 Epiphany/2.18 Firefox/2.0.0.3 (Ubuntu-feisty)

Flock: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.12) Gecko/20070530 Firefox/1.5.0.12 Flock/0.7.14

Comment 23

[Stuart Morgan]

(In reply to comment #21)
> Why should *this* decision be made differently?  What differences would support
> reaching a different conclusion?

The fact that we know we can never be a significant factor in making a change to the way that pages are made, and that we are more interested in martyring our users for a hopeless cause. Other browser Gecko-based browsers that have chosen not to do this presumably either believe that they have enough market share to not be ignored, or feel that this principle (and let's be clear, we are not violating any standards here, so that's all it is) is more important than the experience of their end users.

I don't understand why you are framing this in terms of us needing to justify ourselves to all other Gecko-based browsers; it's not relevant. If all of us made the same decisions about everything then there would only need to be one Gecko browser. This is a Camino decision, and it has been decided by the Camino team, in full awareness of the issues involved.

If you want to discuss this further, I suggest it take place somewhere other than bugzilla.

Comment 24

[Stuart Morgan]

In light of the fact that the examples in comment 22 all do it, I'd suggest again that we use the relevant Firefox version number in our UA as well, rather than checking in the current patch as-is.

Comment 25

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(In reply to comment #23)
> I don't understand why you are framing this in terms of us needing to justify
> ourselves to all other Gecko-based browsers; it's not relevant.

It is relevant if Camino is a non-negligible share of non-Firefox Gecko-based browsers, since it will increase the pressure on others to follow this decision.

> If all of us
> made the same decisions about everything then there would only need to be one
> Gecko browser. This is a Camino decision, and it has been decided by the Camino
> team, in full awareness of the issues involved.

There's a big difference between user-interface decisions and Web-content-handling decisions.  That's one of the reasons there are multiple browsers (Firefox, Camino, SeaMonkey, Epiphany, etc.) using the same layout engine.  Different sets of people are knowledgable about those different areas, as recognized by mozilla.org module ownership.  Firefox owners don't make arbitrary changes to Gecko over the objections of Gecko module owners; I don't see why Camino should be different.

I'm perfectly willing to have a reasonable discussion with Samuel here, but I reject your claim that Gecko module owners have no say over Gecko (which seems like bug 324501 all over again).

Comment 26

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: like so? - trunk

Comment 27

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: like so? - branch

Ultimately I think we want to automate this, but this should do for now.

Comment 28

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Please please please don't add more places where version numbers need to be bumped manually.

Comment 29

[Samuel Sidler (old account; do not CC)]

David, Smokey posted that before reading your latest comment. When you're on the "Create an attachment screen", bugzilla doesn't warn you if there's been new comments. I doubt anyone wants to change the version number manually in any more places than we absolutely have to.

Comment 30

[Samuel Sidler (old account; do not CC)]

Also, if there needs to be further discussion of this, can someone please start a newsgroup post and link to it here?

Given that this is a "vendor comment" (per the document Gerv linked to) and, in this case, Camino is the "vendor", I honestly feel we should move forward with this. If it's something that needs to be backed out later after discussion, we can make that decision. Continuing discussion here isn't going to help.

Comment 31

[Gervase Markham [:gerv]]

I have posted to the Camino newsgroup and to m.d.t.layout, with follow-ups to the latter.

Gerv

Comment 32

[Mark Mentovai]

I think that this is a perfectly valid solution for the near term, given all of the arguments above (note that I wasn't a part of the initial discussion that led to this bug.)

I worry that if enough browsers start advertising "like Firefox", site developers might become more restrictive in their definition of what makes a Firefox.

The decision at this level is ultimately one for the Camino team, and while we solicit and respect input of other members of the Mozilla community, I certainly hope we won't hurt any feelings by taking this position.  It's not the place for the Mozilla project as a whole to dictate what can and can't be present in a specific application's user-agent string (unless, of course, certain rights are infringed).

If we do this, it absolutely should take the relevant Firefox version number from browser/config/version.txt into account.  Conveniently, we already check this file out, so one option is to generate camino/resources/application/all-camino.js from an all-camino.js.in.

If we do this, I'd like to have an idea of how many sites it fixes - running through the dependency list of bug 334967 would be helpful.  It would also be nice to know if it breaks anything.

Comment 33

[Samuel Sidler (old account; do not CC)]

I talked this over with David today. We all agree that this sucks and isn't an ideal thing for the web. However, until web developers improve their browser sniffing code, we have no decent way to improve the experience for our users. Any improvement may very well need dedicated tech evangelism resources from the overall Mozilla project, whether that be MoFo, MoCo, or an organized effort by the community. Until such a time as this improvement happens, I feel allowing our users access to the web as a whole is an obligation we can't afford to ignore.

To be clear, resolving this bug is a loss for idealism, but a win for our users.

Unless there are others in disagreement, let's go ahead and move forward. We can move discussion of ideals and how to improve our TE message to the newsgroup post Gerv made in m.d.t.layout.

I'll also mention that the intent (which I think is said above) is to land this on the 1.8 branch (where Camino 1.6 development is happening) and trunk (where Camino 2.0 development is happening) in a way that pulls the version number from version.txt. By testing our development nightlies, we'll determine if this change has any ill effect on the web and potentially back it out if it does.

I've marked Smokey's patches as obsolete since they don't make the change required but were to be more as a stop-gap measure.

Comment 34

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: first stab at the right way

This is the first stab at doing this the right way.  I'm sure it's not completely right, but it works in my trunk debug, and I wanted to learn something.  So, mento, pointers welcome ;)

Issues:

1) /resources/ is a symlink in the objdir, so this just deposits the "new" all-camino.js right back where the old all-camino.js lived (no project changes, though!)

2) We might want to cvs move all-camino.js to all-camino.js.in, patch that with

+// work around stupid sites sniffing for firefox instead of gecko
+pref("general.useragent.extra.notfox", "(like Firefox/%FOX_APP_VERSION%)");
+

then do the Makefile.in change and cvs remove the old all-camino.js, depending on how much we care about cvs history of our prefs file.

3) Um, there was something else, but I've forgotten it in the process of attaching the patch.

4) This won't apply on the branch (Makefile changes), and I haven't tested except trunk debug, but since I expect r-, I figured I'd start the process rolling ;)

Comment 35

[Gervase Markham [:gerv]]

Samuel: you asked for a discussion to be started, and so I have done so. It would be polite at least to wait until it has concluded. Is the Camino team about to make a release, meaning that this change has to be made very quickly?

Gerv

Comment 36

[Stuart Morgan]

I don't think Sam was really asking for a discussion so much as echoing/clarifying my request that if there was going to be continued argument it not happen in this bug.

Comment 37

[Gervase Markham [:gerv]]

OK, then. Let me put it another way: I'm asking you to please give me a week to make some enquiries and investigate some options.

Gerv

Comment 38

[Mark Mentovai]

Comment on attachment 268877 [details] [diff] [review]
first stab at the right way

	sed -e "s/%FOX_APP_VERSION%/$(FOX_APP_VERSION)/" $(srcdir)/resources/application/all-camino.js.in > $(srcdir)/resources/application/all-camino.js

Don't put your generated file into the srcdir, put it into the objdir.  resources is a symbolic link in the objdir?  Put it somewhere else (and kick the project file appropriately).

It's customary to use @FOX_APP_VERSION@ instead of %FOX_APP_VERSION% (based on what autoconf does), but there are exceptions throughout the tree.

Add something to remove the generated file to a "clean clobber" rule.

The proper way to do handle this autogeneration stuff in the Makefile is to make a rule to build all-camino.js if its dependencies, all-camino.js.in or the browser version.txt file, have changed.  The libs target should depend on all-camino.js.

Comment 39

[Stuart Morgan]

(In reply to comment #37)
> I'm asking you to please give me a week to
> make some enquiries and investigate some options.

Sure, waiting a week before landing this is fine. Given that it's unlikely that there will be an internet-wide solution for the September timeframe of 1.6 though, I don't see any reason not to continue preparing a patch.

Comment 40

[Gervase Markham [:gerv]]

Can I also ask: if there are major sites you know of that have this problem which don't have Tech Evangelism bugs filed about them, please can you file them and make them depend on the tracking bug 334967?

Thanks,

Gerv

Comment 41

[Josh Aas]

I am totally against doing this, fwiw. My reasons are basically the same as dbaron's.

Comment 42

[Stuart Morgan]

(In reply to comment #32)
> If we do this, I'd like to have an idea of how many sites it fixes - running
> through the dependency list of bug 334967 would be helpful.  It would also be
> nice to know if it breaks anything.

With the possible exception of four that I couldn't test because I didn't have the necessary account (Sirius, WSJ, Blackboard, and NatWest), every single one of them. And anecdotally, from several years of investigating most of the user complaints we've received (for those who don't share the opinion that years of active investigation isn't relevant experience in this decision), the vast majority of broken sites are doing an indexOf("Firefox"), and will thus work correctly.

As for breaking things, we know of at least one site with a parser that falls down when confronted with (<anything>) at the end of the UA, because it breaks with the ML version, and many Linux Firefox distros, but I can only think of that one case offhand.

Comment 43

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: take 3, trunk

This works as mento wanted now, thanks to a few pointers from him.  Due to some context issues, separate patches are needed for branch/trunk.

Comment 44

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: take 3, branch version

Comment 45

[Mark Mentovai]

Comment on attachment 269425 [details] [diff] [review]
take 3, trunk

These both look good.  At checkin time, don't forget to cvs remove mozilla/camino/resources/application/all-camino.js .

Comment 46

[Mike Pinkerton (not reading bugmail)]

Comment on attachment 269425 [details] [diff] [review]
take 3, trunk

sr=pink

Comment 47

[Mike Pinkerton (not reading bugmail)]

Comment on attachment 269426 [details] [diff] [review]
take 3, branch version

sr=pink

Comment 48

[Samuel Sidler (old account; do not CC)]

(In reply to comment #37)
> OK, then. Let me put it another way: I'm asking you to please give me a week to
> make some enquiries and investigate some options.

Gerv, any update here before we get this checked in?

Comment 49

[Jesse Ruderman]

See also bug 385999, which proposes *removing* "Firefox" from Firefox's UA string.

Comment 50

[Gervase Markham [:gerv]]

Sam: discussion still seems to be happening in bug 385999; I think it's clear that the profile of this problem is being raised. 

I was more hopeful last week that something could be sorted; it seems we have less Tech Evangelism resource than I imagined. But it would still be good if we could let the discussion play out before checking in, if there's no rush on the Camino side.

Gerv

Comment 51

[Samuel Sidler (old account; do not CC)]

Gerv: The only real rush is because we want to get it checked in so we can have lots of time for users of our nightlies to see if there's anything major that's broken. I'm not sure how long the discussion in bug 385999, but we're targeting a late August / early September release of Camino 1.6. The sooner we land this, the more coverage we can get.

Comment 52

[Håkan Waara]

FWIW, if Firefox goes ahead with bug 385999, I'm in support of that and as a result this change would not make sense.

Comment 53

[Samuel Sidler (old account; do not CC)]

(In reply to comment #52)
> FWIW, if Firefox goes ahead with bug 385999, I'm in support of that and as a
> result this change would not make sense.

Except on the branch... Given that the change in bug 385999 wouldn't see its full effect until Firefox 3 was released, I can see this being useful on the branch.

Comment 54

[Samuel Sidler (old account; do not CC)]

Bug 385999 has been WONTFIXed. The patches in this bug can land.

Comment 55

[Mark Mentovai]

OK, let's get this in, then.  We're not married to it (at least I'm not), but we'd like to get experience with it before our 1.6 release.  We'll continue to monitor 385999 or any replacement bugs to see if they impact our situation.

Thanks to Gerv for looking into alternatives, and everyone else who contributed to the discussion here, on the other bug, and in the newsgroup.

I'll check this in tomorrow morning.

Comment 56

[Mark Mentovai]

Checked in on the trunk and 1.8 branch (for Camino 1.6 but not 1.5.x).

Marking fixed1.8.1.5 which isn't strictly correct because Camino 1.5.x is also Gecko 1.8.1-based.  Have we settled on keywords or whiteboard strings to use for this stuff?

Comment 57

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to comment #56)
> Marking fixed1.8.1.5 which isn't strictly correct because Camino 1.5.x is also
> Gecko 1.8.1-based.  Have we settled on keywords or whiteboard strings to use
> for this stuff?

[camino-1.5.1] and friends in the whiteboard for 1.5.x changes, 1.8.1.x keyword for 1.6 changes.  http://wiki.caminobrowser.org/QA:Keywords_%26_Status_Whiteboard#Status%20Whiteboard