Closed Bug 582421 Opened 14 years ago Closed 12 years ago

Make general.useragent.override temporary

Categories

(Core :: Networking: HTTP, defect)

Product:
Core
Component:
Networking: HTTP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: davemgarrett, Unassigned)

References

(Blocks 1 open bug)

Details

From discussion in bug 581008 about how to best protect users from broken UA strings caused largely by external software abusing general.useragent prefs.

The result of bug 581008 will leave only general.useragent.override to modify user agent strings freely. (and possibly also vendor prefs) This is necessary to let users spoof and test as needed but ban other methods which are abused frequently. In order to resist both dumb overrides from external software and overrides done by the user but forgotten to be reset, I suggest we make this preference temporary and delete it on application startup or shutdown.

If we still wish to allow making this pref permanent via user.js, then either this needs to be reset after loading prefs.js and before user.js or it needs to be reset on shutdown instead of startup.
> I suggest we make this preference temporary and delete it on
> application startup or shutdown.

I disagree, there are other reasons that faulty websites for a user wanting to set the user agent. That being fun (Toaster/1.0), advocacy ("Fred Nonsense" to counter user agent sniffing), permanent testing, or wanting to masquerade as a different browser.

Also, for browsers other than Firefox is may be necessary to continuously run with a user agent string pretending to be e.g. Firefox, for sites to work.

Even for workarounds, what if I a site I visit frequently needs the UA to be set? I don't see how it helps to only temporarily set the UA and then the user having to set it manually after every start.

> resist both dumb overrides from external software

Resetting won't help with that. An extension that wants to be rough will just set it after every start. So, you just annoy users without stopping extensions at all.

Suggest to WONTFIX this proposal.
(In reply to comment #1)
> I disagree, there are other reasons that faulty websites for a user wanting to
> set the user agent. That being fun (Toaster/1.0), advocacy ("Fred Nonsense" to
> counter user agent sniffing)

These aren't normal use cases we should be supporting.

> permanent testing, or wanting to masquerade as a different browser.

This would still be possible by using user.js or via an addon to let users manually keep UA spoofs. Dropping support for appending misc stuff is the topic of bug 581008. Adding misc stuff to a full override is a bad idea, as its versions won't increment with updates.

> Also, for browsers other than Firefox is may be necessary to continuously run
> with a user agent string pretending to be e.g. Firefox, for sites to work.

This is already covered in bug 581008 with Dão's addition of general.useragent.compatMode.firefox.

> Even for workarounds, what if I a site I visit frequently needs the UA to be
> set? I don't see how it helps to only temporarily set the UA and then the user
> having to set it manually after every start.

Leaving a spoofed UA for all other sites might cause problems. Per-site spoofing has been suggested as an additional mechanism, but it's probably the domain of addons.

> > resist both dumb overrides from external software
> 
> Resetting won't help with that. An extension that wants to be rough will just
> set it after every start.

I'm talking about dumb software here, not malicious.
> These aren't normal use cases we should be supporting.

We always have, and so does most other similar software.
I think they are valid use-cases.

This header is sent in every request. This is *my* browser, my software, and I make myself appear to other parties the way *I* want. The browser is called "user agent" for a reason.

> > a user agent string pretending to be e.g. Firefox, for sites to work.
> This is already covered in bug 581008 with Dão's addition of
> general.useragent.compatMode.firefox.

Predictable response, but you didn't read well enough. I said "e.g." Firefox. It's just as valid to masquerade as Opera, WebKit or MSIE9.

> Per-site spoofing has been suggested as an additional mechanism

Agreed. For workarounds, that's the way to go. And would be a *must*-have before implementing your suggestion here.

(However, as said, that doesn't fix the many other reasons for wanting to change UA.)

> I'm talking about dumb software here, not malicious.

It's a fine line. If you're dumb, and you want something, you do what it takes to get it. Setting on every start is not malicious per se.
We have to aim for the vast bulk of users here, not those who want to add the word toaster to their UA string.

Again, I'm not suggesting a ban on UA spoofing or even semi-permanent spoofing. You could still do these things non-temporarily via user.js if you really wanted to, better yet via an addon managing things.

(In reply to comment #3)
> It's just as valid to masquerade as Opera, WebKit or MSIE9.

For what it's worth, the patch in bug 581008 has a quick pref for IE8.

> Setting on every start is not malicious per se.

This would at least let the UA versions update, terminology debate aside. ;)
(In reply to comment #4)
> We have to aim for the vast bulk of users here, not those who want to add the
> word toaster to their UA string.

That's an odd remark, given that "the vast bulk of users" doesn't care at all what there UA is, as long "the web just works".
I still don't understand how you want to prevent extensions which decide to advertise themselves from adding their stuff to the UA string. The removal of .extra.* won't prevent that if .override remains in its current form, even if it's temporary. The add-on would simply check at startup the UA string, and if it doesn't find its token it will add it and write back to .override (that's what has happened before the introduction of the .extra.* prefs, if I'm not mistaken, and would happen again once that other loophole is closed). The only side effect avoid by resetting .override is the freezing of the application token to the current version in the UA string after an update.

How about an alternate way to specify the UA override in a file, assuming that an extension won't be able to modify it? You could make it "userAgent.ini" or something similar to easily recognize, and put the .vendor and .override values in there. This allows the user to edit it but keep it out of the reach of the add-on. Going a step further, make it domain/UA-string pairs, then you have the first building block for a site-specific spoofing!
(In reply to comment #5)
> (In reply to comment #4)
> > We have to aim for the vast bulk of users here, not those who want to add the
> > word toaster to their UA string.
> 
> That's an odd remark, given that "the vast bulk of users" doesn't care at all
> what there UA is, as long "the web just works".

And the web doesn't always work when they have a broken UA.

(In reply to comment #6)
> I still don't understand how you want to prevent extensions which decide to
> advertise themselves from adding their stuff to the UA string. The removal of
> .extra.* won't prevent that if .override remains in its current form, even if
> it's temporary. The add-on would simply check at startup the UA string, and if
> it doesn't find its token it will add it and write back to .override (that's
> what has happened before the introduction of the .extra.* prefs, if I'm not
> mistaken, and would happen again once that other loophole is closed). The only
> side effect avoid by resetting .override is the freezing of the application
> token to the current version in the UA string after an update.

First of all, the fixing of completely broken UAs and at least having the versions update is a good thing.

Secondly, we keep hitting this concept in various different bugs: we can't stop every bad thing, and shouldn't try. If we remove all the valid ways to modify the UA, then the best we can do is to consider any addons that then hack heavily to keep doing so Wrong and label them as malicious. Ditching other UA mod methods gets rid of the current plague of UA breakage and making this temp would fix any other normal breakage. If some 3rd party software decided to start a little war against the ban of this pointless identification, yeah, the best we can do is blocklist them; somehow, someway, it can be hacked around, always. That doesn't mean we stop trying to fix the lesser problems.

> How about an alternate way to specify the UA override in a file, assuming that
> an extension won't be able to modify it?

Impossible assumption.
(In reply to comment #7)
> And the web doesn't always work when they have a broken UA.

Well, that's why we have those .extra. prefs, so that interested users can add stuff to their UA in an easy and civilized manner. :-P

> First of all, the fixing of completely broken UAs

What's a "completely broken UA"? Violating RfC 1945/2068?
If that's the only concern, most violating UA can be made "legal" bei enclosing them in parens.

> and at least having the versions update is a good thing.

Not really, actually, if the user wants otherwise.

> we can't stop every bad thing, and shouldn't try.

Better to concentrate on real problems. :-P
(In reply to comment #8)
> Well, that's why we have those .extra. prefs, so that interested users can add
> stuff to their UA in an easy and civilized manner. :-P

They're going away in bug 581008 which this is an extension of.

> What's a "completely broken UA"? 

A UA that still says Firefox 2 when you're using Firefox 3.6, as an example. Sniffing this causes many sites to not work. See bug 577865 and its references. (though that bug may no longer be needed with bug 581008)
I don't see how this helps. Either add-ons will respect this and user-agent-switcher-using users will be annoyed they have to set their preferred override every startup, or the pushy add-ons will simply add their junk every start.

Note that we cannot prevent this by adding review requirements for add-ons distributed through AMO, some of the worst offenders are distributed via other channels (e.g. .NET)
(In reply to comment #10)
> or the pushy add-ons will simply add their
> junk every start.

... without freezing the string beyond version changes and when the add-on has been uninstalled.
(In reply to comment #10)
> Either add-ons will respect this and
> user-agent-switcher-using users will be annoyed they have to set their
> preferred override every startup

There's no reason why an add-on like User Agent Switcher shouldn't override the UA string with every startup.
> There's no reason why an add-on like User Agent Switcher shouldn't
> override the UA string with every startup.

"User Agent Switcher could not be installed because it is not compatible with Minefield 4.0b3pre."
So, not a solution for the "annoyed user" / developer.
(In reply to comment #13)
> > There's no reason why an add-on like User Agent Switcher shouldn't
> > override the UA string with every startup.
> 
> "User Agent Switcher could not be installed because it is not compatible with
> Minefield 4.0b3pre."
> So, not a solution for the "annoyed user" / developer.

I'm pretty sure you're capable of editing user.js. Other than that, can we please focus on end users? You are free to run Firefox 3.6 if you feel that dealing with development builds is too much of a burden.
> can we please focus on end users?

Sure. Then this is a wontfix, because normal end-users don't know what User-Agent is and don't go <about:config>.
And we already established that extensions can still permanently override the UA string.
Leaves the argument of not freezing it, but that's what the general.useragent.extra.* pref solves. Oh, wait, you're removing that (bug 581008).
(In reply to comment #15)
> > can we please focus on end users?
> 
> Sure. Then this is a wontfix, because normal end-users don't know what
> User-Agent is and don't go <about:config>.

This bug is about protecting unaware users from add-ons overriding the string, so your reasoning completely fails to apply.

> And we already established that extensions can still permanently override the
> UA string.

Yep, but with this bug being fixed they won't accidentally freeze it.

> Leaves the argument of not freezing it, but that's what the
> general.useragent.extra.* pref solves. Oh, wait, you're removing that (bug
> 581008).

Correct, so your argument collapses. I suggest you start over.
Or maybe you don't remove .extra.*? That might be a solution, too, no?

We seem to agree on:
* We want to allow users to override the UA string
  (otherwise you wouldn't suggest the UA string switcher extension)
* We don't want extensions to change the UA string

Now, if we:
* Prevent AMO extensions from changing the UA string prefs
* keep .override, and keep it permanent, as now
* keep .extra.*, as now
We'll get:
* Users can change .override, as they please, and won't be annoyed.
  They are not required to use an extension either.
* AMO extension will not change the UA string, per policy
* Rough third-party extension will keep changing .extra.* and will not
  freeze the UA string.

If we make .override temporary and remove .extra.*, we get:
* Users who've always set .override will be annoyed.
* I have to constantly install, adapt, update extensions just to get the
  UA string.
* AMO extensions will not change the UA string, per policy
* Rough third-party extension will set .override on every start, and
  will not freeze it (but we have less control than with .extra.*)

So, in practice, I can't see an advantage, apart from annoying users.
No, most cruft added to the UA string doesn't actually come from add-ons on AMO. Even if it did, extra.* simply shouldn't be provided in the first place. It has very few valid use cases that will be covered if we fix bug 581008 and this one.
> most cruft added to the UA string doesn't actually come from add-ons on AMO.

Good, but that's not the point of my comment. My point is that third-party applications can do *anything* a user can do, so as long as you allow users to change the string at all (which is a given), rough code (which you say is the main problem) can also do it, and can do it even easier than users can. So, you're trying to fix an unfixable problem. You can only attack it from the legal side (e.g. stop Microsoft from appending ".NET CLR").
No, I don't say rough code is the problem at all, because it's not rough by any means right now, nor illegal -- it uses the tool we provide, and shouldn't provide.
Right. So, whether they set .extra.* or set .override on every start doesn't make a difference.
Well it does make a difference. extra.* is a tool tailored for something we don't want. You put it in defaults.js and you're done. override is not tailored for this. Removing extra.* pushes them to make a decision, to either stop altering the UA string or jump through hoops and be sneaky. Some may decide to do the latter, but then we're in a better position to block them.
As to the annoyed at temporariness argument:
1) If the user is mucking around via about:config they should hopefully be simply knowledgeable enough to just be taught this is the new behavior.
2) I on at least one occasion have been annoyed at the fact that I had forgotten to reset an override after a needed spoof. This bug is intended to also fix simple mistakes like this that have unintended longer term consequences. If this is the first time a user used the override pref, probably just after looking it up, forgetting to reset or not realizing the need to reset is a problem. (though obviously lesser in severity than the overall mess we're trying to deal with)
3) I don't care if we annoy a few users slightly to make significant progress in fixing broken UA string problems.
So where does this cruft added to the UA string come from if it doesn't come from add-ons or from users?
(In reply to comment #24)
> So where does this cruft added to the UA string come from if it doesn't come
> from add-ons or from users?

It can come from either add-ons or users, or both, but wiping it out at app restart means that:
- it won't remain once the extension that put it there is disabled or uninstalled
- the user can set it, but if he forgets about it, it will go away at next startup
As I've already said in bug 581008 comment 95, I am against this suggestion because it makes it for careless add-on writers just harder but not impossible to change the user-agent string permanently (or at least as long as the extension is active).
I find it not acceptable to make this setting less useful for such a little gain.
Why do you think this would make the setting less useful?
(In reply to comment #26)
> just harder but not impossible

Nothing can be made impossible, so that's not a relevant point here.
I think that *if* we are going to make it temporary, we should remove the pref altogether and just make setting the override a method on nsIHttpProtocolHandler.
(In reply to comment #29)
> I think that *if* we are going to make it temporary, we should remove the pref
> altogether and just make setting the override a method on
> nsIHttpProtocolHandler.

That would be fine to allow an extension to do what it needs, but it would prevent the core use case of a user just quickly (temporarily) spoofing something using override for a site that won't let it in for no reason.

Now, if we also had a per-site override mechanism (using content prefs?) then just getting rid of general override entirely would probably be ok, though I'm sure you could still make a case where it could be needed or at least helpful.
I see no usecase for this...
If you want to override one site only for compat reasons, then override that site only, and permanently. It makes no sense to apply that override to all sites, potentially for days, and then suddenly have it gone just because you had to restart the browser. Who says you won't visit the site again? Then you have to set it again.
If you want to have a different useragent in general, for whatever reasons, like I do, then the temporariness also forces you to set it manually every time - not viable.
So, you need an extension either way. And extensions on trunk have always been problematic (and trunk users, even though not the only ones, are more likely to need this and understand what it does).
(In reply to comment #31)
> If you want to have a different useragent in general, for whatever reasons,
> like I do, then the temporariness also forces you to set it manually every time
> - not viable.

Or put it in user.js as mentioned multiple times before.
(In reply to comment #30)
> Now, if we also had a per-site override mechanism (using content prefs?)

Well, we can't do one right now, because that would need some way to change the window.navigator object in JS per-site and core code has no way for doing that and patches that would enable it are not going forward. See bug 526209 and its dependencies. Anyhow, that leads OT on this bug right here. I still think making .override temporary and forbidding add-ons on AMO to set .override without explicit user interaction is the only way out of the problems we're likely to run into when bug 581008 is being deployed.
Bug 589361 would subsume this. I suggest we close this out in favor of that (or morph it into removing general.useragent.override entirely).
I really don't think we should remove general.useragent.override entirely. It's a quick and easy for a user to do an override in an established way. I don't think we should force users to resort to installing an extension to do it unless they want more features. This is could be done in addition to bug 589361.
This never got pursued further and probably isn't important enough to do so at this point. There's also site-specific UA spoofing now which makes overriding the UA this way less needed. This could potentially be useful if there's ever some widely installed addon that breaks the UA via this pref, but that sort of thing might be better served via a blocklist entry and hotfix addon to clean up, unless it were to become a common practice.

No point to leaving this open; closing.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.