1499478 - navigator.userAgent violates HTML specification when RFP enabled

Status: RESOLVED INVALID

(Core :: DOM: Security, defect, P3)

Description

[Moony Owl]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0

Steps to reproduce:

1. Enable ResistFingerprinting
2. Compare User-Agent HTTP header with navigator.userAgent value


Actual results:

navigator.userAgent is not equal to User-Agent HTTP header


Expected results:

navigator.userAgent must return the default `User-Agent` value.
A default `User-Agent` value is a user-agent-defined value for the `User-Agent` header.
https://html.spec.whatwg.org/multipage/system-state.html#dom-navigator-useragent
https://fetch.spec.whatwg.org/#default-user-agent-value
TL;DR: navigator.userAgent _must_ be equal to User-Agent HTTP header.

Comment 1

[Moony Owl]

The problem is caused by some changes that came from #1404608 and #1409269.
Originally, reporters of these bugs asked the following:
>There should be an override that allows the spoofed useragent to be changed to a useragent that is based on Linux 3.11 and newer.
>but would it be possible to choose another, very common user agent on mobile so that it doesn't load desktop pages

Instead, we got the following:
>There is no way to spoof OS in navigator.userAgent property.
>HTTP User-Agent string differs from  navigator.userAgent property.

Comment 2

[laszlo.bialis]

I am assigning a component to this issue, based on the two mentioned bugs, in order to involve the development team and get an opinion on this.

Comment 3

[Tim Huang[:timhuang]]

Thanks for reporting this. 

I have tested for Windows and Mac OS and I didn't see the same issue here. The navigator.userAgent and User-Agent HTTP header both show the same value. And according to [1], the value of navigator.userAgent is coming from the same source of the `User-Agent` header. So, they should be the same. 

Could you provide more details about this? Thanks. 

[1] https://searchfox.org/mozilla-central/rev/9cb3e241502a2d47e2d5057ca771324a446b6695/dom/base/Navigator.cpp#1660-1695

Comment 4

[Moony Owl]

(In reply to Tim Huang[:timhuang] from comment #3)
> Thanks for reporting this. 
> 
> I have tested for Windows and Mac OS and I didn't see the same issue here.
> The navigator.userAgent and User-Agent HTTP header both show the same value.
Try to test it on x64 Windows. On my Windows 7 x64 I got the following:

navigator.userAgent: "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0"
HTTP User-Agent:  "Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0"

Comment 5

[Daniel Veditz [:dveditz]]

Tom: not sure how important this is to the Tor/RFP effort

Comment 6

[Tom Ritter [:tjr] (OOTO until mid-August)]

I am unsure of the current behavior; the code - when I last looked at it a month ago - sure seemed to use the codepaths. The values reported in Comment 4 obviously differ.

What I can say for certain is that Mozilla intends to upstream a patch from Tor that will definitively violate the spec. We intend to match Tor's behavior of reporting one system via the HTTP Header no matter what platform the user is on; but report the correct platform in the navigator object in Javascript.  This is Tor #26146 and https://gitweb.torproject.org/tor-browser.git/patch/?id=511d1d8

So we could use this bug to resolve this issue; but it's more likely we will wontfix this in favor of a new bug dealing strictly with the behavior we intend to enable.

Comment 7

[Moony Owl]

>We intend to match Tor's behavior of reporting one system via the HTTP Header no matter what platform the user is on; but report the correct platform in the navigator object in Javascript.
Mozilla going to implement disclosure of real platform (in RFP mode) without giving any choice to opt-out from it.
1. Is that reading of your message correct?
2. Do you really think that users want such anti-privacy features to be implemented?

Comment 8

[Thorin [:thorin]]

(In reply to Moony Owl from comment #4)
> Try to test it on x64 Windows. On my Windows 7 x64 I got the following:
> 
> navigator.userAgent: "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0"
> HTTP User-Agent:  "Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0"

On my x64 Windows 7 I get the same values (FF63) [1]
HTTP_USER_AGENT
- Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0
userAgent
- Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0

[1] https://whoer.net/ - click on Extended Version, scroll down to the near bottom

Comment 9

[Thorin [:thorin]]

(In reply to Moony Owl from comment #7)
> Mozilla going to implement disclosure of real platform (in RFP mode)

RFP already limits both (HTTP/JS) to 4 OSes. The proposal, or TBB patch, was to further limit the HTTP_USER-AGENT to 2 OSes. This would provide less entropy for Mac/Linux users on sites that do not gather the OS via JS, but allow for less breakage on sites that do use it (under the current RFP setup, they are already giving this info away). Maybe Tom can explain it better.

Comment 10

[Tom Ritter [:tjr] (OOTO until mid-August)]

(In reply to Moony Owl from comment #7)
> >We intend to match Tor's behavior of reporting one system via the HTTP Header no matter what platform the user is on; but report the correct platform in the navigator object in Javascript.
> Mozilla going to implement disclosure of real platform (in RFP mode) without
> giving any choice to opt-out from it.
> 1. Is that reading of your message correct?

Yes.

> 2. Do you really think that users want such anti-privacy features to be
> implemented?

Well, there's a few ways to respond to this.

Firstly: We... already do that. We currently report your real OS in RFP Mode. The change I'm talking about will actually report your real OS _less_ than we currently do.

It's import to note though, that Firefox does not and can not rigorously protect your Operating System even if we did lie in the User Agent field. We leak it through your OS networking stack (which we will never be able to prevent unless you are sending Firefox through a proxy), through system UI rendering, through WebGL information. Probably others too.

Finally, Resist Fingerprinting mode is not a feature of Firefox we support for users, it is under-the-covers changes we make for Tor. I'm happy to have Firefox users enjoy and use the mode, and report issues - but at this point in time our primary service is to Tor. Tor decided on this course of action, and as part of the Tor Uplift project we intend to incorporate their changes into Firefox to make their development process simpler.  Tor Browser is able to mitigate those above OS leaks in a way that Firefox cannot. (Although even their mitigations are imperfect.)  Until Firefox incorporates RFP as a supported feature, we do intend to make RFP do what Tor wants.



But like I said; I do appreciate the bug report and details, the behavior you are seeing does seem to be a bug in the current implementation, and your bug report will help make RFP, Tor Browser, and what I hope will be a future full-fledged Firefox feature better!

Comment 11

[Moony Owl]

(In reply to Tom Ritter [:tjr] from comment #10)
>Firefox does not and can not rigorously protect your Operating System even if we did lie in the User Agent field
It's not a reason to do nothing to protect it.

>We leak it through your OS networking stack (which we will never be able to prevent unless you are sending Firefox through a proxy), through system UI rendering, through WebGL information. Probably others too.
1. WebGL, Canvas, WebAudio, Network fingerprinting and some others are not actual for Tor Browser.
2. Many websites don't use such methods, they just rely on navigator.userAgent. Previously they were not be able to detect real OS, but now they can do it. And users can't do something about it because general.useragent.override no longer works in RFP mode.

>Firstly: We... already do that. We currently report your real OS in RFP Mode.
Since they (I mean Tor Browser developers) introduced this feature, they_must_ provide a way to disable it. Otherwise it looks like an intentional action against all those users who want to protect their privacy.
Users are free to decide what to report about their browser / OS versions. Users didn't told Tor Browser devs to decide it for them.

>but at this point in time our primary service is to Tor
Okay, I'll try to tell the above to Tor Browser devs.

Comment 12

[Moony Owl]

(In reply to Simon Mainey from comment #8)
> On my x64 Windows 7 I get the same values (FF63) [1]
> HTTP_USER_AGENT
> - Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101
> Firefox/60.0
> userAgent
> - Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101
> Firefox/60.0
> 
> [1] https://whoer.net/ - click on Extended Version, scroll down to the near
> bottom
Probably, it's fixed on FF63.
comment #4 is actual for FF60 ESR and Tor Browser.

Comment 13

[Tom Ritter [:tjr] (OOTO until mid-August)]

We talked through this today again, specifically if we want to allow the user to override the User Agent when RFP is turned on. We previously considered this in Bug 1489903. We thought about it from the angle of "Well, if a user wants to use these prefs to make RFP _more_ restrictive is that enough of a reason to allow it to support it?"

While the notion was pretty sympathetic, ultimately we were too worried about the other direction. Users who have set the ovderride prefs and forgotten about it, or set it basically to anything other than a single very carefully chosen value will then become extremely fingerprintable, even uniquely identifiable when Resist Fingerprinting is enabled. For these users (who we think would be the majority of users with the override pref set) we have actually make Resist Fingerprinting Mode more dangerous and created a Highly Fingerprintable Mode. So we haven't changed from our previous decision in 1489903 to keep RFP overrideing the user agent override prefs.

Given that, and that the original issue in this bug seems to be resolved, I'm going to close this.