1769022 - RFP: userAgent.AppendInt(spoofedVersion) !== 102

Status: RESOLVED FIXED

(Core :: DOM: Security, defect, P3)

Description

[Thorin [:thorin]]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

Desktop, Android FF102

• enable RFP
• check userAgent

Returns 91, should return 102 (but thinks 104 is the next one)

The code in GetSpoofedVersion() could probably do with a clean up. Why do we still need to reference 78? And we really do a better way to check for the next ESR version. I think after 102, the next is slated to be 114 (as mentioned in a Thunderbird post, but of course that probably hasn't been decided and is hearsay)

[1] https://hg.mozilla.org/mozilla-central/rev/b9c628ba6b0a#l1.42
[2] https://searchfox.org/mozilla-central/source/toolkit/components/resistfingerprinting/nsRFPService.cpp#586

Comment 1

[Thorin [:thorin]]

pinging chris

Comment 2

[Thorin [:thorin]]

we really do [need] a better way to check for the next ESR version

I stumbled on this by mistake, and while I'm sure someone would have found it before 102 hit stable, it's not ideal, and leaves us scrambling. Surely we can do better (although we don't know for sure the next version). I thought Ehsan mentioned a solution somewhere, but IDK where he's at.

Can we set a hard coded next value to only be checked (and throw an alert) when we're current spoofed version + 10: e.g.

• current spoof = 102
• hard coded next spoof = 102 (because we don't know the next one)
• when 112 rolls out, if current = next throw an alert

Comment 3

[Thorin [:thorin]]

Attachment: esrcheck.png

https://arkenfox.github.io/TZP/tzp.html#useragent

Comment 4

[Chris Peterson [:cpeterson]]

I'll take a look at this bug. We can update the hard-coded ESR version and the "guess next ESR version" calculation. That code is hacky, but it at least gives us a chance that we might not need to fix this function every 12 months when a new ESR version is released.

Maybe we don't even need to spoof the ESR version? How much fingerprinting benefit to people using regular Firefox with resistFingerprinting gain from using the most recent ESR version instead of their real Firefox version? If privacy and security protections are important to them, they are probably using a recent Firefox version, which would have more users (to hide among) than ESR. That's not true for the smaller number of people using Firefox Nightly or Beta, but we could make builds from those channels spoof the current Release channel version (Nightly version - 2 and Beta version - 1).

Comment 5

[Thorin [:thorin]]

Maybe we don't even need to spoof the ESR version

We should bring Richard in from tor project. There are a couple of RFP UA things that do not make sense in the FF world vs the Tor Browser world [1], and there was an email discussion about putting some RFP changes behind another pref, e.g. privacy.resistFingerprinting.tb.enabled, but that would require divergent code

IMO I believe version spoofing and the HTTP header limited to two OSes (we should match JS and return four) could be dropped for FF. This solves

• "feature detection" on sites that use ua parsing - such as AMO where Multi-AccountContainers requires v95 or something. It's a footgun we don't need. Could have sworn we still had an open ticket about how AMO gets the version + RFP but I can't find it
• HTTP headers example: bug 1610762, don't be fooled by the title (they may have been the case in the past and just reinforces the version spoofing issue), it is actually mismatched HTTP header vs JS for platform details, and affects more than just Netflix, such as DIsney+, Soundcloud

[1] TB

• doesn't care about Netflix (and DRM is disabled) and are more concerned with giving away easy OS info in headers for server logs. And of course it makes sense to limit to two when in Safest mode (albeit not 100% foolproof)
• desktop is based on ESR, but there was talk of moving to a rapid release cycle, so this may not always be the case
• android, there is no ESR, so it makes no sense, except that they probably want to reduce some entropy somehow - ESR versioning makes no sense in the GV world, IMO

Comment 6

[Thorin [:thorin]]

How much fingerprinting benefit to people using regular Firefox with resistFingerprinting gain

ultimately none, and Tor Browser is the in the same boat - https://arkenfox.github.io/TZP/tests/versions.html - the question is, do we want to make it harder for scripts, or just accept it in the FF world (TB world may have a different viewpoint)

Comment 7

[Thorin [:thorin]]

also see https://bugzilla.mozilla.org/show_bug.cgi?id=1529391#c1 for a previous discussion

but we could make builds from those channels spoof the current Release channel version (Nightly version - 2 and Beta version - 1).

that was a no in the past: see https://bugzilla.mozilla.org/show_bug.cgi?id=1466027#c5

concern about not testing code paths before they make it to Release

but if we're not ESR spoofing then that particular point would be moot

Comment 8

[Chris Peterson [:cpeterson]]

Also, if RFP users using non-ESR versions of Firefox report the current Release channel version (Nightly version - 2 and Beta version - 1), they might run into fewer webcompat issues because they are reporting a version number that more closely matches their build's real version number (and its web capabilities and bug fixes) instead of an ESR version that's up to 12 months behind their build.

Comment 9

[Thorin [:thorin]]

(In reply to Simon Mainey from comment #5)

IMO I believe version spoofing and the HTTP header limited to two OSes (we should match JS and return four) could be dropped for FF. This solves ...

ESR102 would be a nice flag day. Awaiting Richard's reply, but we can address that in bug 1610762

Comment 10

[Chris Peterson [:cpeterson]]

Tom, do you have an opinion on making RFP report Firefox's real version instead of spoofing the most-recent ESR version?

RFP's "predict the next ESR version" code is fragile and bugs in its calculation are only caught if someone runs a debug ESR build after a new ESR is branched. (AFAIK, Mozilla doesn't run tests on ESR builds.) If RFP reported Firefox's real version, we could avoid having to find and fix version spoofing bugs like this every year or two. Nightly, Beta, Release, and ESR would all use the same non-spoofing code paths and return their real versions.

You closed bug 1466027 (make Nightly and Beta spoof the Release channel version) as wontfix because Nightly and Beta are meant to test code before hits Release and attackers can still detecting Nightly and Beta's real versions. But that bug didn't resolve the question of whether RFP should report non-TB Firefox's actual version in Nightly, Beta, and Release. There are more Release channel users than ESR users, so this would benefit Firefox RFP users (on the Release channel, but reduce effectiveness on Nightly or Beta) without impacting TB users.

Comment 11

[Chris Peterson [:cpeterson]]

Nightly 102 will merge to Beta in two weeks (2022-05-30). If we don't come to a conclusion in this bug in a day or two, I'll just post a patch to fix the current ESR version spoofing for ESR 102.

Comment 12

[Daniel Veditz [:dveditz]]

Assigning Chris as the caretaker assignee at this point.

Comment 13

[richard (Tor Project)]

See inline:

(In reply to Simon Mainey from comment #5)

[1] TB

• doesn't care about Netflix (and DRM is disabled) and are more concerned with giving away easy OS info in headers for server logs. And of course it makes sense to limit to two when in Safest mode (albeit not 100% foolproof)

This is my understanding as well (the decision on this is from before my time w/ Tor), we keep js-reported UA strings reporting the correct OS in order to avoid breakage for users who are are expecting a more traditional browser experience, while unifying them in HTTP headers to allow users w/o JS enabled (eg in Safest) to be less fingerprintable.

• desktop is based on ESR, but there was talk of moving to a rapid release cycle, so this may not always be the case

There are no plans to move to rapid-release anytime soon for desktop.

• android, there is no ESR, so it makes no sense, except that they probably want to reduce some entropy somehow - ESR versioning makes no sense in the GV world, IMO

If the code is rather brittle, we could always optionally make the reported esr version a compile-time constant.

Comment 14

[Thorin [:thorin]]

TB world: I think desktop could drop the version spoofing, but android perhaps round down to multiples of 3 or 4 (just to create larger buckets as people update) if you wanted to make it harder for scripts, but ultimately it's futile (and manifests in other metrics)

If we could agree that version spoofing is not worth it for everyone (TB, TBA, FF-everywhere), then rip it out. Do we need more input from gk, tjr, sysrqb ?

Comment 15

[Tom Ritter [:tjr] (OOTO until mid-August)]

Tor Desktop should report the first ESR minor release. That is to say 91.0, 102.0, etc. We can't remove version spoofing for Tor/ESR entirely because we would report minor releases, e.g. 91.1, 102.1.1. Because Tor Desktop major version is always ESR, the spoofing doesn't need to calculate an ESR version, it can use the real major version.

Tor Android - JS feature detection is a thing and can narrow down the major version, but it's not perfect. It should definitely round to at least major versions. Maybe it should round to ESR, I defer to Richard. It seems like he wanted to keep it at ESR though...

For FF, IMO there is some value in spoofing to ESR version (feature detection is a thing but it's not perfect) but at a minimum we should round to major version. I am close-ish to kind-of fixing Bug 1610762 via RFP Exceptions on the UA header in Bug 1737829 but still those prefs are experimental.

On the whole it seems like the plan is either:

1. Make the code use the real major version and spoof the minor version to .0.0 in all situations
2. Do (1) on Desktop and on Android hardcode the Spoof Version to 102 (compile-time constant)
3. Hardcode the Spoof Version to 102 (compile-time constant) on all platforms

I lean toward (3) which maintains current behavior, but I think (2) is okay as well. To answer your question, I think we cannot eliminate the need to update the code every so often as we don't want to change behavior for Tor for Android (unless I misunderstood Richard) so doing (1) for Firefox but not Tor doesn't avoid that problem for us.

Comment 16

[Thorin [:thorin]]

Attachment: esrver.png

(In reply to Tom Ritter [:tjr] (ni for sec-approvals and such) from comment #15)

Tor Desktop should report the first ESR minor release. That is to say 91.0, 102.0, etc. We can't remove version spoofing for Tor/ESR entirely because we would report minor releases, e.g. 91.1, 102.1.1. Because Tor Desktop major version is always ESR, the spoofing doesn't need to calculate an ESR version, it can use the real major version.

Are you sure about that? Pretty sure this was covered in all channels for all users when chris did entropy cleanup about 2 years ago

Unless I'm missing something, 1. is already resolved, regardless of RFP/ESR

For FF, IMO there is some value in spoofing to ESR version (feature detection is a thing but it's not perfect)

you said that not perfect bit twice :) I'll refer to comment 6 which is cross platform, and can't be tampered with by prefs, and I've never seen it fail (although I can't claim it will never have an edge case/config), and it's very fast - code starts here, as you can see almost all them don't even touch the dom or are accessible to extensions

additionally, these sort of items in the test, and many others behind prefs which most users will never change, manifest automatically in JS attack type templates, such as enumerating

• `Object.getOwnPropertyNames() such as an iframe.contentWindow = 800+ properties
• Object.getPrototypeOf(cssStyleDeclaration) - 1000+ properties
• etc, all of which constantly change

But, yes: not everyone is going to do that. So I agree there is a tiny bit of value (remembering that randomized canvas already nails naive scripts), but I do not think it is worth it vs the compat gains/footguns/troubleshooting. And it doesn't meet TB's threat model: we're protecting against advanced scripts ultimately, and this is child's play to work around (don't get me wrong, making it harder for scripts is not a bad thing)

I am close-ish to kind-of fixing Bug 1610762 via RFP Exceptions on the UA header in Bug 1737829 but still those prefs are experimental.

For Firefox users, being able to add a site exception down the track solves that, but only after the user has had to troubleshoot. And giving up all the other RFP protections on those site exceptions for something so minor that we can't hide, seems counterintuitive.

I think, and I don't want to labor the point too many times (sorry), for at least Firefox users on all platforms, we can remove version spoofing

TB is up to Richard - if you want to do something for android then you might as well leave in the desktop part should desktop ever change to rapid release - so you're going to want a flag or pref or as Richard said "If the code is rather brittle, we could always optionally make the reported esr version a compile-time constant"

And the HTTP header OS mismatch we can deal with elsewhere

Comment 17

[Chris Peterson [:cpeterson]]

For now, I will submit a patch that just updates the version spoofing to use the new ESR version 102. If someone feels strongly about reporting the latest major version instead of spoofing the ESR version, we can open a new bug.

That said, I think reporting the real major version ("91.0" which ESR and dot releases' UA strings already do, not "91.9.1") would:

• Improve RFP's effectiveness (and webcompat) for Firefox Desktop and Tor Android users (because there are more Release channel users than ESR users).
• Be neutral for Tor Desktop (because its real major version is the ESR version).
• But reduce RFP's effectiveness for Firefox Nightly and Beta users (because there are fewer of them than ESR users).

Firefox Android has no ESR version. The Google Play Store auto-updates users to new versions within just a few days after release, so Tor Android spoofing a version older than the latest would greatly reduce RFP's effectiveness.

Comment 18

[Thorin [:thorin]]

oh well, not going to make the same points we all seem to agree on and know repeatedly in new issues (and email), so i guess nothing will ever happen and we'll stay on another cycle of yearly tweaks with compat issues for FF users (who we should be grateful they test this, I know, it's not front facing) .. missing an opportunity here guys :)

As for ESR version, just hardcode 102 and ignore the future, I'm sure I or someone can warn about the next one. We haven't had a single cadence in step yet: 60-68-78-91-102-x place your bets :)

Comment 19

[Viktor Jägersküpper]

(In reply to Chris Peterson [:cpeterson] from comment #17)

If someone feels strongly about reporting the latest major version instead of spoofing the ESR version, we can open a new bug.

This is why I filed bug 1529391 in the first place, so we could just reopen it. If I understand the issue correctly, spoofing the ESR version is mainly protecting ESR users from simple fingerprinting techniques (because release users with RFP enabled also report the ESR version which makes the group with the ESR version larger), although ESR users (implicitly) accept that they are not getting the latest privacy features (including RFP). Are other privacy enhancements (e.g. RFP improvements) even backported to ESR or is that only done in Tor Browser as additional patches? If the former doesn't happen, I don't understand why release users should be the ones that might run into problems because of the version spoofing.

And I agree with Simon that we are wasting time for this every year.

Comment 20

[Thorin [:thorin]]

I don't want keep adding noise - I agree, reopen bug 1529391 whoever has the ability to do so - link to this ticket with all the info/pros/cons/ideas, and it revisit/discuss it there. PS: the argument for non-ESR users making the ESR group look bigger is moot: see the point about not fooling anyone (also IDK, but I would suspect that the ESR group is already rather large, more than TB users - not that it's a competition)

Comment 21

[Chris Peterson [:cpeterson]]

(In reply to Tom Ritter [:tjr] (sec-approvals paused until after merge) from comment #15)

On the whole it seems like the plan is either:

1. Make the code use the real major version and spoof the minor version to .0.0 in all situations
2. Do (1) on Desktop and on Android hardcode the Spoof Version to 102 (compile-time constant)
3. Hardcode the Spoof Version to 102 (compile-time constant) on all platforms

I lean toward (3) which maintains current behavior, but I think (2) is okay as well. To answer your question, I think we cannot eliminate the need to update the code every so often as we don't want to change behavior for Tor for Android (unless I misunderstood Richard) so doing (1) for Firefox but not Tor doesn't avoid that problem for us.

Rereading the comments here, I see that Tom would accept option (2) using the real major version on Desktop, so I will submit a patch to implement (2).

(In reply to Tom Ritter [:tjr] (sec-approvals paused until after merge) from comment #15)

Tor Android - JS feature detection is a thing and can narrow down the major version, but it's not perfect. It should definitely round to at least major versions. Maybe it should round to ESR, I defer to Richard. It seems like he wanted to keep it at ESR though...

Richard, do you want Tor Android to return its real major version or Desktop's ESR version? I think hardcoding version 102 for Tor Android is a bad idea. There is no Android ESR, so the few Firefox Android users who will have a version 102 UA after Mozilla releases Firefox Android version 103 in July will be the Tor Android users. My patch here returns Desktop's ESR version (102) to match Tor Android's current behavior, but I'm happy to submit a new patch.

Comment 22

[Thorin [:thorin]]

Richard

Is there a consideration here that spoofing on android for TB helps mitigate any lag/delay in users updating, and from multiple sources, to help keep a larger bucket with this spoof? - I think the guardian project's fdroid is particularly bad, but IANAE on android updates and TBA sources. That said, it's a long time between ESR flag days (a year)

If you wanted to future-proof the code... here's my idea

• an ESR version itself doesn't signify anything (it's not matching desktop TB, the userAgent strings are clearly different), so we could start at any number ... but, if Richard wants a spoofing cadence, then how about twice a year: note that 102 is a multiple of 6, which makes it simpler so just round down the real version to a multiples of 6, and never think about it again

Comment 23

[Chris Peterson [:cpeterson]]

Attachment: Bug 1769022 - resistFingerprinting: report Firefox's real major version instead of the ESR version. r?tjr

For RFP Firefox, report the real major version because there are more Firefox Release channel users than ESR users and, by returning the real major version, we won't have to update kKnownEsrVersion for future ESR versions. The Tor Desktop is based on ESR, so its real major version is the ESR major version.

Special case Android to return Desktop's ESR major version ("102.0"). Firefox Android has no ESR, but this matches Tor Android's current behavior.

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/89e459d1b6cc
resistFingerprinting: report Firefox's real major version instead of the ESR version. r=tjr

Comment 25

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/89e459d1b6cc