1719070 - Add Nimbus client code for "Firefox/100.0" UA string experiment

Status: RESOLVED FIXED

(Core :: Networking: HTTP, task, P3)

Description

[Chris Peterson [:cpeterson]]

We would like to run an experiment to test whether a UA string with a three-digit Firefox version number will break many sites. This new temporary general.useragent.experiment.firefoxVersion pref can override the UA string's Firefox version.

If many sites are broken, we might need to freeze the UA string's Firefox version at some two-digit number like "Firefox/99.0":

Researching User Agent Parts Freezing:
https://mozilla-hub.atlassian.net/browse/FFXP-239

FWIW, I've been browsing the web with a "Firefox/100.0" UA string for about four months and have only found one webcompat bug:

Slack's menu buttons don't work in Firefox version 100:
https://github.com/webcompat/web-bugs/issues/67866

Comment 1

[Chris Peterson [:cpeterson]]

I will be working with the Normandy experiment team to launch this experiment during the upcoming Nightly 92 cycle.

An alternative to adding this new temporary pref is to have the experiment set the general.useragent.override pref. The disadvantage of setting using the override pref is that we would not be able to set platform-appropriate UA strings for each platform. We would either need to restrict our experiment to just Windows users, set a Windows UA string on macOS and Linux, or run three different experiments (one each for Windows, macOS, and Linux).

Comment 2

[Chris Peterson [:cpeterson]]

Attachment: Bug 1719070 - Add temporary pref for "Firefox/100.0" UA string experiment. r?hsivonen!,#necko-reviewers

We would like to run an experiment to test whether a UA string with a three-digit Firefox version number will break many sites. This new temporary general.useragent.experiment.firefoxVersion pref can override the UA string's Firefox version. If many sites are broken, we might need to freeze the UA string's Firefox version at some two-digit number like "Firefox/99.0".

Comment 3

[Chris Peterson [:cpeterson]]

Attachment: Bug 1719070 - UA: Merge some unnecessary add_tasks and functions in User-Agent string tests. r?#necko-reviewers

Depends on D119053

Comment 4

[Dragana Damjanovic [:dragana]]

Can you tell me more about the experiment? Are you planning to watch some telemetry probes and compare them between control and study branch or you want to rely on bug and webcompat reports?

Comment 5

[Chris Peterson [:cpeterson]]

(In reply to Dragana Damjanovic [:dragana] from comment #4)

Can you tell me more about the experiment? Are you planning to watch some telemetry probes and compare them between control and study branch or you want to rely on bug and webcompat reports?

Good questions. I haven't written the experiment plan yet, but the general idea is:

1. Launch a Nightly experiment with the treatment branch using a UA string the pretends to have a three-digit version number like Mozilla/5.0 (Windows NT 10.0; rv:100.0) Gecko/20100101 Firefox/100.0
2. Hope that users in the "Firefox 100" treatment branch will file bugs (in Bugzilla or webcompat.com) for any broken sites.
3. Compare user-engagement telemetry (such as days of active users, number of active hours, and number of pages loaded) to see if the Firefox 100 users browse any less (presumably because some sites are broken for them).

We are primarily hoping for bug reports. If the user-engagement metrics look bad, we will know there is a problem but have no actionable information to debug. We probably would not run this experiment on the Beta or Release channels. If there are too many broken sites or problems we can't fix, our backup plan is to freeze the UA string's Firefox version at some two-digit version number like Firefox/99.0.

If you think there is a more effective or less disruptive way to test for these broken sites, just let me know.

Firefox Nightly will hit version 100 in March 2022.

Comment 6

[Chris Peterson [:cpeterson]]

Attachment: Bug 1719070 - UA: Add support for a Nimbus experiment to override Firefox's User-Agent string. r?#necko-reviewers!,andreio

Depends on D119054

Comment 7

[Chris Peterson [:cpeterson]]

Here is a link to the first draft of the "Firefox 100 User-Agent Experiment" plan:

https://mana.mozilla.org/wiki/display/FIREFOX/Firefox+100+User-Agent+Experiment

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/abf70d233377
UA: Merge some unnecessary add_tasks and functions in User-Agent string tests. r=necko-reviewers,dragana
https://hg.mozilla.org/integration/autoland/rev/ab01c8a94b77
UA: Add support for a Nimbus experiment to override Firefox's User-Agent string. r=andreio,necko-reviewers,dragana

Comment 9

[Iulian Moraru]

Backed out for causing bc failures on browser_navigator.js.

Push where the failure appeared
Push where tier 1 failures appeared

Failure log

Backout link

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0d251cd32e26
UA: Merge some unnecessary add_tasks and functions in User-Agent string tests. r=necko-reviewers,dragana
https://hg.mozilla.org/integration/autoland/rev/56e4a835369f
UA: Add support for a Nimbus experiment to override Firefox's User-Agent string. r=andreio,necko-reviewers,dragana

Comment 11

[Chris Peterson [:cpeterson]]

(In reply to Iulian Moraru from comment #9)

Backed out for causing bc failures on browser_navigator.js.

Thanks. I fixed the test failure on 32-bit Windows 7 and relanded. Resolving as fixed.

Comment 12

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/0d251cd32e26
https://hg.mozilla.org/mozilla-central/rev/56e4a835369f