Closed Bug 1719070 Opened 3 months ago Closed 2 months ago

Add Nimbus client code for "Firefox/100.0" UA string experiment

Categories

(Core :: Networking: HTTP, task, P3)

task

Tracking

()

RESOLVED FIXED
92 Branch
Tracking Status
firefox-esr78 --- wontfix
firefox89 --- wontfix
firefox90 --- wontfix
firefox91 --- wontfix
firefox92 --- fixed

People

(Reporter: cpeterson, Assigned: cpeterson)

References

(Depends on 1 open bug, )

Details

Attachments

(2 files, 1 obsolete file)

We would like to run an experiment to test whether a UA string with a three-digit Firefox version number will break many sites. This new temporary general.useragent.experiment.firefoxVersion pref can override the UA string's Firefox version.

If many sites are broken, we might need to freeze the UA string's Firefox version at some two-digit number like "Firefox/99.0":

Researching User Agent Parts Freezing:
https://mozilla-hub.atlassian.net/browse/FFXP-239

FWIW, I've been browsing the web with a "Firefox/100.0" UA string for about four months and have only found one webcompat bug:

Slack's menu buttons don't work in Firefox version 100:
https://github.com/webcompat/web-bugs/issues/67866

I will be working with the Normandy experiment team to launch this experiment during the upcoming Nightly 92 cycle.

An alternative to adding this new temporary pref is to have the experiment set the general.useragent.override pref. The disadvantage of setting using the override pref is that we would not be able to set platform-appropriate UA strings for each platform. We would either need to restrict our experiment to just Windows users, set a Windows UA string on macOS and Linux, or run three different experiments (one each for Windows, macOS, and Linux).

We would like to run an experiment to test whether a UA string with a three-digit Firefox version number will break many sites. This new temporary general.useragent.experiment.firefoxVersion pref can override the UA string's Firefox version. If many sites are broken, we might need to freeze the UA string's Firefox version at some two-digit number like "Firefox/99.0".

Can you tell me more about the experiment? Are you planning to watch some telemetry probes and compare them between control and study branch or you want to rely on bug and webcompat reports?

Flags: needinfo?(cpeterson)

(In reply to Dragana Damjanovic [:dragana] from comment #4)

Can you tell me more about the experiment? Are you planning to watch some telemetry probes and compare them between control and study branch or you want to rely on bug and webcompat reports?

Good questions. I haven't written the experiment plan yet, but the general idea is:

  1. Launch a Nightly experiment with the treatment branch using a UA string the pretends to have a three-digit version number like Mozilla/5.0 (Windows NT 10.0; rv:100.0) Gecko/20100101 Firefox/100.0
  2. Hope that users in the "Firefox 100" treatment branch will file bugs (in Bugzilla or webcompat.com) for any broken sites.
  3. Compare user-engagement telemetry (such as days of active users, number of active hours, and number of pages loaded) to see if the Firefox 100 users browse any less (presumably because some sites are broken for them).

We are primarily hoping for bug reports. If the user-engagement metrics look bad, we will know there is a problem but have no actionable information to debug. We probably would not run this experiment on the Beta or Release channels. If there are too many broken sites or problems we can't fix, our backup plan is to freeze the UA string's Firefox version at some two-digit version number like Firefox/99.0.

If you think there is a more effective or less disruptive way to test for these broken sites, just let me know.

Firefox Nightly will hit version 100 in March 2022.

Flags: needinfo?(cpeterson) → needinfo?(dd.mozilla)
Attachment #9229687 - Attachment is obsolete: true
Attachment #9229686 - Attachment is obsolete: true
Depends on: 1722265
Summary: Add temporary pref for "Firefox/100.0" UA string experiment → Add Nimbus client code for "Firefox/100.0" UA string experiment
Attachment #9229687 - Attachment is obsolete: false
Attachment #9229687 - Attachment description: Bug 1719070 - Merge some unnecessary add_tasks and functions. r?#necko-reviewers → Bug 1719070 - UA: Merge some unnecessary add_tasks and functions in User-Agent string tests. r?#necko-reviewers

Here is a link to the first draft of the "Firefox 100 User-Agent Experiment" plan:

https://mana.mozilla.org/wiki/display/FIREFOX/Firefox+100+User-Agent+Experiment

Pushed by cpeterson@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/abf70d233377
UA: Merge some unnecessary add_tasks and functions in User-Agent string tests. r=necko-reviewers,dragana
https://hg.mozilla.org/integration/autoland/rev/ab01c8a94b77
UA: Add support for a Nimbus experiment to override Firefox's User-Agent string. r=andreio,necko-reviewers,dragana

Backed out for causing bc failures on browser_navigator.js.

Push where the failure appeared
Push where tier 1 failures appeared

Failure log

Backout link

Flags: needinfo?(cpeterson)
Pushed by cpeterson@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0d251cd32e26
UA: Merge some unnecessary add_tasks and functions in User-Agent string tests. r=necko-reviewers,dragana
https://hg.mozilla.org/integration/autoland/rev/56e4a835369f
UA: Add support for a Nimbus experiment to override Firefox's User-Agent string. r=andreio,necko-reviewers,dragana

(In reply to Iulian Moraru from comment #9)

Backed out for causing bc failures on browser_navigator.js.

Thanks. I fixed the test failure on 32-bit Windows 7 and relanded. Resolving as fixed.

Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Flags: needinfo?(dd.mozilla)
Flags: needinfo?(cpeterson)
Resolution: --- → FIXED
Target Milestone: --- → 92 Branch
Regressions: 1724635
Depends on: 1731523
You need to log in before you can comment on or make changes to this bug.