1660057 - Add new Fission experiment prefs

Status: VERIFIED FIXED

(Core :: DOM: Content Processes, task, P2)

Description

[Chris Peterson [:cpeterson]]

We want to hide the "Open Non-Fission Window" File menu item (added in bug 1594227) when Fission is started by an experiment so users in the experiment don't know they have Fission enabled. We don't need to bother removing the [F] suffix from Fission tab tooltips because it is pretty hidden.

We will probably need to add a new Fission start pref to be remote flipped by experiment. Having a separate pref will also allow the Normandy experiment server to enroll existing "fission.autostart = true" users into our Fission experiment.

We should audit (bug 1609791) code that reads the fission.autostart pref directly to make sure it works with the new Fission experiment pref. Most code should check whether the relevant nsILoadContext has the useRemoteSubframes flag set instead of reading the fission.autostart pref directly. We will need to make sure the useRemoteSubframes implementation honors both the fission.autostart pref and new experiment pref.

Comment 1

[Chris Peterson [:cpeterson]]

We've decided we don't need to bother removing the "Open Non-Fission Window" File menu item (or [F] suffix from Fission tab tooltips) for our Fission Nightly experiment. These UI elements are only enabled in Nightly, so they won't be a problem for our Beta or Release experiments.

We do still want a new Fission experiment pref so remote pref flips don't stomp on the user's fission.autostart pref opt-in.

Comment 2

[Chris Peterson [:cpeterson]]

WONTFIX because:

1. Showing Fission door hanger to users who have manually opted into Fission is not a problem.
2. We want to exclude users who have already opted into Fission from the experiment. They are biased users and we don't want to disable Fission for them if they are assigned to the experiment's control branch.

Comment 3

[Chris Peterson [:cpeterson]]

Reopening because we do need new experiment prefs.

After talking with the Normandy experiments team, I think we need three Fission prefs:

1. The fission.autostart pref set by the user (and the Fission checkbox in about:preferences#experimental).
2. fission.experiment.enrollmentStatus: A new experiment enrollment pref set asynchronously by Normandy client code.
3. fission.experiment.startupEnrollmentStatus: A new experiment pref to persist the Normandy enrollment status at shutdown by Fission code (like WebRender bug 1503776).

fission.autostart is a boolean, but the new experiment prefs 2 and 3 should be numbers (enums) because we need to differentiate three different experiment states:

• State 0: UNKNOWN. Default value: client doesn't know whether the user is enrolled in the experiment or not. Fission should be disabled.
• State 1: ENROLLED_CONTROL. User is enrolled in the experiment, but they are in the control branch so Fission should still be disabled.
• State 2: ENROLLED_TREATMENT. User is enrolled in the experiment and they are in the treatment branch so Fission should be enabled (when Firefox next launches).

The proposal is:

• When Firefox starts up, Fission mode will activate if either fission.autostart (pref 1) or persisted pref 3 are true. fission.autostart will only be toggled by the user messing in about:config or about:preferences, not by code.
• Persisted ref 3 == true means that, in the previous browser session, Normandy detected the user is in the Fission experiment.
• Perf 2 is toggled remotely by Normandy code and copied during browser shutdown into pref 3.

Constraints we are working around:

• We want to exclude existing fission.autostart == true users, but we can't both exclude fission.autostart == true users from our experiment and enable the experiment using the same fission.autostart pref.
  • If Normandy sets fission.autostart = true when enrolling a user, the user would then become ineligible for the experiment and will be auto-unenrolled.
  • The experiment team recommends we use a separate Fission experiment pref.
• Normandy might set Fission pref too late during Firefox startup to enable Fission.
  • Normandy changes a "default branch" pref (instead of "user branch" pref), so the pref change is in-memory only and won't be persisted to the next session. When Firefox restarts, the fission.autostart pref will still have its default value (false).
  • Later during Firefox startup, the Normandy initialization code runs, sees that the user is enrolled in the Fission experiment, and then sets fission.autostart = true. That will happen on every Firefox startup and is the normal process for Normandy initialization.
  • WebRender works around this by persisting the dirty default branch pref value at shutdown and then initializes WebRender during Firefox startup based on that persisted value. See WebRender bug 1503776.

WebRender's Normandy default pref (akin to pref 2 above) is called gfx.webrender.all.qualified. WebRender's persisted user pref (akin to pref 3 above) is called gfx.webrender.all.qualified.default, which seems like a confusing name for a user pref.

Comment 4

[Chris Peterson [:cpeterson]]

These new experiment prefs should probably be locked on Beta and Release like fission.autostart. We don't want users to enable Fission in Beta or Release by manually flipping these experiment prefs (until we're ready to run our Beta and Release experiments).

Comment 5

[Chris Peterson [:cpeterson]]

I updated the requirements in comment 3. The new experiment prefs 2 and 3 should be numbers (enums), not booleans, because we need to differentiate three different experiment states.

Comment 6

[Chris Peterson [:cpeterson]]

kmag's pref names:

• Pref #2: fission.experiment.enrollmentStatus
• Pref #3: fission.experiment.startupEnrollmentStatus

Comment 7

[Kris Maglione [:kmag]]

Attachment: Bug 1660057: Add preferences for Fission rollout Normandy experiment. r=nika

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/737f0e3e2f94
Add preferences for Fission rollout Normandy experiment. r=nika,marionette-reviewers,perftest-reviewers

Comment 9

[Razvan Maries]

Backed out for perma failures.

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=2a126c0cc022c6a9c6c1c2e1e31d6445dbe3c4d1&searchStr=xpcshell&failure_classification_id=2

Logs:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=316632726&repo=autoland&lineNumber=3887
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=316633313&repo=autoland&lineNumber=2903
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=316632948&repo=autoland&lineNumber=4194
https://treeherder.mozilla.org/logviewer.html#?job_id=316625639&repo=autoland

Backout: https://hg.mozilla.org/integration/autoland/rev/be27e8e5cabfcb90f382470cbb6afa8c7566ca15

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/baa88a6bec96
Add preferences for Fission rollout Normandy experiment. r=nika,marionette-reviewers,perftest-reviewers

Comment 11

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/baa88a6bec96

Comment 12

[Maria Heres, inactive, contact Ecosystem QA]

I am marking this as verified fixed based on the testing performed for PI-778 - QA sign-off for Fission experiment recipe. The complete list of bugs found during testing can be found here.

The experiment was tested using Firefox Nightly 83.0a1 (Build IDs 20201004212809 through 20201009041754) on Windows 10, Windows 7 and macOS 10.15.