1373707 - Include users with only webextensions in Release to receive Multi using the system add-on

Status: RESOLVED FIXED

(Firefox :: Extension Compatibility, defect)

Description

[:shell escalante]

Multi is first going out to users without add-ons in 54 (throttling up).  After that is to all the users that qualify and looks good, we need to expand to Release users that have only WebExtensions.  WebExtensions + Multi cohort was been in Beta 54.

Pros:
Expanding to that population will de-risk 57, by exposing any issues early in what will be the 57 release configuration.  We only saw one issue in Beta (see below), there are no other known issues.  It also gets Multi out to a broader audience, ideally improving their experience before 57.

Cons:
The Beta difference in stability between multi + no add-ons and multi + webextension cohorts was 1 more crash per 25,000 hours of use with WebExtensions.  The small increase is outweighed by the benefits. The difference was a higher EnqueuePromiseReactionJob crash occurrence in WE.  Still the Pros of expanding outweigh this Cons. The scale is misleading - but if needed is here stability between cohorts https://sql.telemetry.mozilla.org/queries/4355#8685

Mitigation:
we can watch that crash signature for a big change between before and after the new cohort is added and remove Webextensions from getting Multi if needed.  

Note: 
Please engage with Ben to validate in telemetry that the targeting is hitting the right add-ons.  Along with QA, that has been the most successful way of finding issues in targeting before release.

Comment 1

[Blake Kaplan (:mrbkap) (inactive)]

Attachment: Bug 1373707 - Enable e10s-multi for users with webextensions on Release.

Review commit: https://reviewboard.mozilla.org/r/149914/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/149914/

Comment 2

[Blake Kaplan (:mrbkap) (inactive)]

Attachment: Bug 1373707 - Rename some things to clarify what's going on.

Felipe, let me know if you like this. Otherwise, I'll only land the first
changeset here.

Review commit: https://reviewboard.mozilla.org/r/149916/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/149916/

Comment 3

[Blake Kaplan (:mrbkap) (inactive)]

Comment on attachment 8878599 [details]
Bug 1373707 - Enable e10s-multi for users with webextensions on Release.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/149914/diff/1-2/

Comment 4

[:Felipe Gomes (needinfo for replies!)]

Comment on attachment 8878599 [details]
Bug 1373707 - Enable e10s-multi for users with webextensions on Release.

https://reviewboard.mozilla.org/r/149914/#review154680

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/615ffcb208c4
Enable e10s-multi for users with webextensions on Release. r=Felipe

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/615ffcb208c4

Comment 7

[Blake Kaplan (:mrbkap) (inactive)]

Comment on attachment 8878599 [details]
Bug 1373707 - Enable e10s-multi for users with webextensions on Release.

Approval Request Comment
[Feature/Bug causing the regression]: n/a
[User impact if declined]: Needed for e10s-multi rollout strategy.
[Is this code covered by automated tests?]: No.
[Has the fix been verified in Nightly?]: Not yet.
[Needs manual test from QE? If yes, steps to reproduce]: Will send email to pi-request.
[List of other uplifts needed for the feature/fix]: None.
[Is the change risky?]: No.
[Why is the change risky/not risky?]: Only changes the e10s rollout addon.
[String changes made/needed]: n/a

Comment 8

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8878599 [details]
Bug 1373707 - Enable e10s-multi for users with webextensions on Release.

Limit for e10s-multi cohort on release - let's uplift this for beta 5.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/1279eb02d4ae