1329015 - 51: expanding addon/e10s experiment system-addon for end of Beta to ride to Release

Status: VERIFIED FIXED

(Firefox :: Extension Compatibility, defect, P1)

Description

[:shell escalante]

Shell owes Felipe the list of add-on IDs and minimum versions to update the system add-on "allow list" for expansion in Release 51 based on Beta results.

Right now Jorge is looking to see which are already covered in Release 50 criteria - so we can give Felipe the delta that need "allow" listing.

https://docs.google.com/spreadsheets/d/1mv7ae8f9VwWou_zGUdkOcYdy0JuApIIlamjySxTVKr0/edit#gid=1934790911

Unless there's enough user volume gain (requested from jorge from AMO) in some of the ones that have between 50 and 99 users... I'll use the 100 users list (lower risk).

Comment 1

[:shell escalante]

Felipe please add these to "allow list" - send any questions to Shell =]

ID  / Name / version or higher

[e-mail]	Application Update Service Helper		1
[e-mail]	Web Compat	1
[e-mail]	Multi-process staged rollout	1.6
[e-mail]	Pocket	1.0.5
[e-mail]	Facebook	   2
[e-mail]	Yahoo Mail	1
[e-mail]	Google+ (no version - so 0 or higher)
[e-mail]	Twitter		1
[e-mail]	Gmail		1
[e-mail]	Dangerous Websites Blocker		4.0.10.15
[e-mail]	LinkedIn		1
{C7AE725D-FA5C-4027-BB4C-787EF9F8248A}	RelevantKnowledge	1.0.0.4
[e-mail]	Tumblr	1
{95E84BD3-3604-4AAC-B2CA-D9AC3E55B64B}	Adblocker for Youtube™	2.0.0.78

Comment 2

[:shell escalante]

so 51 population will be the list (shell is updating to expand criteria), plus mpc=true, plus webextensions, and blocking tab mix plus.  

we will unblock tab mix plus in Beta 52.  It is likely fine now based on emails with author - but want to see used in beta population with a few of our power users before going to release.  Found out ok mid way through Beta and couldn't change the variable.

Comment 3

[:shell escalante]

Felipe - i'm getting the answer to one question on this list still - but is 774 add-ons too many to extend the e10s experiment to with the "allow list"?  More is better, but I can remove the lower Beta count ones to get to any number that would work for the system add-on.

I'm finding out if we need to keep the weird version numbers (ex: 4.28.1-signed.1-signed)

https://docs.google.com/spreadsheets/d/15mEISyLtI9W1U-39HIBqnGy6bhxWMlfvUT2e-9PBHgw/edit#gid=671211326

Comment 4

[:shell escalante]

So, the version numbers are intentional (ex: 4.28.1-signed.1-signed) and needed as part of the list.

51 Release should still go to all webextensions, all MPC=true, and exclude the same as 50 Release.

The new additions are this list of 774 "allow" add-ons.  The first tab has the name, ID, and version here: https://docs.google.com/spreadsheets/d/15mEISyLtI9W1U-39HIBqnGy6bhxWMlfvUT2e-9PBHgw/edit#gid=671211326

Comment 5

[:shell escalante]

[Tracking Requested - why for this release]: this is the bug for expanding the system add-on to target a bigger add-ons e10s activation combo in release 51, based on the results of Beta 51.  

we didn't want to expand to all MPC=False (nearly all add-ons) in one release, because this would be too many users at once.  This change is a middle step.

this expands e10s availability to 774 new add-ons that were present for more than 50 users in the Beta population.  Beta let us identify a group of add-ons that newly qualified (based on MPC=False), where stability and performance numbers were still good.

Comment 6

[:Felipe Gomes (needinfo for replies!)]

Attachment: Bug 1329015. Configure list for addons on 51 release based on the top addons seen during the experiment on 51 beta.

Review commit: https://reviewboard.mozilla.org/r/103872/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/103872/

Comment 7

[Gerry Chang [:gchang]]

Track 51+ as this expands the system add-on to target a bigger add-on e10s combo in 51 release.

Comment 8

[Mike Conley (:mconley) (:⚙️)]

Comment on attachment 8825793 [details]
Bug 1329015. Configure list for addons on 51 release based on the top addons seen during the experiment on 51 beta.

https://reviewboard.mozilla.org/r/103872/#review104964

Looks good to me - though I didn't hand-compare the list of add-ons and their versions against the ones in the bug. I assume this was machine generated?

::: toolkit/mozapps/extensions/internal/E10SAddonsRollout.jsm:968
(Diff revision 1)
> +  "[e-mail]": {minVersion: "4.0.5", installs: 50},
> +  "[e-mail]": {minVersion: "135.3", installs: 50},
> +  "[e-mail]": {minVersion: "1.0.0.7", installs: 50},
> +};
> +
> +

Nit - needless newline.

Comment 9

[:Felipe Gomes (needinfo for replies!)]

Comment on attachment 8825793 [details]
Bug 1329015. Configure list for addons on 51 release based on the top addons seen during the experiment on 51 beta.

https://reviewboard.mozilla.org/r/103872/#review104964

> Nit - needless newline.

I'll ignore this to autoland on inbound and will clean-up later

Comment 10

[:shell escalante]

Comment on attachment 8825793 [details]
Bug 1329015. Configure list for addons on 51 release based on the top addons seen during the experiment on 51 beta.

Approval Request Comment
[Feature/Bug causing the regression]: expanding e10s to more users with add-ons, mid step to going to all - would like in 1/12 beta 
[User impact if declined]: slows down expansion of e10s to more users 
[Is this code covered by automated tests?]: no - validated by felipe and mconley
[Has the fix been verified in Nightly?]: na
[Needs manual test from QE? If yes, steps to reproduce]: na
[List of other uplifts needed for the feature/fix]: NA
[Is the change risky?]: no
[Why is the change risky/not risky?]: known type of change to existing system add-on
[String changes made/needed]:

Comment 11

[:Felipe Gomes (needinfo for replies!)]

Comment on attachment 8825793 [details]
Bug 1329015. Configure list for addons on 51 release based on the top addons seen during the experiment on 51 beta.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/103872/diff/1-2/

Comment 12

[Liz Henry (:lizzard) (relman/hg->git project)]

Best to mark the particular versions "affected" - otherwise this may not show up on various queries for relman and sheriffs.

Comment 13

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8825793 [details]
Bug 1329015. Configure list for addons on 51 release based on the top addons seen during the experiment on 51 beta.

Tweaks to the top addons list for 51/e10s configuration based on recent data. Let's uplift this for beta 14 build today.

Comment 14

[:Felipe Gomes (needinfo for replies!)]

Thanks a lot, Liz!

https://hg.mozilla.org/releases/mozilla-aurora/rev/8d0ece73d3278ee81807f7d5dcdd402f994954dc
https://hg.mozilla.org/releases/mozilla-beta/rev/31ebd0e871c54c2e212d2fcc30adaf986bcae367

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7d5040b2835e
Configure list for addons on 51 release based on the top addons seen during the experiment on 51 beta. r=mconley

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/7d5040b2835e

Comment 17

[:shell escalante]

shell checking on independent validation that the targeting is working right (though high confidence).  Felipe tested it and I'll test - but it's good to have some QA validation, since they test are good at poking holes.
ex: take one webextension, one mpc=true, one addon in the list, and one not in the list
    make sure they have the desired effect on e10s

Comment 18

[krupa raj [:krupa--use this to needinfo]]

Ica, Can you please verify that the targeting in Beta is working? Thanks!

Comment 19

[Mike Kaply [:mkaply]]

The Amazon Assistant doesn't work properly with e10s yet, but is on this list. Were any partner add-ons on this list validated with those partners?

Comment 20

[Vasilica Tamas, QA [:vtamas]]

Tested this bug on Firefox 51.0 RC (20170118123726) under Windows 10 64-bit, Ubuntu 16.04 32-bit and Mac OS X 10.12.1 and noticed the following:
 
 - installed multiple compatible add-ons ( https://docs.google.com/spreadsheets/d/15mEISyLtI9W1U-39HIBqnGy6bhxWMlfvUT2e-9PBHgw/edit#gid=671211326 ),MPC=true add-ons and webextensions -> e10s remained enabled

 - installed mpc=false add-ons (e.g. Easy Access, LastPass, Screengrab, UnMHT, Autofill Forms, JS Print Setup, S3.Google Translator, Vimperator) -> e10s was disabled by add-on

Based on this testing I am marking this bug as Verified.