Closed
Bug 1229104
Opened 8 years ago
Closed 8 years ago
[meta] Run an e10s A/B experiment on Beta 44
Categories
(Firefox :: General, defect)
Tracking
()
RESOLVED
FIXED
People
(Reporter: vladan, Assigned: vladan)
References
Details
Attachments
(3 files, 2 obsolete files)
2.72 KB,
patch
|
Felipe
:
review+
|
Details | Diff | Splinter Review |
9.80 KB,
patch
|
Details | Diff | Splinter Review | |
5.73 KB,
application/x-xpinstall
|
Details |
Beta 43 experiment (bug 1225074) had to be scrapped, so now we're aiming for beta 1 of Beta 44 (merges on December 15th). We should be able to reuse the experiment code from bug 1225074 almost 100%.
Assignee | ||
Updated•8 years ago
|
Assignee: nobody → vladan.bugzilla
Assignee | ||
Comment 1•8 years ago
|
||
Correction, the experiment code to be reused is in bug 1222894
Comment 2•8 years ago
|
||
It may be useful to track this or just check up on it from a worry list rather than tracking all the individual bugs. Up to you Ritu!
status-firefox44:
--- → affected
tracking-firefox44:
--- → ?
Comment 3•8 years ago
|
||
Vladan, should this be retested again when we get closer to fx44b1? Or the testing that was done in 1222894 sufficient enough?
Assignee | ||
Comment 4•8 years ago
|
||
(In reply to Kamil Jozwiak [:kjozwiak] from comment #3) > Vladan, should this be retested again when we get closer to fx44b1? Or the > testing that was done in 1222894 sufficient enough? There will be uplifts, so we should probably retest the a11y bits just to make sure. We will also revert to an earlier version of the experiment, but only the a11y code-paths will be changed
Assignee | ||
Comment 5•8 years ago
|
||
New experiment xpi Jason, can you sign this?
Flags: needinfo?(jthomas)
Assignee | ||
Comment 6•8 years ago
|
||
Attachment #8697113 -
Flags: review?(felipc)
Assignee | ||
Comment 7•8 years ago
|
||
Attachment #8697113 -
Attachment is obsolete: true
Attachment #8697113 -
Flags: review?(felipc)
Attachment #8697121 -
Flags: review?(felipc)
Updated•8 years ago
|
Attachment #8697121 -
Flags: review?(felipc) → review+
Assignee | ||
Comment 8•8 years ago
|
||
The start date is in the past (November) but the minimum version is Beta 44. Beta 44 will be released on Monday, December 14th next week. The end date is end-of-day December 23rd (GMT). The experiment duration is 10 days. Sampling is still 15% of Beta population.
Assignee | ||
Comment 9•8 years ago
|
||
Comment 10•8 years ago
|
||
Please see attached.
Attachment #8697112 -
Attachment is obsolete: true
Flags: needinfo?(jthomas)
Assignee | ||
Comment 11•8 years ago
|
||
Kamil: the Beta 44 build to test should be available here shortly https://treeherder.mozilla.org/#/jobs?repo=try&revision=b470943a4c8a
Assignee | ||
Comment 12•8 years ago
|
||
(In reply to Vladan Djeric (:vladan) -- please needinfo | PTO Dec 14-18. Email or IRC if urgent from comment #11) > Kamil: the Beta 44 build to test should be available here shortly > https://treeherder.mozilla.org/#/jobs?repo=try&revision=b470943a4c8a There was a bug in the last config, this link should have it fixed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7ec80684f5a1&selectedJob=14557557
Comment 13•8 years ago
|
||
Went through verification using the following build: - http://archive.mozilla.org/pub/firefox/try-builds/vdjeric@mozilla.com-7ec80684f5a1f105fe1a8a135a6b02c5e35ac40d/try-win32/ - https://treeherder.mozilla.org/#/jobs?repo=try&revision=7ec80684f5a1 ** Note: I could only test on Win as that's the only build available Test Plan/Results: https://public.etherpad-mozilla.org/p/e10sBETAExperimentTesting Looks like everything is working as expected other than the possible issues I mentioned below. Vlad, I remember reading the mailing lists about the experiment not being installed on profiles that have a large amount of add-ons? Was that logic implemented into the experiment? Let me know if that needs testing as well.. Possible Issues: - browser.tabs.remote.disabled-for-a11y;false and browser.tabs.remote.disabled-for-a11y;true are not being removed once the experiment is terminated/expires - no e10s UI under about:preferences (pretty sure this is expected but double checking)
Flags: needinfo?(vladan.bugzilla)
Assignee | ||
Comment 14•8 years ago
|
||
> Vlad, I remember reading the mailing lists about the experiment not being installed on profiles that > have a large amount of add-ons? Was that logic implemented into the experiment? Let me know if that > needs testing as well.. No, we never added this logic to the experiment code. Can you send me a link to this discussion? > browser.tabs.remote.disabled-for-a11y;false and browser.tabs.remote.disabled-for-a11y;true > are not being removed once the experiment is terminated/expires This pref is set by Firefox code, not the experiment, so I think this is fine. Felipe, can you confirm? > no e10s UI under about:preferences (pretty sure this is expected but double checking) This is fine.
Flags: needinfo?(vladan.bugzilla) → needinfo?(felipc)
Comment 15•8 years ago
|
||
> No, we never added this logic to the experiment code. Can you send me a link > to this discussion? https://groups.google.com/forum/#!topic/mozilla.dev.platform/OTOwv2PHfV8
Comment 16•8 years ago
|
||
> > browser.tabs.remote.disabled-for-a11y;false and browser.tabs.remote.disabled-for-a11y;true
> > are not being removed once the experiment is terminated/expires
>
> This pref is set by Firefox code, not the experiment, so I think this is
> fine. Felipe, can you confirm?
Yep, it's the correct behavior
Flags: needinfo?(felipc)
Comment 17•8 years ago
|
||
> Yep, it's the correct behavior Looks like the only thing left is to double check and make sure if we need the logic to check and see if there's any add-ons installed before enabling the experiment. I believe the original concern was that some add-ons are still not working correctly with e10s and can cause performance problems. Apparently LastPass [1] doesn't play well with e10s so if we end up enabling e10s on a profile with LastPass, we might anger some users. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1008768
Given that 44.0b1 (desktop) was pushed to Beta channel only a few hours ago, should we consider extending the experiment end date as the telemetry data will only start accumulating now?
Flags: needinfo?(vladan.bugzilla)
Comment 19•8 years ago
|
||
Extending would be good. Seems to be running correctly. Just a small number of beta users are currently updated onto 44, making sample low. Overall e10s crash rate is 25% (+-10%) up. Browser halved. Plugin two-thirds. Low on samples Low memory account for a few percent more. Fixed in 45 bug 1225829 up to 5% of the increase. Content shutdown timeout likely to be a couple. Couple current high signatures showing user trying multiple times to restore crashing content. https://crash-stats.mozilla.com/search/?product=Firefox&ActiveExperiment=%3De10s-enabled-beta-20151214%40experiments.mozilla.org&date=%3E%3D2015-12-18&ActiveExperimentBranch=%3Dcontrol&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports https://crash-stats.mozilla.com/search/?product=Firefox&ActiveExperiment=%3De10s-enabled-beta-20151214%40experiments.mozilla.org&date=%3E%3D2015-12-18&ActiveExperimentBranch=%3Dexperiment&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports
Assignee | ||
Comment 20•8 years ago
|
||
(In reply to Ritu Kothari (:ritu) from comment #18) > Given that 44.0b1 (desktop) was pushed to Beta channel only a few hours ago, > should we consider extending the experiment end date as the telemetry data > will only start accumulating now? Yes, I will extend its end date
Flags: needinfo?(vladan.bugzilla)
Tracked for FF44 and marked as "fixed" given that the experiment is now on and will likely end after 44.0b4
Assignee | ||
Updated•8 years ago
|
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•