Similar to bug 1319206 (GPU Process A/B test on Windows in Nightly 53) we want to A/B test GPU Process in Beta 53 on Windows in support of bug 1264543. Currently Beta users who qualify have GPU Process on by default (~35% of users); we want to turn GPU Process off for half of these users to compare the stability. All other factors being equal this should help us determine the impact of GPU Process prior to release.
> Product: Firefox Beta
> Version: 53.0
> OS: Windows (all)
> Channel: beta
> Build ID: 2017-03-07+ (53.0b1+)
> Language: en-US
> Sample ratio: 50%
> Start Date: March 27, 2017
> End Date: April 10, 2017
The proposed dates will give us roughly 2 weeks of testing and leave just over a week for pre-release stabilization.
Created attachment 8850730 [details] [diff] [review]
David, can you please review this patch? It's essentially the same as the experiment we ran on Nightly updated for Beta. Basically it should take a random selection of Beta users with who have GPU Process enabled and flip that pref.
Created attachment 8850734 [details]
Can you please sign the xpi?
Created attachment 8850738 [details]
Please see attached.
Thanks Jason and David for the quick follow up.
Cory, :ekr suggested you might be able to push this to staging for me. Can you take care of this?
If not, I'll ask Felipe in the morning.
Felipe, can you please take care of the push to staging? I've not heard back from :ckprice yet.
Sorry for the noise, Felipe. I neglected to check IRC scroll back and it looks like :ckprice is looking in to this.
This needs RelMan approval. Usually via an "Intent to Ship" email: https://wiki.mozilla.org/Firefox/Go_Faster/System_Add-ons/Process#Intent_to_Ship_and_RelMan_Approval
I've also NI'd Julien
302 → liz
This is a telemetry experiment, right? Not a system add-on? The process is different here, pushing to staging is just a matter of landing this on http://hg.mozilla.org/webtools/telemetry-experiment-server/ as long as it is reviewed
Yes, this is just a Telemetry Experiment. I'd like this pushed to staging so I can test it. Assuming it passes my tests I will then send I2S email to RelMan for approval to push live.
(In reply to Anthony Hughes (:ashughes) [GFX][QA][Mentor] from comment #10)
> Yes, this is just a Telemetry Experiment. I'd like this pushed to staging so
> I can test it. Assuming it passes my tests I will then send I2S email to
> RelMan for approval to push live.
To clarify I am following the instructions starting from https://wiki.mozilla.org/QA/Telemetry/Developing_a_Telemetry_Experiment#Testing_on_Beta.
I've tested this in a local build, had the code reviewed, had the XPI signed. If I am not mistaken the next step should be push to staging, test on staging, email RelMan, then push live.
Yes, it's fine to get this onto staging and then send the Intent to Ship before deployment (according to the guidelines here: https://wiki.mozilla.org/Telemetry/Experiments ). Thanks!
Pushed to staging:
Should be in https://telemetry-experiment-dev.allizom.org/ in a couple of minutes.
I think Cory was following the process for system add-ons.
> I think Cory was following the process for system add-ons.
Yes, my mistake.
I've signed off the staged experiment and sent an email to RelMan for approval.
Thanks, looks good to me! Please go ahead and deploy.
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #17)
> Thanks, looks good to me! Please go ahead and deploy.
Thanks Liz, I sent the deploy request via bug 1350100.
This is now live and tested as working in production.