Run a Telemetry experiment to vet GPU Process on Windows prior to release in Firefox 53

RESOLVED FIXED

Status

()

Core
Graphics
RESOLVED FIXED
5 months ago
5 months ago

People

(Reporter: ashughes, Assigned: ashughes)

Tracking

(Blocks: 1 bug)

53 Branch
All
Windows
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox53+ fixed)

Details

(Whiteboard: [gfx-noted])

Attachments

(3 attachments)

Similar to bug 1319206 (GPU Process A/B test on Windows in Nightly 53) we want to A/B test GPU Process in Beta 53 on Windows in support of bug 1264543. Currently Beta users who qualify have GPU Process on by default (~35% of users); we want to turn GPU Process off for half of these users to compare the stability. All other factors being equal this should help us determine the impact of GPU Process prior to release.

Proposed Criteria:
>      Product: Firefox Beta
>      Version: 53.0
>           OS: Windows (all)
>      Channel: beta
>     Build ID: 2017-03-07+ (53.0b1+)
>     Language: en-US
> Sample ratio: 50%
>   Start Date: March 27, 2017
>     End Date: April 10, 2017

The proposed dates will give us roughly 2 weeks of testing and leave just over a week for pre-release stabilization.
Created attachment 8850730 [details] [diff] [review]
bug1350092-v1.patch

David, can you please review this patch? It's essentially the same as the experiment we ran on Nightly updated for Beta. Basically it should take a random selection of Beta users with who have GPU Process enabled and flip that pref.

Thanks
Attachment #8850730 - Flags: review?(dvander)
Created attachment 8850734 [details]
experiment.xpi

Jason,

Can you please sign the xpi?
Flags: needinfo?(jthomas)
Blocks: 1350100

Comment 3

5 months ago
Created attachment 8850738 [details]
experiment.xpi signed

Please see attached.
Flags: needinfo?(jthomas)
Attachment #8850730 - Flags: review?(dvander) → review+
Thanks Jason and David for the quick follow up.

Cory, :ekr suggested you might be able to push this to staging for me. Can you take care of this? 

If not, I'll ask Felipe in the morning.
Flags: needinfo?(cprice)
Felipe, can you please take care of the push to staging? I've not heard back from :ckprice yet.
Flags: needinfo?(felipc)
Sorry for the noise, Felipe. I neglected to check IRC scroll back and it looks like :ckprice is looking in to this.
Flags: needinfo?(felipc)
This needs RelMan approval. Usually via an "Intent to Ship" email: https://wiki.mozilla.org/Firefox/Go_Faster/System_Add-ons/Process#Intent_to_Ship_and_RelMan_Approval

I've also NI'd Julien
Flags: needinfo?(cprice) → needinfo?(jcristau)
302 → liz
Flags: needinfo?(jcristau) → needinfo?(lhenry)
This is a telemetry experiment, right? Not a system add-on? The process is different here, pushing to staging is just a matter of landing this on http://hg.mozilla.org/webtools/telemetry-experiment-server/ as long as it is reviewed
Yes, this is just a Telemetry Experiment. I'd like this pushed to staging so I can test it. Assuming it passes my tests I will then send I2S email to RelMan for approval to push live.
(In reply to Anthony Hughes (:ashughes) [GFX][QA][Mentor] from comment #10)
> Yes, this is just a Telemetry Experiment. I'd like this pushed to staging so
> I can test it. Assuming it passes my tests I will then send I2S email to
> RelMan for approval to push live.

To clarify I am following the instructions starting from https://wiki.mozilla.org/QA/Telemetry/Developing_a_Telemetry_Experiment#Testing_on_Beta. 

I've tested this in a local build, had the code reviewed, had the XPI signed. If I am not mistaken the next step should be push to staging, test on staging, email RelMan, then push live.
Flags: needinfo?(cprice)
Yes, it's fine to get this onto staging and then send the Intent to Ship before deployment (according to the guidelines here: https://wiki.mozilla.org/Telemetry/Experiments ). Thanks!
Pushed to staging:
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/8caa6ce7abe680000b2b8b76f0fca9b99f65fc7a

Should be in https://telemetry-experiment-dev.allizom.org/ in a couple of minutes.

I think Cory was following the process for system add-ons.
> I think Cory was following the process for system add-ons.
Yes, my mistake.
Flags: needinfo?(cprice)
Thanks everyone!
status-firefox53: --- → affected
tracking-firefox53: --- → +
Flags: needinfo?(lhenry)
I've signed off the staged experiment and sent an email to RelMan for approval.
Thanks, looks good to me! Please go ahead and deploy.
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #17)
> Thanks, looks good to me! Please go ahead and deploy.

Thanks Liz, I sent the deploy request via bug 1350100.
This is now live and tested as working in production.
Status: NEW → RESOLVED
Last Resolved: 5 months ago
Resolution: --- → FIXED

Updated

5 months ago
status-firefox53: affected → fixed
You need to log in before you can comment on or make changes to this bug.