Closed Bug 1558614 Opened 5 years ago Closed 5 years ago

[Experiment] Pref-Flip: WebRender AMD Performance 68 Fx 68.0 Release

Tracking

(firefox68+ fixed)

Status:

RESOLVED FIXED

Tracking Flags:

Tracking

Status

firefox68

fixed

People

(Reporter: experimenter, Assigned: jbonisteel)

References

Details

User Story

Experiment Type: Pref Flip Experiment

    What is the preference we will be changing

gfx.webrender.all.qualified

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment enabled 50%:

Value: true

Control branch
        
- Treatment disabled 50%:

Value: false

This is the group that we will disable WebRender for so we can compare performance against those that do have WebRender
        

    What version and channel do you intend to ship to?

30% of Release Firefox 68.0

    Are there specific criteria for participants?

Prefs: none
Experiments: none

Any additional filters:
1. Windows 10
2. normandy.telemetry.main.environment.system.gfx.features.wrQualified.status == 'available'
3. AMD graphics card (PCI vendor ID for primary device is 1002); see normandy.telemetry.main.environment.system.gfx.adapters[0].vendorID
Countries: 

Locales: 

    What is your intended go live date and how long will the experiment run?

Jul 16, 2019 - Aug 13, 2019 (28 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

overall crash reports ≤ 5% increase in crash rate
OOM crash reports ≤ 5% increase in crash rate
shutdown crashes ≤ 5% increase in crash rate
CANVAS_WEBGL_SUCCESS ≤ 5% regression in median of fraction "True" per user
COMPOSITE_TIME Median per-user fraction of slow frames < 0.5% (absolute)
CONTENT_FRAME_TIME_VSYNC ≤ 5% regression in median of per-user fraction of slow events
CONTENT_FULL_PAINT_TIME ≤ 5% regression in fraction of slow paints (> 16 ms), ≤ 5% regression in median of per-user means
DEVICE_RESET_REASON ≤ 5% increase in reset rate
FX_PAGE_LOAD_MS_2 ≤ 5% regression in median of per-user means

Crash (/submission/reset) rates will be measured as events per 1,000 usage hours. A "slow event" means 16 ms or 200% vsync.

Do you plan on surveying users at the end of the experiment?  No.
Strategy and Insights can help create surveys if needed

    Who is the owner of the data analysis for this experiment?

tdsmith

    Will this experiment require uplift?

False

    QA Status of your code:

Green

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/webrender-amd-performance-68/

Experimenter Bot

Reporter

Description

•

5 years ago

WebRender AMD Performance 68

The goal of this study is to hold back a small population of eligible users from enabling WebRender, another milestone of which is riding the trains to release 68, in order to assess its effect on performance and engagement metrics. Our specific target for 68 is desktop, Windows 10 with AMD graphics cards.

More information: https://experimenter.services.mozilla.com/experiments/webrender-amd-performance-68/

Tim Smith (inactive) 👨‍🔬 [:tdsmith]