Closed Bug 1722551 Opened 4 years ago Closed 4 years ago

[Experiment] Pref-Flip Experiment: Full JS Parsing Experiment Fx 94.0 to 94.0 Nightly

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: experimenter, Assigned: denispal)

References

(
URL
)

Details

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

dom.script_loader.full_parse

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment Enable full parsing 50%:

Value: true

Force off-thread full JS parsing on speculative loads
        
- Control Default 50%:

Value: false

Default behavior
        

    What version and channel do you intend to ship to?

100% of Nightly Firefox 94.0 to 94.0

    Are there specific criteria for participants?

Experiments:  with the filter below  - we want to avoid the 70% of nightly users in https://experimenter.services.mozilla.com/experiments/ongoing-fission-nightly-rollout-clone/.  so we don't get a few fission users (that might not be in that experiment) that might muddy the data - i believe we should use a fission exclusion rule.

Only include clients WITHOUT fission turned on:  so do not enroll anyone with Pref:  fission.autostart=true
(based on past experience - i don't think we can guarantee that they would have false - so seems safest to EXCLUDE users with it set to true)
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Sep 07, 2021 - Sep 28, 2021 (21 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

We will measure the following metrics:
PERF_PAGE_LOAD_TIME_MS
FX_PAGE_LOAD_MS_2
JS_PAGELOAD_EXECUTION_MS
JS_PAGELOAD_DELAZIFICATION_MS
KEYPRESS_PRESENT_LATENCY
SCROLL_PRESENT_LATENCY
MEMORY_HEAP_ALLOCATED
MEMORY_TOTAL
MEMORY_UNIQUE
MEMORY_JS_GC_HEAP

    Who is the owner of the data analysis for this experiment?

cdowhygelund@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

https://mozilla-hub.atlassian.net/browse/PI-1150

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/full-js-parsing-experiment/

Full JS Parsing Experiment

Perform off-thread full parsing of any JavaScript files that are speculatively fetched during page load. Pages should load faster for a small trade off in memory.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DO-545
More information: https://experimenter.services.mozilla.com/experiments/full-js-parsing-experiment/

We've finished testing the experiment and sent a GREEN sign-off via email. You can see it here.

User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: Full JS Parsing Experiment → [Experiment] Pref-Flip Experiment: Full JS Parsing Experiment Fx 94.0 to 94.0 Nightly
Start Date: 2021-09-07 End Date: 2021-09-28
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
See Also: → 1736057
You need to log in before you can comment on or make changes to this bug.