Closed Bug 1722112 Opened 2 months ago Closed 28 days ago

[Experiment] Pref-Flip Experiment: Fission Process Count Fx 92.0 to 94.0 Nightly

Categories

(Shield :: Shield Study, task, P3)

Tracking

(firefox90 disabled, firefox91 disabled, firefox92 affected)

RESOLVED FIXED
Tracking Status
firefox90 --- disabled
firefox91 --- disabled
firefox92 --- affected

People

(Reporter: experimenter, Assigned: cpeterson)

References

()

Details

(Whiteboard: [fission:m8])

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

dom.ipc.processCount.webIsolated

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment process-count-4 50%:

Value: 4

This treatment branch uses up to four content processes per site.
        
- Control process-count-1 50%:

Value: 1

The control branch uses one content process per site.
        

    What version and channel do you intend to ship to?

100% of Nightly Firefox 92.0 to 94.0

    Are there specific criteria for participants?

* Require telemetry environment's "fissionEnabled" == true so we only draw from the ~65% of Nightly users who are already using Fission.

NOTE: We want this experiment to run for all Fission Nightly users (so 50% of them are treatment, 50% are control). Therefore we need to target 100% of Nightly because the fissionEnabled filter will select just the Fission users.
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Aug 03, 2021 - Aug 31, 2021 (28 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will you use to make these decisions? What metrics are you using to measure success?

Same as all of our Fission Beta experiments, the most recent being https://experimenter.services.mozilla.com/experiments/fission-beta-91-experiment/

We want to be able to monitor the difference in stability and performance metrics between clients with and without Fission enabled. The Fission Experiment Plan (https://docs.google.com/document/d/1ZXAHep4v1lW-ZpLMtnTXAtbXSNP8Q-gr-RjUha2Yb14/edit#heading=h.asctkf7cbxa7) has a detailed list of telemetry probes (and expectations for each probe: up, down, or flat).

We plan to monitor these metrics:

Memory metrics:

- `MEMORY_TOTAL`
- `MEMORY_UNIQUE_CONTENT_STARTUP`
- `CYCLE_COLLECTOR_MAX_PAUSE`
- `CYCLE_COLLECTOR_MAX_PAUSE_CONTENT`
- `GC_MAX_PAUSE_MS_2`
- `GC_MAX_PAUSE_MS_2_CONTENT`
- `GC_MS`
- `GC_MS_CONTENT`
- `GC_SLICE_DURING_IDLE`
- `GC_SLICE_DURING_IDLE_CONTENT`

Performance metrics:

- `CHECKERBOARDING_SEVERITY`
- `CHILD_PROCESS_LAUNCH_MS`
- `CONTENT_FRAME_TIME_VSYNC`
- `FX_NEW_WINDOW_MS`
- `FX_TAB_SWITCH_COMPOSITE_E10S_MS`
- `KEYPRESS_PRESENT_LATENCY_MS`
- `INPUT_EVENT_RESPONSE_MS`
- `TIME_TO_FIRST_CONTENTFUL_PAINT_MS`
- `TIME_TO_FIRST_INTERACTION_MS`
- `PERF_PAGE_LOAD_TIME_MS`

Engagement metrics:

- Number of users submitting pings
- `browser.engagement.total_uri_count`
- `browser.engagement.tab_open_event_count`
- `browser.engagement.max_concurrent_tab_count`
- `SUBSESSION_LENGTH`
- `CONTENT_PROCESS_COUNT`
- `CONTENT_PROCESS_MAX`
- `LOADED_TAB_COUNT`

Stability metrics:

- Main process crash rate (per 1000 usage hours), 32-bit and 64-bit
- Content process crashes (per 1000 usage hours) (excluding ShutDownKills), 32-bit and 64-bit
- `crash_tab_ui_presented`
- `crash_subframe_ui_presented`
- `oom_crashes` (per 1000 usage hours), 32-bit and 64-bit
- `shutdown_crashes` (per 1000 usage hours)

Do you plan on surveying users at the end of the delivery? No.

 Estimated Total Enrolled Clients: 360,000 (assuming we enroll 15% of 1.2M Beta + Dev Edition users running Windows or macOS)

    Who is the owner of the data analysis for this experiment?

cdowhygelund@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

TBD? This experiment is not much different from our Fission Beta experiments like https://experimenter.services.mozilla.com/experiments/fission-beta-91-experiment/

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/fission-process-count/

Fission Process Count

Track the stability and performance effects of increasing the number of Fission content processes.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DS-1674
More information: https://experimenter.services.mozilla.com/experiments/fission-process-count/

Whiteboard: [fission:m8]
User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: Fission Process Count → [Experiment] Pref-Flip Experiment: Fission Process Count Fx 92.0 to 94.0 Nightly
Start Date: 2021-08-03 End Date: 2021-08-31
Status: NEW → RESOLVED
Closed: 28 days ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.