Closed Bug 1722112 Opened 3 years ago Closed 3 years ago

[Experiment] Pref-Flip Experiment: Fission Process Count Fx 92.0 to 94.0 Nightly

Tracking

(firefox90 disabled, firefox91 disabled, firefox92 affected)

Status:

RESOLVED FIXED

Tracking Flags:

Tracking

Status

firefox90

---

disabled

firefox91

---

disabled

firefox92

---

affected

People

(Reporter: experimenter, Assigned: cpeterson)

References

(
URL
)

Details

(Whiteboard: [fission:m8])

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

dom.ipc.processCount.webIsolated

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment process-count-4 50%:

Value: 4

This treatment branch uses up to four content processes per site.
        
- Control process-count-1 50%:

Value: 1

The control branch uses one content process per site.
        

    What version and channel do you intend to ship to?

100% of Nightly Firefox 92.0 to 94.0

    Are there specific criteria for participants?

* Require telemetry environment's "fissionEnabled" == true so we only draw from the ~65% of Nightly users who are already using Fission.

NOTE: We want this experiment to run for all Fission Nightly users (so 50% of them are treatment, 50% are control). Therefore we need to target 100% of Nightly because the fissionEnabled filter will select just the Fission users.
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Aug 03, 2021 - Aug 31, 2021 (28 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will you use to make these decisions? What metrics are you using to measure success?

Same as all of our Fission Beta experiments, the most recent being https://experimenter.services.mozilla.com/experiments/fission-beta-91-experiment/

We want to be able to monitor the difference in stability and performance metrics between clients with and without Fission enabled. The Fission Experiment Plan (https://docs.google.com/document/d/1ZXAHep4v1lW-ZpLMtnTXAtbXSNP8Q-gr-RjUha2Yb14/edit#heading=h.asctkf7cbxa7) has a detailed list of telemetry probes (and expectations for each probe: up, down, or flat).

We plan to monitor these metrics:

Memory metrics:

- `MEMORY_TOTAL`
- `MEMORY_UNIQUE_CONTENT_STARTUP`
- `CYCLE_COLLECTOR_MAX_PAUSE`
- `CYCLE_COLLECTOR_MAX_PAUSE_CONTENT`
- `GC_MAX_PAUSE_MS_2`
- `GC_MAX_PAUSE_MS_2_CONTENT`
- `GC_MS`
- `GC_MS_CONTENT`
- `GC_SLICE_DURING_IDLE`
- `GC_SLICE_DURING_IDLE_CONTENT`

Performance metrics:

- `CHECKERBOARDING_SEVERITY`
- `CHILD_PROCESS_LAUNCH_MS`
- `CONTENT_FRAME_TIME_VSYNC`
- `FX_NEW_WINDOW_MS`
- `FX_TAB_SWITCH_COMPOSITE_E10S_MS`
- `KEYPRESS_PRESENT_LATENCY_MS`
- `INPUT_EVENT_RESPONSE_MS`
- `TIME_TO_FIRST_CONTENTFUL_PAINT_MS`
- `TIME_TO_FIRST_INTERACTION_MS`
- `PERF_PAGE_LOAD_TIME_MS`

Engagement metrics:

- Number of users submitting pings
- `browser.engagement.total_uri_count`
- `browser.engagement.tab_open_event_count`
- `browser.engagement.max_concurrent_tab_count`
- `SUBSESSION_LENGTH`
- `CONTENT_PROCESS_COUNT`
- `CONTENT_PROCESS_MAX`
- `LOADED_TAB_COUNT`

Stability metrics:

- Main process crash rate (per 1000 usage hours), 32-bit and 64-bit
- Content process crashes (per 1000 usage hours) (excluding ShutDownKills), 32-bit and 64-bit
- `crash_tab_ui_presented`
- `crash_subframe_ui_presented`
- `oom_crashes` (per 1000 usage hours), 32-bit and 64-bit
- `shutdown_crashes` (per 1000 usage hours)

Do you plan on surveying users at the end of the delivery? No.

 Estimated Total Enrolled Clients: 360,000 (assuming we enroll 15% of 1.2M Beta + Dev Edition users running Windows or macOS)

    Who is the owner of the data analysis for this experiment?

cdowhygelund@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

TBD? This experiment is not much different from our Fission Beta experiments like https://experimenter.services.mozilla.com/experiments/fission-beta-91-experiment/

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/fission-process-count/

Experimenter Bot

Reporter

Description

•

3 years ago

Fission Process Count

Track the stability and performance effects of increasing the number of Fission content processes.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DS-1674
More information: https://experimenter.services.mozilla.com/experiments/fission-process-count/

Chris Peterson [:cpeterson]

Assignee