Closed Bug 1669753 Opened 1 year ago Closed 10 months ago

[Experiment] Pref-Flip Experiment: New Print UI Fx 82.0 to 82.0 Release

Categories

(Shield :: Shield Study, task, P3)

Tracking

(firefox82+ fixed)

RESOLVED FIXED
Tracking Status
firefox82 + fixed

People

(Reporter: experimenter, Assigned: RT)

References

(Blocks 1 open bug, )

Details

(Whiteboard: [no-nag])

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

print.tab_modal.enabled

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment Treatment 50%:

Value: true

New print UI
        
- Control Control 50%:

Value: false

Old print UI
        

    What version and channel do you intend to ship to?

3% of Release Firefox 82.0 to 82.0

    Are there specific criteria for participants?

Prefs:

Experiments:

Any additional filters:
Target users that have upgraded to the latest 82 dot release (which I believe would be 82.0.1)
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Nov 03, 2020 - Nov 24, 2020 (21 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will
you use to make these decisions? What metrics are you using to measure success
Success is measured by:
- print error/print success rate on treatment is better or equal to print error/print success rate on control
- average prints per user are similar on control and treatment

Analysis plan:
- When deviations exist identify what high print error rates correlate with (hardware characteristics, software characteristics, browser configuration)
- When deviations exist identify what abnormal print outputs per user correlate with (hardware characteristics, software characteristics, browser configuration)

Do you plan on surveying users at the end of the delivery?
No

    Who is the owner of the data analysis for this experiment?

rbologa@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

Yellow per last QA report sent Oct 13th; 
QA Status: GREEN. 
For more information see https://experimenter.services.mozilla.com/experiments/new-print-ui/#comment1192.

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/new-print-ui/

New Print UI

Validate that the new print UI does not regress Firefox print user experience.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DS-879
More information: https://experimenter.services.mozilla.com/experiments/new-print-ui/

Whiteboard: [no-nag]

QA for small scale new Print UI Experiment
Firefox Release 82

We have finished performing a smoke test on [PI-837] QA for small scale new Print UI experiment.

Quality status: GREEN - SHIP IT

Why is this experiment green?

  • Delivery mechanism:
    • No issues found.
  • Product specific:
    • No issues found.

Testing Summary:

Tested Platforms:

  • Windows 10
  • macOS 10.15
  • Linux MX 4.19/Ubuntu 20.04

Tested Firefox versions:

  • Firefox Release 82.0 (BuildID 20201014125134)

Regards,
Ciprian Muresan
Ecosystem QA Team

User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: New Print UI → [Experiment] Pref-Flip Experiment: New Print UI Fx 82.0 to 82.0 Release
Status: NEW → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED
User Story: (updated)
You need to log in before you can comment on or make changes to this bug.