Closed Bug 1591985 Opened 6 years ago Closed 6 years ago

[Experiment] Pref-Flip Experiment: Unimplemented HTMLDocument API use counters in beta Fx 71.0 Beta

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(firefox71+ fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox71 + fixed

People

(Reporter: experimenter, Unassigned)

References

(
URL
)

Details

User Story

Experiment Type: Pref Flip Experiment

    What is the preference we will be changing

dom.missing_prop_counters.enabled

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment enabled 50%:

Value: true

Collection enabled
        
- Treatment disabled 50%:

Value: false

Collection disabled
        

    What version and channel do you intend to ship to?

2% of Beta Firefox 71.0

    Are there specific criteria for participants?

none
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Nov 05, 2019 - Nov 12, 2019 (7 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will
you use to make these decisions? What metrics are you using to measure success

n/a; data collection study

Do you plan on surveying users at the end of the experiment? Yes/No.

no.

    Who is the owner of the data analysis for this experiment?

tismith@mozilla.com

    Will this experiment require uplift?

False

    QA Status of your code:

Not started. https://jira.mozilla.com/browse/PI-341
Signed off as GREEN, for more information see https://experimenter.services.mozilla.com/experiments/unimplemented-htmldocument-api-use-counters-in-beta/#comment509

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/unimplemented-htmldocument-api-use-counters-in-beta/

Unimplemented HTMLDocument API use counters in beta

Enable use counters for unimplemented HTMLDocument use counters

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

More information: https://experimenter.services.mozilla.com/experiments/unimplemented-htmldocument-api-use-counters-in-beta/

[Tracking Requested - why for this release]: I plan to run this experiment in beta 71.

tracking-firefox71: --- → ?

Unimplemented HTMLDocument API use counters
Firefox Beta 71.0bx

We have finished testing the Unimplemented HTMLDocument API use counters experiment.

QA’s recommendation: GREEN - SHIP IT

Reasoning:

  • We haven’t found any issues during testing.

Testing Summary:

  • Verified that the “dom.missing_prop_counters.enabled” pref can be set to “true” value.
  • Verified that the “dom.missing_prop_counters.enabled” pref doesn’t reset to its default value after restarting/reopening the browser.
  • Verified that the “USE_COUNTER2_HTMLDOCUMENT_ADOPTEDSTYLESHEETS_DOCUMENT” histogram is generated after visiting and closing the page provided in the Testing Instructions.
  • Verified that the specific histogram changes its value after visiting and closing the page provided in the Testing Instructions using the following actions:
    • “Close tab” button
    • “Close Other Tabs” option from context menu
    • “Close Tabs to the Right” option from context menu
  • Verified that the specific histogram changes its value after visiting and closing the page provided in the Testing Instructions on a New Window.
  • Verified that the specific histogram changes its value after visiting and closing page provided in the Testing Instructions on a New Private Window.
  • Verified that the specific histogram changes its value after reloading the page provided in the Testing Instructions.
  • Verified that the specific histogram changes its value after closing 2 duplicate tabs with the page provided in the Testing Instructions.
  • Verified that the specific histogram starts counting from 0 after restarting/reopening the browser.
  • Haven't noticed any slowdown in browser starting or while loading several websites.

Tested Platforms:

  • Windows 10 x64
  • Ubuntu 16.04 x64
  • macOS 10.14.5

Tested Firefox versions:

  • Firefox Beta 71.0b6 (en-US, fr, es-ES)
User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: Unimplemented HTMLDocument API use counters in beta → [Experiment] Pref-Flip Experiment: Unimplemented HTMLDocument API use counters in beta Fx 71.0 Beta
Start Date: 2019-11-05 End Date: 2019-11-12
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
status-firefox71: affected → fixed
User Story: (updated)
You need to log in before you can comment on or make changes to this bug.