Open Bug 1628146 Opened 4 years ago Updated 4 years ago

[Experiment]: Generic Experiment: About:Welcome Pull Factor Reinforcement

Categories

(Shield :: Shield Study, task, P3)

Product:
Shield
Component:
Shield Study
Type:
task

Tracking

(Not tracked)

People

(Reporter: experimenter, Assigned: pdahiya)

References

(
URL
)

Details

About:Welcome Pull Factor Reinforcement

4 branch experiment different variants of about:welcome with a goal of testing new experiment framework and get insights on whether reinforcing pull-factors improves retention. Test deployment of multiple branches using new experiment framework

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DO-186
More information: https://experimenter.services.mozilla.com/experiments/aboutwelcome-pull-factor-reinforcement/

About:Welcome Pull Factor Reinforcement
Firefox Beta 76

We have finished testing the About:Welcome Pull Factor Reinforcement experiment part from [PI-580] Remotely-configured Multivariate Experiments on the latest version of Firefox Beta 76.0b8 downloaded from the treeherder and using the steps provided in the “Experimentation Platform Design for about:welcome in 76” document.

Quality status: GREEN - SHIP IT

Why is this experiment green?

  • We have not found any issues during testing.

Testing Summary:

  • Verified that the user is successfully enrolled in each branch based on the Normandy User ID.
  • Verified that the UI and all the page elements respect each branch’s specifications.
  • Verified that the cards redirect to the correct websites.
  • Verified that a “Private Window” is successfully opened after the “Browse Privately” card’s button is clicked.
  • Verified that the telemetry pings are correctly displayed in the “Browser Console”.
  • Verified that the telemetry pings respect the format from the specifications.
  • Verified that the “utm_term” from the card URL respects the one received in the specifications.
  • Verified that there are no issues at the browser resize.
  • Verified that all variations of the “about:welcome” page respect the browser’s theme colors.
  • Verified that the card’s buttons can be accessed through keyboard navigation.

Tested Platforms:

  • Windows 10 x64

Tested Firefox versions:

  • Firefox Release 76.0b8 en-US (Build ID: 20200422191054)
You need to log in before you can comment on or make changes to this bug.