Closed Bug 1666373 Opened 1 month ago Closed 1 month ago

Use experiment manager to control wizardless password import experiment

Categories

(Firefox :: Messaging System, enhancement)

enhancement

Tracking

()

VERIFIED FIXED
83 Branch
Iteration:
83.1 - Sept 21 - Oct 4
Tracking Status
firefox82 - verified
firefox83 --- verified

People

(Reporter: Mardak, Assigned: Mardak)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

We need to turn on event recording as well as only activate the direct import behavior if the experiment says so.

Use ExperimentAPI instead of prefs to enable event recording and feature behavior. Depends on D90635.

Depends on D90635

Pushed by elee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/296c8ba967c9
Use experiment manager to control wizardless password import experiment r=sfoster,andreio
Depends on: 1666850
Pushed by elee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/666ea3fc3f1c
Use experiment manager to control wizardless password import experiment r=sfoster,andreio
Status: NEW → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
Target Milestone: --- → 83 Branch

[Tracking Requested - why for this release]: The wizardless password import autocomplete experiment is scheduled for 82

QA steps: Note, this experiment is different from the previous experiment bug 1618311 in 77 which controlled experiment with a pref -- this is now controlled with experiment manager.

  1. install remote settings devtool https://github.com/mozilla-extensions/remote-settings-devtools/releases/download/1.2.1/remote-settings-devtools@mozilla.com-1.2.1-signed.xpi
  2. click remote settings devtool icon in the top right toolbar
  3. switch top right environment dropdown to "Stage (preview)"
  4. find row for "main-preview/nimbus-desktop-experiments"
  5. click "Force Sync" on the right of that row
  6. restart firefox
  7. navigate to a page with login field, e.g,. bugzilla home page

Expect:

  • about:telemetry -> Environment Data -> Experiments should show "password-autocomplete-wizardless.branch" with either "control" or "treatment" value
  • about:telemetry -> Events should have a row like exp_import impression formfill {"autofillResult": "4", "loginsCount": "0"} matching the previous fx77 experiment for each page with login field for both "control" and "treatment" branches
  • if also testing bug 1665802, about:telemetry -> Events should have a row for exp_import event click chrome {"profilesCount": "1"} when clicking on an import autocomplete for both "control" and "treatment" branches (but the wizard-or-not behavior will depend on the branch)

The "stage (preview)" experiment is set up as 50% control/50% treatment with no targeting or sampling, so that will change with the final deployed experiment, which will have most users not enrolled. The unenrolled behavior would be as if skipping steps 1-6 above, and that should result in none of the "exp_import" event telemetry getting recorded and behavior would be to always show the migration wizard as in 81.

Iteration: --- → 83.1 - Sept 21 - Oct 4
Flags: needinfo?(edilee) → needinfo?(cmuresan)

I have verified that the expected results, Environment Data, impression, pop-up, and click pings are collected in both the control and treatment branches, and have also verified that the wizard-less password import is successful in the treatment branch. The verification was performed on the latest Nightly build 83.0a1 (BuildID 20200927215113) using Windows 10 and macOS 10.15.

Status: RESOLVED → VERIFIED
Flags: needinfo?(cmuresan)

Comment on attachment 9177100 [details]
Bug 1666373 - Use experiment manager to control wizardless password import experiment r?sfoster,andreio

Beta/Release Uplift Approval Request

  • User impact if declined: Unable to run experiment to improve retention with a streamlined password import flow
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: Comment 6
  • List of other uplifts needed: Bug 1666373, Bug 1665802
  • Risk to taking this patch: Medium
  • Why is the change risky/not risky? (and alternatives if risky): Bug 1666373 gets experiment manager working for new profiles and bug 1665802 adds the new functionality to control with the changes here.

The code here touches LoginHelper which is used on multiple platforms including android and adds a new dependency on Experiment Manager, which we discovered issues like bug 1666373.

  • String changes made/needed: None
Attachment #9177100 - Flags: approval-mozilla-beta?
Flags: qe-verify+
QA Whiteboard: [qa-triaged]

Comment on attachment 9177100 [details]
Bug 1666373 - Use experiment manager to control wizardless password import experiment r?sfoster,andreio

approved for 82.0b6

Attachment #9177100 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

I have verified that the expected results, Environment Data, impression, pop-up, and click pings are collected in both the control and treatment branches, and have also verified that the wizard-less password import is successful in the treatment branch on the latest Beta build 82.0b6 (BuildID 20201001171107) using Windows 10 and macOS 10.15.

Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.