1733687 - Store the user's response to the Firefox Suggest onboarding opt-in dialog/modal

Status: VERIFIED FIXED

(Firefox :: Address Bar, task, P2)

Description

[Drew Willcoxon :adw]

We should store the user's response to the Firefox Suggest opt-in dialog somewhere in Firefox (prefs?). That would allow us to make better per-user decisions in the future about potentially showing the dialog again and probably other things too. We do have telemetry corresponding to each of the ways the dialog can be dismissed, but we should also have something accessible within the product that will enable Firefox to have different UXes for different users depending on whether/how they opted in.

Currently all we do when the user opts in is set the two browser.urlbar.suggest.quicksuggest prefs to true. If the user does not opt in, we don't set or record anything other than telemetry. That means we don't have a way to distinguish between these two cases:

1. The user declines the opt-in
2. The user accepts the opt-in but later disables suggestions in about:preferences

Or for that matter:

1. The user accepts the opt-in
2. The user declines the opt-in but later enables suggestions in about:preferences

And maybe more importantly, we don't store exactly how the user declined the opt-in: whether they clicked the not-now link, the customize button, or the learn-more link. If they clicked the not-now link for example, there may be an expectation that the dialog will be shown again sometime later.

The DOH rollouts tried various opt-ins and recorded a history/state of the user's choices, so we should look to them as an example.

Comment 1

[Drew Willcoxon :adw]

DoH uses several prefs. It actually looks more complex than Firefox Suggest in terms of number of prefs and how they interact. The most relevant ones for us AFAICT as examples of storing user opt-in are below, and I don't think there's anything surprising or unconventional.

• doh-rollout.doorhanger-decision: String values like UIOk, UIDisabled. Looks like these are left over from when the rollout was implemented in an extension and aren't actually set in m-c.
• doh-rollout.self-enabled: Seems to be the actual switch for whether the feature is enabled. Set to true when doorhanger-decision is UIOk, cleared when it's UIDisabled (source).
• doh-rollout.mode: An integer value that AFAICT effectively indicates whether the rollout is enabled or disabled. 0 is disabled, 2 is enabled. Undefined looks like maybe an indication that state needs to be reset.

Sources:

• https://searchfox.org/mozilla-central/rev/c3d7964c593e0bedabea2fea0b35ba243cf9e696/browser/components/doh/DoHController.jsm
• https://searchfox.org/mozilla-central/rev/c3d7964c593e0bedabea2fea0b35ba243cf9e696/browser/components/doh/DoHConfig.jsm

Comment 2

[Drew Willcoxon :adw]

Attachment: Bug 1733687 - Part 1: Dispatch an event to content with a flag when a SubDialog is closed by pressing the Escape key.

In bug 1733687 we'd like to be able to tell when the Firefox Suggest subdialog
is closed by pressing the Escape key as opposed to the only other obvious way of
dismissing it, by clicking the "Not now" link inside it. There doesn't seem to be
an easy way to do that right now because content isn't notified in this case.

I followed the example of aborting the dialog, where we set _closingEvent to
an event with { detail: { abort: true }}. When Escape is pressed, I similarly
set _closingEvent with { detail: { escapeKeyPressed: true }}. We dispatch
the event to content on close just like we do with abort events.

Comment 3

[Drew Willcoxon :adw]

Attachment: Bug 1733687 - Store in a pref the user's response to the Firefox Suggest onboarding dialog and add two new telemetry event objects.

This does a couple of things:

• Instead of using the not_now telemetry event object for the cases where the
  dialog is closed by the Escape key or some other atypical way, reserve
  not_now specifically for clicks on the "Not now" link and add two new
  objects: dismissed_escape_key and dismissed_other. That should give us a
  little better understanding of how the dialog is being dismissed.
• Add a new browser.urlbar.quicksuggest.onboardingDialogChoice pref that
  stores exactly the same values as the telemetry event. e.g., if the dialog is
  accepted, then we'll record a telemetry event whose object is accept and
  we'll also store accept in the pref.

I figure if we decide to show the onboarding again for people who have already
seen it, (1) we'll use this pref to decide the flow for any given user, and (2)
we'll need to add another pref for the user's response to the "v2" dialog, or
maybe we could morph this one into an array of responses or something more
complex like that.

Comment 4

[Drew Willcoxon :adw]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5427936cafa4b9797c91587f575e7300c1d82cf3
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c9f82a1f68f12a3dfa7d85d74821c9c8e8bc79e4

Comment 5

[Drew Willcoxon :adw]

Attachment: request.md

Comment 6

[Teon Brooks [:teon]]

Comment on attachment 9244597 [details]
request.md

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file Events.yaml and the Probe Dictionary.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Drew Willcoxon is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---

Result: datareview+

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cf36885c69e9
Store in a pref the user's response to the Firefox Suggest onboarding dialog and add two new telemetry event objects. r=dao,mythmon

Comment 8

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/cf36885c69e9

Comment 9

[Drew Willcoxon :adw]

STR for QA

1. Enroll in Firefox Suggest online
2. Restart Firefox to trigger the online opt-in onboarding modal dialog
3. Click the Allow button
4. Open about:preferences and verify that browser.urlbar.quicksuggest.onboardingDialogChoice is accept
5. Please repeat the STR for the other controls in the dialog: the Customize button, Learn more link, and Not now link
6. Please repeat the STR but instead of clicking a button, press: the Enter key to accept the dialog, and the Escape key to dismiss the dialog
7. Please repeat the STR but instead of clicking a button, use a keyboard shortcut to quit Firefox (e.g., Command-Q on Mac). The dialog should close and another dialog should open that asks if you want to quit Firefox. You can cancel that quit dialog.

Depending on how you closed the dialog, the browser.urlbar.quicksuggest.onboardingDialogChoice pref should have the following values:

• Allow button: accept
• Customize button: settings
• Learn more link: learn_more
• Not now link: not_now_link
• Enter key: accept
• Escape key: dismissed_escape_key
• Quit dialog opened: dismissed_other

Comment 10

[Drew Willcoxon :adw]

Comment on attachment 9244055 [details]
Bug 1733687 - Store in a pref the user's response to the Firefox Suggest onboarding dialog and add two new telemetry event objects.

Beta/Release Uplift Approval Request

• User impact if declined: This is required for Firefox Suggest rollouts.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: Please see comment 9
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This change simply updates a pref when the user interacts with the Firefox Suggest onboarding dialog. Has thorough automated tests.
• String changes made/needed:

Comment 11

[Ryan VanderMeulen (PTO, back 6-April)]

Comment on attachment 9244055 [details]
Bug 1733687 - Store in a pref the user's response to the Firefox Suggest onboarding dialog and add two new telemetry event objects.

Approved for 94.0b4.

Comment 12

[Ryan VanderMeulen (PTO, back 6-April)]

https://hg.mozilla.org/releases/mozilla-beta/rev/6348c24c98c8

Comment 13

[Cosmin Muntean [:cmuntean], Ecosystem QA]

I have verified this issue on the latest Beta 94.0b4 (Build ID: 20211010185747) and the latest Nightly 95.0a1 (Build ID: 20211010214947) on Windows 10 x64, macOS 10.15.7 and Ubuntu 20.04 x64.

• The "browser.urlbar.quicksuggest.onboardingDialogChoice" pref, the "browser.urlbar.quicksuggest.onboardingDialogChoice" preference, and the "contextservices.quicksuggest" event are correctly registered for each scenario and have the correct values as described in comment 9.