Closed Bug 1730618 Opened 1 month ago Closed 1 month ago

Allow Nimbus to enable Firefox Suggest online and override the offline default

Categories

(Firefox :: Address Bar, task, P1)

task
Points:
3

Tracking

()

VERIFIED FIXED
94 Branch
Iteration:
94.1 - Sep 6 - Sept 19
Tracking Status
firefox92 --- verified
firefox93 --- verified
firefox94 --- verified

People

(Reporter: adw, Assigned: adw)

References

Details

Attachments

(2 files)

With bug 1729776, we now default-enable Firefox Suggest offline (for US en users), which means we set the two prefs that control Firefox Suggest suggestions to true. Nimbus doesn't have any way of setting them back to false for the online scenario.

This reworks the fix to bug 1729776.

Currently Nimbus doesn't have a way to force the two suggestions prefs [1] on or
off. That might seem surprising since we've run experiments already. Initially
we defaulted the two prefs to true but we defaulted the separate feature-gate
pref [2] to false, and it was the feature-gate pref we controlled via Nimbus. At
some point we changed the defaults to false and then in Firefox flipped them to
true after showing the onboarding dialog. As a result we've never needed to
override the two suggestion prefs via Nimbus.

The problem now is that we default-enable offline (for US en users), so we set
all three prefs to true. For the online rollout, we need to keep the
feature-gate pref enabled but disable the suggestion prefs, and there's no way
to do that.

My first idea was to add new Nimbus variables to override the two suggestion
prefs. The prefs would keep their default true values but be overridden by
Nimbus. But that doesn't work because there's no way for Firefox to tell whether
the prefs are true because the user has opted in (overriding Nimbus) or because
they still have their default values. Setting the prefs to true on the user
branch doesn't have any effect because they're also true on the default branch.
Or maybe there's a way I don't know about to force them to true on the user
branch, but even if there were, it seems brittle to rely on a value being set on
the user branch to distinguish between the two cases. (This is a potential
problem for any prefs that are controlled by both Nimbus and the user. So far
the prefs we've been using via Nimbus have all been hidden feature-gate-type
things and implementation details.)

We already have a quickSuggestScenario variable. We currently use it only to
tell what we should send in the telemetry ping (bug 1729576) and whether some
parts of the prefs UI should be shown. This revision makes it much more
important by treating it as the source of truth for the user's scenario. It now
determines the default values of related prefs, including the two suggestions
prefs.

The logic is:

If quickSuggestScenario is non-null:
  scenario = quickSuggestScenario
Else (e.g., there's no rollout):
  If the user is US en:
    scenario = offline
  Else:
    scenario = history

After determining the scenario, it's set it as
browser.urlbar.quicksuggest.scenario on the default branch. There's an
existing pref observer in UrlbarPrefs, and I added a case for this pref so that
when it's updated we also update all the other prefs that depend on the
scenario. This way when the pref is set -- either due to Nimbus update or by
changing it on about:config -- all the other prefs stay in sync.

I kept the default value of browser.urlbar.quicksuggest.scenario but removed
it as the fallback for quickSuggestScenario. If it still both had a default
and remained the fallback, then it would be impossible to tell when Nimbus is
trying to override it, because any fetch of the value from Nimbus would just
return the fallback pref's value if there is no override.

I considered instead removing the default value and keeping it as the fallback.
The drawback of that is that unenrollments would not take effect until restart.
I actually tried this approach first, and in tests, after mock experiments were
unenrolled, the pref values remained what they were when the experiment was
active.

It might also be possible to not have the browser.urlbar.quicksuggest.scenario
pref at all. We could call NimbusFeatures directly to get the scenario. However,
currently we cache and access Nimbus variables through UrlbarPrefs, as we do
with prefs, and I don't want to add an inconsistency.

This revision also fixes bug 1730596 since it was easy to do given that I needed
a way to prevent indirect recursive updates to the scenario, and I can use that
for bug 1730596 too (the _updatingFirefoxSuggestScenario bool).

[1] browser.urlbar.suggest.quicksuggest and browser.urlbar.suggest.quicksuggest.sponsored
[2] browser.urlbar.quicksuggest.enabled

Pushed by dwillcoxon@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d79cc34cdb51
Allow Nimbus to enable Firefox Suggest online and override the offline default. r=mythmon,preferences-reviewers,mstriemer
Duplicate of this bug: 1730596
Flags: qe-verify+
Flags: in-testsuite+
Status: ASSIGNED → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
Target Milestone: --- → 94 Branch

STR for QA

The fix for this bug should not cause any obvious or immediate changes. For US users with an en profile, Firefox Suggest offline should still be enabled by default, and for other users it should remain disabled.

So first, please repeat the STR from bug 1729776 comment 3, both the "US region & en locales" case and the "Non-US region & non-en locales" case.

The purpose of this bug is to allow us to enable the online scenario using Nimbus. Ideally you would verify it by actually using Nimbus. If that's not possible right now or you're unsure how to do it, please let me and mythmon know. If you're having trouble QA'ing using Nimbus, we should fix that ASAP. (I don't actually know how to do it but I can ask mythmon.)

If you can't use Nimbus, you can also verify by manually setting some prefs. I'll provide STR for both processes just in case. If there's a large delay in getting Nimbus set up for you -- like, a day -- then please go ahead and verify this by setting prefs manually and then come back once Nimbus is working.

STR:

  1. As mentioned, first please repeat the STR from bug 1729776 comment 3 for both cases

  2. In the remaining steps, continue using the US en profile so that Firefox Suggest offline is enabled by default (or start with a new US en one if it's easier, that's fine too)

  3. When using Nimbus:

    1. With Firefox running, enable the online scenario using Nimbus. Don't restart Firefox afterward. (I'm assuming that Firefox will receive the rollout from Nimbus immediately and not require a restart. I haven't used Nimbus before, so if a restart is actually required, please go ahead.)
    2. Open about:telemetry#events-tab and verify that there are no events in the contextservices.quicksuggest category

    When setting prefs manually:

    1. Set browser.urlbar.quicksuggest.scenario = "online"
    2. Open about:telemetry#events-tab and verify that there are no events in the contextservices.quicksuggest category
    3. Restart
    4. Set browser.urlbar.quicksuggest.enabled = true
  4. Verify the following prefs are set. They should not be bold, i.e., they should be default values -- except for browser.urlbar.quicksuggest.enabled if you just set it manually in the previous step.

browser.urlbar.quicksuggest.enabled: true
browser.urlbar.quicksuggest.shouldShowOnboardingDialog: true
browser.urlbar.suggest.quicksuggest: false
browser.urlbar.suggest.quicksuggest.sponsored: false
  1. Restart
  2. Verify the online onboarding dialog is shown after startup
  3. Click one of the controls in the dialog and verify that the expected behavior is correct. e.g., if you click the Allow button, verify that sponsored and non-sponsored suggestions are shown and the checkboxes in about:preferences#privacy are checked, etc.
  4. Restart and verify that you are still opted in or out, depending on which control you clicked in step 7. e.g., if you clicked Allow, then you should still be opted in to sponsored and non-sponsored suggestions.

For this bug by itself, the important things are verifying the prefs in step 4, the onboarding in step 6, and the behavior after restart in step 8. Ultimately when all bugs have been fixed and the patches have been uplifted, I'd like to coordinate with you to fully test both offline and online end-to-end.

I have verified this issue using the latest Nightly 94.0a1 (Build ID: 20210917093507) build on Windows 10 x64, macOS 11.6, and Linux Ubuntu 20.04 x64.

I have repeated the steps from bug 1729776 comment 3 for both the "US region & en locales" and the "Non-US region & non-en locales" cases. Everything went as expected.

  • I have used the "US region & en locales" build where I set "nimbus.debug" pref to "true".
  • I have forced enrolled in the online branch using the treatment link from the recipe.
  • Without restarting, I have verified that there are no "contextservices.quicksuggest" events in the telemetry panel.
  • I have verified that the preferences are set as default:

browser.urlbar.quicksuggest.enabled: true
browser.urlbar.quicksuggest.shouldShowOnboardingDialog: true
browser.urlbar.suggest.quicksuggest: false
browser.urlbar.suggest.quicksuggest.sponsored: false

  • I have verified that the Onboarding modal is displayed after restarting the browser.

  • I have verified that after accepting the suggestions, the QS feature is enabled.

  1. The sponsored and non-sponsored results are displayed.
  2. The checkboxes from the Privacy pane are selected.
  • I have restarted the browser and confirmed that the QS feature remained enabled.

  • I have verified that the "Not now" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.

  • I have verified that the "Customize in settings" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.

  • I have verified that the "Learn more" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.
Status: RESOLVED → VERIFIED

Approval Request Comment
[Feature/Bug causing the regression]: Firefox Suggest offline/online rollouts
[User impact if declined]: Needed for important rollouts on 93 and 92
[Is this code covered by automated tests?]: Yes
[Has the fix been verified in Nightly?]: Yes
[Needs manual test from QE? If yes, steps to reproduce]: Please see 7
[List of other uplifts needed for the feature/fix]: See uplift coordination spreadsheet
[Is the change risky?]: Medium risk relative to other uplifts needed in the patch stack
[Why is the change risky/not risky?]: This is one of the more substantial patches in the uplift stack. It sets several prefs on startup depending on a Nimbus variable and the user's region and locale. The prefs are all related to Firefox Suggest. There are no user-visible changes with this patch alone. (A previous similar patch in bug 1729776 that has already been uplifted enabled the Firefox Suggest scenario by default for en-* users in the US. This patch keeps that behavior and doesn't make any new user-visible changes.) The fix has been verified by QA on Nightly. Note this patch is slightly different from the one that landed on m-c because some other unrelated bugs landed on m-c in the meantime. The differences between the patches are not significant. I've manually verified this patch locally on 93/beta.
[String changes made/needed]:

Attachment #9241856 - Flags: approval-mozilla-release?
Attachment #9241856 - Flags: approval-mozilla-beta?

Comment on attachment 9241856 [details] [diff] [review]
93/beta & 92/release patch

Approved for 93.0b7.

Attachment #9241856 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
QA Whiteboard: [qa-triaged]

I have verified this issue using the latest Beta 93.07 (Build ID: 20210919190049) build on Windows 10 x64, macOS 11.6, and Linux Ubuntu 20.04 x64.

I have repeated the steps from bug 1729776 comment 3 for both the "US region & en locales" and the "Non-US region & non-en locales" cases. Everything went as expected.

  • I have used the "US region & en locales" build where I set "nimbus.debug" pref to "true".
  • I have forced enrolled in the online branch using the treatment link from the recipe.
  • Without restarting, I have verified that there are no "contextservices.quicksuggest" events in the telemetry panel.
  • I have verified that the preferences are set as default:

browser.urlbar.quicksuggest.enabled: true
browser.urlbar.quicksuggest.shouldShowOnboardingDialog: true
browser.urlbar.suggest.quicksuggest: false
browser.urlbar.suggest.quicksuggest.sponsored: false

  • I have verified that the Onboarding modal is displayed after restarting the browser.

  • I have verified that after accepting the suggestions, the QS feature is enabled.

  1. The sponsored and non-sponsored results are displayed.
  2. The checkboxes from the Privacy pane are selected.
  • I have restarted the browser and confirmed that the QS feature remained enabled.

  • I have verified that the "Not now" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.

  • I have verified that the "Customize in settings" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.

  • I have verified that the "Learn more" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.

Comment on attachment 9241856 [details] [diff] [review]
93/beta & 92/release patch

Approved for 92.0.1.

Attachment #9241856 - Flags: approval-mozilla-release? → approval-mozilla-release+

I have repeated the steps from bug 1729776 comment 3 for both the "US region & en locales" and the "Non-US region & non-en locales" cases. Everything went as expected.

  • I have used the "US region & en locales" build where I set "nimbus.debug" pref to "true".
  • I have forced enrolled in the online branch using the treatment link from the recipe.
  • Without restarting, I have verified that there are no "contextservices.quicksuggest" events in the telemetry panel.
  • I have verified that the preferences are set as default:

browser.urlbar.quicksuggest.enabled: true
browser.urlbar.quicksuggest.shouldShowOnboardingDialog: true
browser.urlbar.suggest.quicksuggest: false
browser.urlbar.suggest.quicksuggest.sponsored: false

  • I have verified that the Onboarding modal is displayed after restarting the browser.

  • I have verified that after accepting the suggestions, the QS feature is enabled.

  1. The sponsored and non-sponsored results are displayed.
  2. The checkboxes from the Privacy pane are selected.
  • I have restarted the browser and confirmed that the QS feature remained enabled.

  • I have verified that the "Not now" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.

  • I have verified that the "Customize in settings" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.

  • I have verified that the "Learn more" button doesn't enable the QS feature.

  1. The sponsored and non-sponsored results are not displayed.
  2. The checkboxes from the Privacy pane are not selected.
  • I have restarted the browser and confirmed that the QS feature remains disabled.
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.