1740818 - Data collection can be disabled when upgrading Firefox after having opted in to online

Status: VERIFIED FIXED

(Firefox :: Address Bar, defect, P1)

Description

[Drew Willcoxon :adw]

Carmen and her team caught a bug shown in this video where you opt in to online, upgrade Firefox, and then the data collection toggle is off but it should be on.

STR:

1. Start with 94.0.1
2. Set nimbus.debug=true
3. Enroll in the offline vs. online treatment branch (i.e., online)
4. Restart
5. Verify the opt-in modal appears
6. Click Allow suggestions
7. Verify both Fx Suggest checkboxes are checked in about:prefs
8. Quit
9. Start Nightly with the same profile
10. Open about:preferences#privacy

Actual: Data collection toggle is off
Expected: Data collection toggle should be on

QA was able to reproduce this only on a couple of Ubuntu 20.04 machines but not on other Linux machines or Windows or Mac.

I can't reproduce it on Ubuntu 20.04.3 LTS or Mac, but I think I see how it could happen. It's a technical problem due to timing. You have to be unlucky to hit it, but I can't say how how likely it is. It's not related to Ubuntu or your platform and I bet if you tried it on enough machines you'd eventually hit it on Windows or Mac too. Carmen mentioned that when the problem happens, browser.search.region is not present before step 4, but I don't think that's related either.

Technical description below:

I think the problem is due to a race: The Nimbus urlbar feature may not be initialized by the time we migrate Suggest preferences. The data collection pref is set here -- and only here -- so we're not hitting that path like we should be. I think that's because scenario is "offline" instead of "online" because the Nimbus urlbar feature hasn't finished initializing. When I add an artificial delay right before this line in the Nimbus client implementation, then I can reproduce it. i.e., add this right before the await this._store.load():

        const { setTimeout } = ChromeUtils.import(
          "resource://gre/modules/Timer.jsm"
        );
        await new Promise(r => setTimeout(r, 3000));

Comment 1

[Drew Willcoxon :adw]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d1580bae61c345752b9d5b74756d0a12e5cbf444

Comment 2

[Drew Willcoxon :adw]

Carmen, I'm guessing you're more likely to hit this on slow machines, especially with slow disks. Would you say your two Ubuntu machines/VMs are slower or have slower disks than your others?

Could you please try the following builds? I made them for all platforms just in case but I know you said it only happens on Linux.

• Linux: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DpAqUY3aQdGDcrACHJHLbQ/runs/0/artifacts/public/build/target.tar.bz2
• Windows: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/FTd3GTpDSb2dvjas7t8Exw/runs/0/artifacts/public/build/target.zip
• Mac: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ZuiBVcQwQnuqSuHIKGiitA/runs/0/artifacts/public/build/target.dmg

Here's the Treeherder link for the builds in case you need it.

These builds won't fix profiles that have already hit the problem, but they should prevent it in the first place.

If the problem still happens, then could you please open the browser console, filter on messages with "XXXadw", and copy-paste them here?

Comment 3

[Carmen Fat [:cfat] - Ecosystem QA]

• Yes, the Ubuntu machines are indeed slower compared to the other machines where we did not encounter this issue.
• Following the STR and the try-build provided in comment 2 fixes the issue: the data collection toggle is now on and the browser.urlbar.quicksuggest.dataCollection.enabled pref is correctly set to true.

Comment 4

[Drew Willcoxon :adw]

Attachment: Bug 1740818 - Wait for the urlbar Nimbus feature to be initialized before updating the Firefox Suggest scenario and performing pref migrations.

The problem here is a race: Even though the user is enrolled in the online
scenario via Nimbus, the Nimbus urlbar variables may not be initialized by the
time we migrate Suggest preferences on startup. The data-collection pref is set
when we migrate prefs -- and only there --
so we're not hitting that path like we should be. That's because the scenario
can still be the default value of "offline".

Fortunately the Nimbus client API gives us a ready promise we can await that is
resolved when the feature is loaded and ready, so all we need to do is await it
before trying to update the scenario and do migrations.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5727efb9600d
Wait for the urlbar Nimbus feature to be initialized before updating the Firefox Suggest scenario and performing pref migrations. r=nanj

Comment 6

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/5727efb9600d

Comment 7

[Drew Willcoxon :adw]

Comment on attachment 9250548 [details]
Bug 1740818 - Wait for the urlbar Nimbus feature to be initialized before updating the Firefox Suggest scenario and performing pref migrations.

Beta/Release Uplift Approval Request

• User impact if declined: We need this for the Firefox Suggest preferences redesign targeting 95/94.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: Please see the comments here in the bug
• List of other uplifts needed: Please see uplift spreadsheet: https://docs.google.com/spreadsheets/d/1LavihS-VOPFYEyum7mrx6FKXmuQeHi9xQHfGNSxjnoY/edit?usp=sharing
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is a one-line change that fixes a timing-related bug. This code path is covered by automated tests, but this bug specifically does not have a test because it's difficult to make one.
• String changes made/needed:

Comment 8

[Pascal Chevrel:pascalc (PTO until April 26)]

Comment on attachment 9250548 [details]
Bug 1740818 - Wait for the urlbar Nimbus feature to be initialized before updating the Firefox Suggest scenario and performing pref migrations.

Approved for 95 beta 8, thanks.

Comment 9

[Pascal Chevrel:pascalc (PTO until April 26)]

https://hg.mozilla.org/releases/mozilla-beta/rev/6fcee12cf437

Comment 10

[Drew Willcoxon :adw]

[Tracking Requested - why for this release]: We need this for the Firefox Suggest preferences redesign targeting 95/94.

Comment 11

[Drew Willcoxon :adw]

Comment on attachment 9250548 [details]
Bug 1740818 - Wait for the urlbar Nimbus feature to be initialized before updating the Firefox Suggest scenario and performing pref migrations.

Beta/Release Uplift Approval Request

• User impact if declined: We need this for the Firefox Suggest preferences redesign targeting 95/94.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: Please see the comments here in the bug
• List of other uplifts needed: Please see uplift spreadsheet: https://docs.google.com/spreadsheets/d/1LavihS-VOPFYEyum7mrx6FKXmuQeHi9xQHfGNSxjnoY/edit?usp=sharing
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is a one-line change that fixes a timing-related bug. This code path is covered by automated tests, but this bug specifically does not have a test because it's difficult to make one.
• String changes made/needed:

Comment 12

[Carmen Fat [:cfat] - Ecosystem QA]

• I have verified this bug on the latest Nightly 96.0a1 (Build ID: 20211116212601), Beta 95.0b8 (Build ID: 20211116185800) and 94.0.2 try build (Build ID: 20211115193242) on the two Ubuntu 20.04 x64 machines where the issue was previously reproduced.
• The data collection toggle is now ON and the browser.urlbar.quicksuggest.dataCollection.enabled pref is correctly set to true for the following scenarios:

1. Opt in to online scenario -> Upgrade Firefox -> Data collection toggle is ON
2. Enroll in experiment -> Select Not now option from modal -> manually check Contextual suggestions option -> Upgrade Firefox -> Data collection toggle is ON

Comment 13

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9250548 [details]
Bug 1740818 - Wait for the urlbar Nimbus feature to be initialized before updating the Firefox Suggest scenario and performing pref migrations.

Approved for 94.0.2.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/a4835d8e5832

Comment 15

[Carmen Fat [:cfat] - Ecosystem QA]

• I have verified this bug on the Firefox 94.0.2 candidate build (Build ID: 20211117154346) on the two Ubuntu 20.04 x64 machines where the issue was previously reproduced.
• The data collection toggle is now ON and the browser.urlbar.quicksuggest.dataCollection.enabled pref is correctly set to true for the following scenarios:

1. Opt in to online scenario -> Upgrade Firefox -> Data collection toggle is ON
2. Enroll in experiment -> Select Not now option from modal -> manually check Contextual suggestions option -> Upgrade Firefox -> Data collection toggle is ON.