1714304 - [Phase 2 & Phase 3] The “contextservices.quicksuggest” enable/disable telemetry event is not registered after a restart

Status: VERIFIED FIXED

(Firefox :: Address Bar, defect, P1)

Description

[Maria Heres, inactive, contact Ecosystem QA]

Attachment: telemetry.gif

[Affected Versions]:

• Firefox Release 89.0 (Build ID: 20210527174632)
• Firefox Beta 90.0b1 (Build ID: 20210601190019)
• Firefox Nightly 91.0a1 (Build ID: 20210602214447)

[Affected Platforms]:

• Windows 10
• macOS 10.15.7
• Ubuntu 20.04

[Prerequisites]:

• Have Firefox open.
• From the “about:config” page, have the following preferences set:
  firefoxSuggestLabelsEnabled set to true
quickSuggestEnabled set to true
quickSuggestShouldShowOnboardingDialog set to false
quickSuggestShowOnboardingDialogAfterNRestarts set to 2
browser.search.region set to “US”.

[Steps to reproduce]:

1. Restart Firefox.
2. Navigate to “about:preferences#search”.
3. Check and uncheck “Show search suggestions ahead of browsing history in address bar results”.
4. Navigate to “about:telemetry#events-tab” and observe the events.

[Expected result]:

• The “contextservices.quicksuggest” “enabled” and “disabled” events are displayed.

[Actual result]:

• The “contextservices.quicksuggest” “enabled” and “disabled” events are not displayed.

[Notes]:

• The events are displayed if attempted before restart.
• Attached is a recording of the issue.

Comment 1

[Nan Jiang [:nanj]]

To be clear, this event would be only triggered on "Show Firefox Suggest in the address bar (suggested and sponsored results)", not "Show search suggestions ahead of browsing history in address bar results".

I just verified locally, the events were recorded as expected when I toggled this pref.

Closing this as invalid. Feel free to reopen if it still doesn't work for you.

Edit: sorry, taking a closer look at the screencast, it does click on the right pref. Reopened it for further investigation.

Comment 2

[Nan Jiang [:nanj]]

I managed to reproduce this in beta 90 with the following steps:

• Create a new profile
• Force enroll into the treatment branch of the V2 rollout as follows:
  • Flip nimbus.debug to enable Nimbus debugging mode
  • Copy and paste about:studies?optin_slug=firefox-suggest-v2&optin_branch=treatment&optin_collection=nimbus-preview in the urlbar and hit enter
• Open "about:telemetry#events-tab"
• Open "about:preferences#search" and uncheck "QuickSuggest"
• Restart the browser
• Check "QuickSuggest" again on the about:preferences page

:adw - It's weird that this toggle event was recorded in the onPrefChanged() handler, in this case, somehow it didn't get to there. I wonder if it's related to this line?

I've also tested it with my patch for bug 1714344, it seems to work just fine.

Comment 3

[Drew Willcoxon :adw]

Investigating

Comment 4

[Drew Willcoxon :adw]

AFAICT so far, when I do the steps in comment 2 on an m-c build, NimbusFeatures.urlbar.getVariable("quickSuggestEnabled") is false right on startup but then at some point switches to true. The problem is that none of our NimbusFeatures.urlbar.onUpdate() callbacks are called -- not the one in UrlbarPrefs or the one in UrlbarProviderQuickSuggest -- which means that UrlbarProviderQuickSuggest does not enable event telemetry. So when you go to about:preferences#search, quickSuggestEnabled is true and we show the checkbox, and when you toggle the checkbox UrlbarProviderQuickSuggest correctly adds the event, but event recording remains disabled so it's not actually recorded.

Did something change recently WRT to Nimbus startup and/or its onUpdate callbacks?

Comment 5

[Drew Willcoxon :adw]

Or maybe this just never worked right after restarting.

Comment 6

[Nan Jiang [:nanj]]

Did something change recently WRT to Nimbus startup and/or its onUpdate callbacks?

:andreio, do you know anything about this?

Also, would force enrollment have any side-effects? Such as not triggering any callbacks such as NimbusFeatures.urlbar.onUpdate()?

Comment 7

[Drew Willcoxon :adw]

Maria, Nan and I can reproduce the bug with Nan's STR in comment 2, but we can't with your original STR including the prerequisites that involve setting prefs. I tested with 89, 90 Beta, and mozilla-central.

I noticed in your video that you have an about:studies tab open. Is it possible that you enabled the feature by enrolling in the experiment similar to Nan's STR and you did not manually set any prefs?

Comment 8

[Andrei Oprea [:andreio]]

We're fixing this in Nimbus API atm in Bug 1714449 and will uplift to 90.
I think what's happening here based on the STR is that we do not fire an update event for the urlbar feature after the experiment is loaded from disk.

Comment 9

[Maria Heres, inactive, contact Ecosystem QA]

(In reply to Drew Willcoxon :adw from comment #7)

Maria, Nan and I can reproduce the bug with Nan's STR in comment 2, but we can't with your original STR including the prerequisites that involve setting prefs. I tested with 89, 90 Beta, and mozilla-central.

I noticed in your video that you have an about:studies tab open. Is it possible that you enabled the feature by enrolling in the experiment similar to Nan's STR and you did not manually set any prefs?

This was tested using the Nimbus study we use for testing Phase 2, with the prerequisites being the preferences I am told were initially used in the recipe. The name of the about:preferences#search option was a mistake in the original STR, it should indeed be "Show Firefox Suggest in the address bar (suggested and sponsored results)".

Comment 10

[Nan Jiang [:nanj]]

:cmuntean confirmed that this issue was also reproducible with the regular enrollment.

I've communicated the impact with Natalie, no further action required for now. We will let Nimbus fix bug 1714440 and get it uplifted to 90.

Comment 11

[Drew Willcoxon :adw]

Maria, can you confirm that at least the sponsored results appear in the address bar in this scenario?

Comment 12

[Maria Heres, inactive, contact Ecosystem QA]

(In reply to Drew Willcoxon :adw from comment #11)

Maria, can you confirm that at least the sponsored results appear in the address bar in this scenario?

The Sponsored results are displayed in the address bar, before and after the restart, both for the version where we use Nimbus and a VPN set on US (user.js used) and in which the event is not displayed (issue reproducible), as well as by setting the preferences directly, when the event is displayed (issue not reproducible), where the preferences are:
browser.urlbar.quicksuggest.enabled set to true.
browser.urlbar.experimental.firefoxSuggestLabels.enabled set to true.
browser.urlbar.quicksuggest.showedOnboardingDialog to true.
browser.startup.upgradeDialog.version to 89.

Tested on Windows 10 using Firefox 89.0 (Build ID: 20210527174632), Firefox Beta 90.0b1 (Build ID: 20210601190019), Firefox Nightly 91.0a1 (Build ID: 20210603094827), where Beta and Nightly had the channel changed to release when enrolling in the Nimbus recipe.

Comment 13

[Maria Heres, inactive, contact Ecosystem QA]

After further investigation, the Nightly version used in my last comment had not been recent enough, it seems the suggestions are not displayed after a restart if using the Nimbus recipe on a more recent Nightly version. Bug 1715500 has been filed for this behavior.

Comment 14

[Maria Heres, inactive, contact Ecosystem QA]

The issue is no longer reproducible. The telemetry event for toggling the option on and off is displayed after a restart. Tested on Windows 10, Linux Mint 20 and macOS 11.1 using Firefox Nightly 91.0a1 (Build ID: 20210618092056) while naturally enrolled in phase 2.

Comment 15

[Drew Willcoxon :adw]

Removing myself as the assignee since this was fixed by other bugs and I didn't do anything here.

Comment 16

[Ryan VanderMeulen [:RyanVM]]

What's the status for 90 here?

Comment 17

[Nan Jiang [:nanj]]

What's the status for 90 here?

This should be fixed by Bug 1714449.

Andrei, will you uplift Bug 1714449 to 90?

Comment 18

[Andrei Oprea [:andreio]]

This is confusing but it's actually bug 1715804 that needs to be uplifted (already got the request).

Comment 19

[Maria Heres, inactive, contact Ecosystem QA]

I have verified that the issue is no longer reproducible after Bug 1715804 was uplifted. The telemetry event is sent even after restart on Firefox Beta 90.0b12 (Build ID: 20210624190035) using Windows 10, macOS 11.1 and Linux Mint 20.