Closed Bug 1731373 Opened 3 years ago Closed 3 years ago

Add a hook for Firefox Suggest to notify TelemetryEnvironment when it updates

Categories

(Toolkit :: Telemetry, task, P1)

Product:
Toolkit
Component:
Telemetry
Type:
task
Points:
2

Tracking

()

Status:
VERIFIED FIXED
Milestone:
94 Branch
Iteration:
94.1 - Sep 6 - Sept 19
Tracking Flags:
Tracking Status
firefox92 --- verified
firefox93 --- verified
firefox94 --- verified

People

(Reporter: adw, Assigned: adw)

References

Details

Attachments

(1 file)

Bug 1731373 - Add a hook for Firefox Suggest to notify TelemetryEnvironment when it updates.
48 bytes, text/x-phabricator-request
pascalc
: approval-mozilla-beta+
RyanVM
: approval-mozilla-release+
Details | Review

Bug 1730721 added two Firefox Suggest prefs to telemetry environment, but the environment is not properly updated on startup. For a specific population of users, Firefox Suggest sets the prefs on startup: We flip them from false to true (on the default branch, but that’s not relevant for this discussion). The problem is that it happens between the time EnvironmentCache builds its initial cache in its constructor (by calling _updateSettings) and the time it adds its preferences observer (via _startWatchingPrefs). The result is that the telemetry is wrong.

I talked with chutten about it over Slack, and his suggestion is to have EnvironmentCache observe a specific Firefox Suggest topic that would be called after the prefs are set on startup, similar to how it observes several other similar topics. That fix would be targeted enough that we'd feel OK about uplifting it.

As we discussed, this adds a new observer service notification that Firefox
Suggest can send when it's done updating prefs as a result of a scenario change
and needs TelemetryEnvironment to re-cache them. That's only necessary on
startup when the scenario is set initially (as described in the bug), but this
revision does it unconditionally. That shouldn't be a problem because the only
other time the scenario changes is when Nimbus enrolls/unenrolls users, which
only happens at most twice (or possibily a small number of times maybe, if the
user is enrolled multiple times for some reason; the point is that it's a rare
event and not, like, multiple times a session).

Pushed by dwillcoxon@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7ad7e3bf1a20
Add a hook for Firefox Suggest to notify TelemetryEnvironment when it updates. r=chutten

Carmen, let's verify this bug using the STR in bug 1730721 comment 5. The cases in the QA doc also cover this.

Flags: qe-verify+
Flags: in-testsuite+

https://hg.mozilla.org/mozilla-central/rev/7ad7e3bf1a20

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
status-firefox94: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 94 Branch

Comment on attachment 9241866 [details]
Bug 1731373 - Add a hook for Firefox Suggest to notify TelemetryEnvironment when it updates.

Beta/Release Uplift Approval Request

  • User impact if declined: Needed for important Firefox Suggest rollouts on 93 and 92
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: Please see comment 4
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): This is a targeted fix to TelemetryEnvironment specifically for Firefox Suggest. No user-visible changes, only a fix for telemetry.
  • String changes made/needed:
Attachment #9241866 - Flags: approval-mozilla-release?
Attachment #9241866 - Flags: approval-mozilla-beta?

I manually verified this on 93/beta and also did a try push: https://treeherder.mozilla.org/jobs?repo=try&revision=956874071c8b0d6fb929701e56113367a2e921f4

QA Whiteboard: [qa-triaged]
  • I have verified this issue using the latest Nightly 94.0a1 (Build ID: 20210920102201) build on Windows 10 x64, macOS 11.6 and Linux Ubuntu 20.04 x64.
  • I have enabled the Offline scenario by setting my region to US.
  • I have verified that the value of the userPrefs.browser.urlbar.suggest.quicksuggest and userPrefs.browser.urlbar.suggest.quicksuggest.sponsored probes in the telemetry panel is set to TRUE.
  • I have verified that the telemetry probes values are updated accordingly to the checkboxes:
  1. uncheck only Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is false
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
  1. check the Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
  1. uncheck only Include occasional sponsored suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. uncheck Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is false
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. check the Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. check only Include occasional sponsored suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
Status: RESOLVED → VERIFIED
status-firefox94: fixed → verified

Comment on attachment 9241866 [details]
Bug 1731373 - Add a hook for Firefox Suggest to notify TelemetryEnvironment when it updates.

Approved for 93 beta 8, thanks.

Attachment #9241866 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

Comment on attachment 9241866 [details]
Bug 1731373 - Add a hook for Firefox Suggest to notify TelemetryEnvironment when it updates.

Approved for 92.0.1.

Attachment #9241866 - Flags: approval-mozilla-release? → approval-mozilla-release+

  • I have verified this issue using the Candidate 92.0.1 build (Build ID: 20210920202613) on Windows 10 x64, macOS 10.15.7 and Linux Ubuntu 20.04 x64.

  • I have enabled the Offline scenario by setting my region to US.

  • I have verified that the value of the userPrefs.browser.urlbar.suggest.quicksuggest and userPrefs.browser.urlbar.suggest.quicksuggest.sponsored probes in the telemetry panel is set to TRUE.

  • I have verified that the telemetry probes values are updated accordingly to the checkboxes:

  1. uncheck only Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is false
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
  1. check the Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
  1. uncheck only Include occasional sponsored suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. uncheck Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is false
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. check the Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. check only Include occasional sponsored suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
status-firefox92: fixed → verified

  • I have verified this issue using the Firefox Beta 93.0b8 build (Build ID: 20210921185902) on Windows 10 x64, macOS 10.15.7 and Linux Ubuntu 20.04 x64.

  • I have enabled the Offline scenario by setting my region to US.

  • I have verified that the value of the userPrefs.browser.urlbar.suggest.quicksuggest and userPrefs.browser.urlbar.suggest.quicksuggest.sponsored probes in the telemetry panel is set to TRUE.

  • I have verified that the telemetry probes values are updated accordingly to the checkboxes:

  1. uncheck only Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is false
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
  1. check the Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
  1. uncheck only Include occasional sponsored suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. uncheck Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is false
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. check the Contextual suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is false
  1. check only Include occasional sponsored suggestions option:
  • userPrefs.browser.urlbar.suggest.quicksuggest is true
  • userPrefs.browser.urlbar.suggest.quicksuggest.sponsored is true
status-firefox93: fixed → verified
QA Whiteboard: [qa-triaged]
Flags: qe-verify+
Blocks: 1796202
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: