1795471 - Add metrics (reasons to not schedule) to understand how many users are not using background update on Windows

Status: RESOLVED FIXED

(Toolkit :: Application Update, enhancement, P3)

Description

[Nick Alexander :nalexander [he/him]]

In Bug 1689519, I very carefully added a set of debugging reasons explaining why Firefox is not scheduling the background update background task. However, those reasons aren't included in any metrics ping. This ticket tracks including basic scheduled/not scheduled data and exposing the various reasons background updates aren't scheduled so that we can bring background updates to more (Windows) users.

Comment 1

[Nick Alexander :nalexander [he/him]]

Plan is for Max to pick up this ticket. There are a few parts to this work:

1. Adding a new Glean metric for this data. We have background_update.reasons, which is only sent by the background update task itself. We want to send this for Desktop as well. So we can either:

• duplicate that into an appropriate place for Desktop Glean telemetry (or use a Glean labeled boolean with labels the keys of BackgroundUpdate.reason) or
• figure out how to include that existing measure in Desktop metrics Glean pings. I think just adding - metrics to the send_in_pings: list might be enough to expose it, once it's recorded in Desktop itself.

Bonus points for connecting to GIFFT as well.

2. Recording the new metric. That should be in or around maybeScheduleBackgroundUpdateTask.

3. (Possibly) Including Glean metrics for whether the background task is actually scheduled with Windows, whether it was run, when it was last run, etc.

Ask questions as needed, Max!

Comment 2

[Max]

Attachment: Bug 1795471 - Add metrics to understand why users do not use background update, r=nalexander

Comment 3

[Max]

Attachment: data-review-request.md

Hi! This is my first time requesting a data-review. Please let me know if anything has to be adjusted.

Comment 4

[Megan McCorquodale]

Comment on attachment 9316484 [details]
data-review-request.md

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
   Yes, this will be documented in the Glean dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off?
   Yes, users can opt out of data collection.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?
   Yes, this will be monitored at [e-mail]

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
   Category 2, Interaction Data.

5. Is the data collection request for default-on or default-off?
   Default on.

6. Does the instrumentation include the addition of any new identifiers?
   No new identifiers.

7. Is the data collection covered by the existing Firefox privacy notice?
   Yes.

8. Does the data collection use a third-party collection tool?
   No.

---

data-review +

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f59135d71c5b
Add metrics to understand why users do not use background update, r=nalexander

Comment 6

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/f59135d71c5b

Comment 7

[Max]

Comment on attachment 9315754 [details]
Bug 1795471 - Add metrics to understand why users do not use background update, r=nalexander

Beta/Release Uplift Approval Request

• User impact if declined: Users would have to wait at least one month longer before we could deliver solutions to the most relevant reasons why the background updater is not being used under Windows.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Medium
• Why is the change risky/not risky? (and alternatives if risky): We have added Glean metrics for our Windows users, multiple code paths have been covered by automated tests and the remaining code paths work in the same way (even with the same code) and we also received first plausible results from out of our nightly builds.
• String changes made/needed: none
• Is Android affected?: No

Comment 8

[Donal Meehan [:dmeehan]]

:max it looks like this should be ok to uplift, given you've reviewed some results and it's been in nightly for a week.
Though, could you add some details about what makes the risk Medium? ( for example )

Comment 9

[Max]

:Donal That is good news, thanks. I can confirm, that we already received telemetry data from the nightly builds during the last business week, so from Monday on, but that is only five days. Will it still be possible to uplift the change next Monday?

Regarding the risk: I was first going to choose "low", but found telemetry data as an example in the best_practices section of the "Patch uplifting rules" wiki page. Most of the code is supposed to be triggered primarily in an edge case, when the background update is not possible and is not supposed to impact the execution when everything works as expected. That is why I have not considered the code change to be "hot path" and why I have not set the qe-verify flag. Please let me know if that is how it is supposed to be, because this is my first uplift.

Comment 10

[Donal Meehan [:dmeehan]]

(In reply to Max from comment #9)

:Donal That is good news, thanks. I can confirm, that we already received telemetry data from the nightly builds during the last business week, so from Monday on, but that is only five days. Will it still be possible to uplift the change next Monday?

Regarding the risk: I was first going to choose "low", but found telemetry data as an example in the best_practices section of the "Patch uplifting rules" wiki page. Most of the code is supposed to be triggered primarily in an edge case, when the background update is not possible and is not supposed to impact the execution when everything works as expected. That is why I have not considered the code change to be "hot path" and why I have not set the qe-verify flag. Please let me know if that is how it is supposed to be, because this is my first uplift.

Thanks for the context on the risk.
I can defer taking the patch until Monday as you prefer. I can ping you on slack if there are any concerns.

Comment 11

[Nick Alexander :nalexander [he/him]]

(In reply to Donal Meehan [:dmeehan] from comment #10)

(In reply to Max from comment #9)

:Donal That is good news, thanks. I can confirm, that we already received telemetry data from the nightly builds during the last business week, so from Monday on, but that is only five days. Will it still be possible to uplift the change next Monday?

Regarding the risk: I was first going to choose "low", but found telemetry data as an example in the best_practices section of the "Patch uplifting rules" wiki page. Most of the code is supposed to be triggered primarily in an edge case, when the background update is not possible and is not supposed to impact the execution when everything works as expected. That is why I have not considered the code change to be "hot path" and why I have not set the qe-verify flag. Please let me know if that is how it is supposed to be, because this is my first uplift.

Thanks for the context on the risk.
I can defer taking the patch until Monday as you prefer. I can ping you on slack if there are any concerns.

No need to defer; this is a low-risk uplift.

Comment 12

[Donal Meehan [:dmeehan]]

Comment on attachment 9315754 [details]
Bug 1795471 - Add metrics to understand why users do not use background update, r=nalexander

Approved for 111.0b6

Comment 13

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/releases/mozilla-beta/rev/c5ee57ac63c0