1539154 - Implement telemetry for comparing BITS update download with nsIIncrementalDownload update download

Status: RESOLVED FIXED

(Toolkit :: Application Update, task, P1)

Description

[Robert Strong (they/them - no direct email)]

The specification is being defined in bug 1528278 and I'll post it to this bug when it is finished.

Comment 1

[Robert Strong (they/them - no direct email)]

Chris, I thought recording the application version that was updated from would be simple but after looking into it I'm not sure what the best way to do this would be. For this one I definitely would like to have multiple per session so a scalar won't work afaik. Could you provide me with info as to what can be used? Thanks!

Comment 2

[Robert Strong (they/them - no direct email)]

I might have been overthinking (or underthinking :) the issue with scalars and sessions. All of the data for this bug will be submitted to telemetry after an update has finished which only happens after a restart. With that in mind there shouldn't be any issues with sessions... right?

Comment 3

[Chris H-C :chutten]

Correct! If there's only one piece of information you're recording, and you're only recording it after a successful start of session, then there's no possible conflict.

Comment 4

[Robert Strong (they/them - no direct email)]

Attachment: Bug 1539154 - Add telemetry for the phases of an update. r=bytesized,chutten

This will provide telemetry so the affect of BITS update downloads with nsIIncrementalDownload update downloads can be compared.
Since a scalar can only be recorded once per session there are two scalars for each data point.
The update.startup telemetry is used to record values when an update finished during startup which prevents it from being recorded multiple times during a session.
The update.session telemetry is used to record values when an update finished during a session and there is a guard that prevent it from being recorded multiple times during a session.
All values for update.startup or update.session are submitted at the same time to simplify querying the values.
The tests only cover nsIIncrementalDownload since we don't have the ability to test BITS yet.
There is also some minor test cleanup.

Comment 5

[Robert Strong (they/them - no direct email)]

Attachment: data_collection_review.txt

Comment 6

[Robert Strong (they/them - no direct email)]

Attachment: data_review.csv

Comment 7

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Comment on attachment 9059097 [details]
data_collection_review.txt

data-review+; thanks for collecting those into a table.

---

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, these are scalars documented in Scalars.yaml.

2) Is there a control mechanism that allows the user to turn the data collection on and off? Provide details as to the control mechanism available.

Yes, the [telemetry opt-out mechanism](https://support.mozilla.org/en-US/kb/share-data-mozilla-help-improve-firefox).

3) If the request is for permanent data collection, is there someone who will monitor the data over time?

The request is for a temporary collection; Robert Strong will monitor with data science support.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?

All proposed collection are Category 1.

5) Is the data collection request for default-on or default-off?

Default-on.

6) Does the instrumentation include the addition of **any *new* identifiers**?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

Yes; collection will expire before Firefox 72, expected in 2020.

9) Does the data collection use a third-party collection tool?

No.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5c50775a9771
Add telemetry for the phases of an update. r=bytesized

Comment 9

[Noemi Erli[:noemi_erli]]

Backed out changeset 5c50775a9771 (Bug 1539154) for failures in browser_telemetry_complete_stageFailure.js

Failure appeared here: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&selectedJob=241380907&revision=1bb7a184f2c8ff71dac4b1a54984826a0138c606

Push that was backed out: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=5c50775a9771c31446f43ae444a0f612a3a571e7

Failure logs: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241380907&repo=autoland&lineNumber=6971
Leaks: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241380786&repo=autoland&lineNumber=43873

Backout: https://hg.mozilla.org/integration/autoland/rev/0160424142d14988f7b30595e03f156c31278a42

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/723b79068fb0
Add telemetry for the phases of an update. r=bytesized

Comment 11

[Robert Strong (they/them - no direct email)]

I disabled the tests for staging failures on ASAN and relanded.

I filed bug 1545712 for the staging failure tests failing on ASAN.

Comment 12

[Robert Strong (they/them - no direct email)]

Just a heads up that the leak log is for "OS X 10.10 debug Web platform tests test-macosx64/debug-web-platform-tests-e10s-1 W(wpt1)" which doesn't run this code so it couldn't be responsible for that leak.

Comment 13

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/723b79068fb0