1570652 - Add startup.profile_selection_reason scalar to new profile ping

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Jan-Erik Rediger [:janerik]]

This bug is for implementing putting scalars into the new-profile-ping and adding the startup.profile_selection_reason to it (while also keeping it in the main ping).

Comment 1

[Jan-Erik Rediger [:janerik]]

Attachment: Bug 1570652 - Include scalars in new-profile ping

Comment 2

[Jan-Erik Rediger [:janerik]]

Attachment: Bug 1570652 - Add profile_selection_reason to the new-profile ping store

Depends on D40595

Comment 3

[Jan-Erik Rediger [:janerik]]

Attachment: Data Collection Review

Comment 4

[Jan-Erik Rediger [:janerik]]

Here's the patch for adding scalars to the new profile ping as well as adding the one metric you want. I also did the data collection review request
However, the data is set to expire in version 72. That is rather soon. Do you want to extend it? If so I can just do that in the same patch and data collection review.

Comment 5

[Chris H-C :chutten]

Comment on attachment 9082950 [details]
Data Collection Review

Load balancing to tdsmith.

Comment 6

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Comment on attachment 9082950 [details]
Data Collection Review

The answers to 3 and 4 are inconsistent (if you can answer this question with a SQL join, you can answer this question with current instrumentation) but this collection will avoid the cost of these queries.

1) Is there or will there be **documentation** that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in Scalars.yaml. I filed https://github.com/mozilla/probe-dictionary/issues/60 to additionally represent which stores probes are collected into in the probe dictionary.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the [Firefox telemetry opt-out](https://support.mozilla.org/en-US/kb/share-data-mozilla-help-improve-firefox). Users have a 30 minute window to disable telemetry before the new-profile ping is sent.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?

n/a

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, interaction data.

5) Is the data collection request for default-on or default-off?

Default-on.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No)

Jan-Erik and George are responsible for renewing the probe as needed.

9) Does the data collection use a third-party collection tool?

No.

Comment 7

[George Kaberere [:gkaberere]]

(In reply to Jan-Erik Rediger [:janerik] from comment #4)

Here's the patch for adding scalars to the new profile ping as well as adding the one metric you want. I also did the data collection review request
However, the data is set to expire in version 72. That is rather soon. Do you want to extend it? If so I can just do that in the same patch and data collection review.

Yeah I'd say we should extend it given how soon it's expiring. I'm not sure how the extension process works in terms of when to extend to / who decides. Let me know if I need to do involve someone and I will.

Comment 8

[Chris H-C :chutten]

From data stewardship's POV: extending it six months (about five versions) is typical.

Comment 9

[Jan-Erik Rediger [:janerik]]

Attachment: Data Collection Review (with expiry extension)

Tim, would you be so kind to acknowledge the expiry extension in the data review. Thanks! (Nothing else in there changed, the code patch is also changed now)

Comment 10

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Comment on attachment 9085095 [details]
Data Collection Review (with expiry extension)

Data Collection Review

Acknowledging the extension of the probe expiration to Fx 77.

1) Is there or will there be **documentation** that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in Scalars.yaml. I filed https://github.com/mozilla/probe-dictionary/issues/60 to additionally represent which stores probes are collected into in the probe dictionary.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the [Firefox telemetry opt-out](https://support.mozilla.org/en-US/kb/share-data-mozilla-help-improve-firefox). Users have a 30 minute window to disable telemetry before the new-profile ping is sent.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?

n/a

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, interaction data.

5) Is the data collection request for default-on or default-off?

Default-on.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No)

Jan-Erik and George are responsible for renewing the probe as needed.

9) Does the data collection use a third-party collection tool?

No.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4da46fd56311
Include scalars in new-profile ping r=chutten
https://hg.mozilla.org/integration/autoland/rev/674426059fcc
Add profile_selection_reason to the new-profile ping store r=chutten

Comment 12

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/4da46fd56311
https://hg.mozilla.org/mozilla-central/rev/674426059fcc

Comment 13

[Jan-Erik Rediger [:janerik]]

Comment on attachment 9082949 [details]
Bug 1570652 - Add profile_selection_reason to the new-profile ping store

Beta/Release Uplift Approval Request

• User impact if declined: This data would be very beneficial to have in 69, before the Skyline release (70) to iron out any possible issues before that.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Landed in nightly last week and we verified data is coming in.
  The change itself is covered by automatic tests.
• String changes made/needed:

Comment 14

[Jan-Erik Rediger [:janerik]]

(I shouldn't reopen bugs for uplift-requests, therefore closing again, uplift request stays relevant)

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9082948 [details]
Bug 1570652 - Include scalars in new-profile ping

Improves new-profile telemetry to help find issues ahead of the Skyline release. Approved for 69.0rc1.

Comment 16

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/7808d12b32c2
https://hg.mozilla.org/releases/mozilla-release/rev/177d61eccf2d