1525977 - Data review for Firefox Monitor as an in-tree system add-on

Status: RESOLVED FIXED

(Firefox :: Firefox Monitor, enhancement)

Description

[(inactive) Nihanth Subramanya [:nhnt11]]

Once the add-on lands in tree (with Telemetry disabled) we should get it data-reviewed again.

Comment 1

[(inactive) Nihanth Subramanya [:nhnt11]]

1. What questions will you answer with this data?

• How often does user encounter the feature?
• How does user respond to the feature?

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

• Provide information essential for advancing a business objective such as supporting OKRs.
• Determine whether a product or platform change has an effect on user or browser behavior.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

• There are no alternatives other than the event telemetry to learn how user interacts with this UI.

4. Can current instrumentation answer these questions?

• No.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the Mozilla wiki.

• Technical and interaction data

Measurement Description: Events to measure user engagement with the doorhanger UI
Data Collection Category: Interaction data
Tracking Bug #: Bug 1485651

6. How long will this data be collected?

• I want this data to be collected for 6 months initially (potentially renewable).

7. What populations will you measure?

• Which release channels?

  • Release 66 onwards

• Which countries?

  • N/A

• Which locales?

  • all

• Any other filters?

  • No.

8. If this data collection is default on, what is the opt-out mechanism for users?

• User can opt out under about:preferences#privacy.

9. Please provide a general description of how you will analyze this data.

• Follow our general practice to analyze event telemetry.

10. Where do you intend to share the results of your analysis?

• To Firefox team. Results will likely be shared on a mailing list and/or Bugzilla (bug 1485651)

Comment 2

[(inactive) Nihanth Subramanya [:nhnt11]]

Chris, this is a follow-up to bug 1485650, where you did the data-review for the initial rollout of Firefox Monitor.

6 months have passed since then and it's time to renew the data-review approval. The add-on has now landed in mozilla-central (in 67) with telemetry disabled for now until we get data-review+.

I pasted in the data-review doc in the comment above this one. It's the same as before except the target release has been updated to 66 onwards (we're still shipping the add-on via Balrog for 66 but it'll be in-tree for 67).

Please let me know if there's anything else you need to know for the review.

Comment 3

[Chris H-C :chutten]

Thanks for reaching out! For future data collection reviews please attach the review request as a file and set the data-review flag to ?.

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate? 

https://github.com/mozilla/blurts-addon/blob/master/docs/TELEMETRY.md

In future the collections may be hosted in the m-c Scalars.yaml in which case it will appear in the usual Telemetry places (Probe Dictionary, elsewhere)

Is there a control mechanism that allows the user to turn the data collection on and off?

Standard Telemetry mechanisms apply.

If the request is for permanent data collection, is there someone who will monitor the data over time?**

N/A, is not permanent.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

default-on.

Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No, only counts.

Is the data collection covered by the existing Firefox privacy notice? 

Yes.

Does there need to be a check-in in the future to determine whether to renew the data? 

Yes. :nhnt11, you will be responsible to determine whether to renew or remove these probes before they expire.

---

Result: datareview+

Comment 4

[(inactive) Nihanth Subramanya [:nhnt11]]

Thank you! I'm going to use this bug to land a patch to enable Telemetry.

Comment 5

[(inactive) Nihanth Subramanya [:nhnt11]]

Attachment: Bug 1525977 - Enable Firefox Monitor telemetry. r=johannh

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/30eecb30d2a4
Enable Firefox Monitor telemetry. r=johannh

Comment 7

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/30eecb30d2a4

Comment 8

[Julien Cristau [:jcristau]]

Is this something you wish to turn on in 67? Please request uplift to beta if yes.

Comment 9

[(inactive) Nihanth Subramanya [:nhnt11]]

Yes, it is. I'll be requesting uplift next week, along with bug 1531838. It's unfortunate that these didn't make it into 67 without an uplift, but the test failure in the other bug should be solved now. Thanks for the heads up.

Comment 10

[Pascal Chevrel:pascalc (PTO until April 26)]

Tracking both this bug and bug 1531838 to keep it on my radar as there are potential uplifts.

Comment 11

[(inactive) Nihanth Subramanya [:nhnt11]]

Comment on attachment 9051984 [details]
Bug 1525977 - Enable Firefox Monitor telemetry. r=johannh

Beta/Release Uplift Approval Request

• Feature/Bug causing the regression: Bug 1525519
• User impact if declined: No telemetry data on user interaction with Firefox Monitor UI - impacts our ability to determine the effectiveness of the feature.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It just enables telemetry recording on release.
• String changes made/needed: none

Comment 12

[Pascal Chevrel:pascalc (PTO until April 26)]

Comment on attachment 9051984 [details]
Bug 1525977 - Enable Firefox Monitor telemetry. r=johannh

Uplift approved for 67 beta 12, thanks.

Comment 13

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/releases/mozilla-beta/rev/3772e946500d