1823766 - Data review for monitor.firefox.com (glean.js)

Status: RESOLVED FIXED

(Firefox :: Firefox Monitor, task)

Description

[Robert Helmer [:rhelmer]]

Firefox Monitor (https://monitor.firefox.com) currently uses Google Analytics, but would like to start using Glean (via glean.js) to track more complex business metrics.

Comment 1

[Robert Helmer [:rhelmer]]

Attachment: data_review_request.md

I've generated this with the latest glean_parser and randomly chosen a data steward from https://wiki.mozilla.org/Data_Collection

Comment 2

[Robert Helmer [:rhelmer]]

I've opened a PR with the actual glean config here: https://github.com/mozilla/blurts-server/pull/2924

I don't seem to be able to request review from Github, but I think you can leave comments anyway - please let me know if that's incorrect!

Comment 3

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Comment on attachment 9324253 [details]
data_review_request.md

Summarizing discussions we've had about this on Slack:

There is more ambiguity here between the design questions and the WIP implementation than we typically see in a data-review. I think the right next step with the monitor migration is probably to get you connected to folks on the glean team, so that you can work through the design problems. The data-reviews should naturally become quite simple once you have individual probes designed and implemented.

Handing off to :travis_ to guide you through the process from here :-)

Good luck!

Comment 4

[Travis Long [:travis_]]

:rhelmer you mentioned potentially scheduling a meeting to discuss Monitor's telemetry needs around this review. Is that still your intention? If so, I would like to request that we add Bruno Rosa as well as myself from the Glean team for his Glean.js expertise.

Comment 5

[Robert Helmer [:rhelmer]]

(In reply to Travis Long [:travis_] from comment #4)

:rhelmer you mentioned potentially scheduling a meeting to discuss Monitor's telemetry needs around this review. Is that still your intention? If so, I would like to request that we add Bruno Rosa as well as myself from the Glean team for his Glean.js expertise.

Yes, thanks - I took some time to get GA4 working, but we'd like to run glean.js side-by-side so we can compare the data.

Comment 6

[Travis Long [:travis_]]

Rob, I haven't heard anything from you about this in a while, so I thought I would check up on this and see what the status is?

Comment 7

[Robert Helmer [:rhelmer]]

(In reply to Travis Long [:travis_] from comment #6)

Rob, I haven't heard anything from you about this in a while, so I thought I would check up on this and see what the status is?

Hi - we're working to prioritize this now, it's not at the top of my list at the moment. Can we pause and come back later? We will want this at some point but it's not urgent.

Comment 8

[Travis Long [:travis_]]

No worries, I just wanted to make sure you weren't waiting on me. I'm happy to take this up again whenever you are ready. And, the Glean team is happy to advise you and accept any feedback you might have.

Just let us know if and how we can help!

Comment 9

[Travis Long [:travis_]]

I'm going to clear the data-review for now, we can re-assign it to me once this is ready to move forward again.

Comment 10

[Robert Helmer [:rhelmer]]

Attachment: data_review.md

Comment 11

[Robert Helmer [:rhelmer]]

(In reply to Travis Long [:travis_] from comment #9)

I'm going to clear the data-review for now, we can re-assign it to me once this is ready to move forward again.

Hi Travis, I've been working with folks in data science to come up with the right set of metrics, you can see the working document for more context. Thanks!

Comment 12

[Travis Long [:travis_]]

Comment on attachment 9364187 [details]
data_review.md

Data Review

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the data preferences in the application settings.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

Permanent collection to be monitored by rhelmer

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, interaction data

5. Is the data collection request for default-on or default-off?

Default-on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

7. Is the data collection covered by the existing Firefox privacy notice?

Yes

8. Does the data collection use a third-party collection tool?

No

Result

data-review+

Comment 13

[Travis Long [:travis_]]

Thanks Rob! This all looks good to me.

Comment 14

[Robert Helmer [:rhelmer]]

Thanks, landed in https://github.com/mozilla/blurts-server/pull/3700

Comment 15

[Florian]

We’ve added a new event banner.view here in this PR that records the same telemetry as the already existing page.view and popup.view events.

As discussed on Matrix in #data-stewards there is no additional data review needed, but I’m adding this comment to this initial data review as a future reference.

Comment 16

[Travis Long [:travis_]]

Confirming that my initial data-review still stands and covers the intent and collection of this new event (in the event the Matrix conversation gets lost for some reason).