1704541 - glean_parser could generate a skeleton data review request

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, enhancement, P1)

Description

[Chris H-C :chutten]

Imagine

$ glean_parser --data-review-request 1703318 toolkit/components/glean/metrics_index
!! Reminder: it is your responsibility to complete and check the correctness of this automatically-generated request skeleton before requesting Data Collection Review !!
See https://wiki.mozilla.org/Data_Collection for more details.

DATA REVIEW REQUEST
1. What questions will you answer with this data?

TODO: Fill this in.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

TODO: Fill this in.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

TODO: Fill this in.

4. Can current instrumentation answer these questions?

TODO: Fill this in.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

    Measurement Description | Data Collection Category | Tracking Bug #
    ----------------------- | ------------------------ | --------------
    background_update.states | Ordered list of string descriptions of the states that the update state machine reached. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    background_update.exit_code_success | True if the exit code/status of the background update task is 0, which means success. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    background_update.client_id | The Telemetry client ID of the default profile. | highly_sensitive + technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    background_update.reasons | List of reasons that the background update task did not run. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    background_update.final_state | String description of the final state the update state machine reached. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    background_update.exit_code_exception | True if the exit code/status of the background update task is 3, which means an exception was thrown. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.can_usually_apply_updates | Whether or not the Update Service can usually download and install updates. See `canUsuallyApplyUpdates` in https://searchfox.org/mozilla-central/source/toolkit/mozapps/update/nsIUpdateService.idl. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.auto_download | Per-installation preference "app.update.auto": whether to fetch and install updates without user intervention. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.service_enabled | Preference "app.update.service.enabled": whether the Mozilla Maintenance Service is enabled. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.can_usually_check_for_updates | Whether or not the Update Service can usually check for updates. See `canUsuallyCheckForUpdates` in https://searchfox.org/mozilla-central/source/toolkit/mozapps/update/nsIUpdateService.idl. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.enabled | True when policies are disabled or when the "DisableAppUpdate" is not in effect. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.background_update | Per-installation preference "app.update.background.enabled": whether to fetch and install updates in the background when Firefox is not running. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.can_usually_stage_updates | Whether the Update Service is usually able to stage updates. See `canUsuallyStageUpdates` in https://searchfox.org/mozilla-central/source/toolkit/mozapps/update/nsIUpdateService.idl. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.channel | The update channel. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)
    update.can_usually_use_bits | On Windows, whether the Update Service can usually use BITS. See `canUsuallyUseBits` in https://searchfox.org/mozilla-central/source/toolkit/mozapps/update/nsIUpdateService.idl. | technical | [Bug 1703318](https://bugzilla.mozilla.org/show_bug.cgi?id=1703318)

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

This collection is Glean so is documented on the Glean Dictionary: https://dictionary.protosaur.dev

7. How long will this data be collected?

These data will be collected permanently.
<TODO: identify at least one individual here> will be responsible. 

8. What populations will you measure?

All channels, countries, and locales. No filters.

9. If this data collection is default on, what is the opt-out mechanism for users?

These collections are Glean. The opt-out can be found in the embedding product's preferences.

10. Please provide a general description of how you will analyze this data.

TODO: Fill this in.

11. Where do you intend to share the results of your analys

TODO: Fill this in.

12. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?

No.

Wouldn't that be just nifty? It'd give data review requestors more time to spend on the questions that only they can answer, and speed up reviews for Stewards given the consistency of representation.

I can think of a challenge: we'd have to figure out how to keep the template up-to-date with https://github.com/mozilla/data-review/blob/master/request.md somehow. Maybe we host the template in mozilla/data-review and make it part of the glean_parser review process to pull in the latest version?

Comment 1

[Alessio Placitelli [:Dexter]]

I think this is a great idea! The one thing we should be careful about is not overfitting to Bugzilla. For example:

glean_parser --data-review-request 1703318 toolkit/components/glean/metrics_index

This would work assuming that 1703318 is a bugzilla bug. However, it would not be nice to Fenix and other Glean products outside of m-c.

Comment 2

[Jan-Erik Rediger [:janerik]]

(In reply to Alessio Placitelli [:Dexter] from comment #1)

glean_parser --data-review-request 1703318 toolkit/components/glean/metrics_index

This would work assuming that 1703318 is a bugzilla bug. However, it would not be nice to Fenix and other Glean products outside of m-c.

As I understand it glean_parser should pick out metrics by their associated tracking bugs. A pure number could mean our bugzilla, anything else needs to be the full URL (or we demand the full URL always)

Comment 3

[Chris H-C :chutten]

(In reply to Jan-Erik Rediger [:janerik] from comment #2)

(In reply to Alessio Placitelli [:Dexter] from comment #1)

glean_parser --data-review-request 1703318 toolkit/components/glean/metrics_index

This would work assuming that 1703318 is a bugzilla bug. However, it would not be nice to Fenix and other Glean products outside of m-c.

As I understand it glean_parser should pick out metrics by their associated tracking bugs. A pure number could mean our bugzilla, anything else needs to be the full URL (or we demand the full URL always)

I was thinking we could pull metrics based on pattern-matching the provided token exactly, and being bordered by non-word characters. So 1703318 would work to find an issue by that number on a particularly-voluminous github repo just as well. (or JIRA. https://jira.mozilla.com/browse/DO-338 could match DO-338 or 338, but the namespacing makes for collisions if you only match on the numbers instead of the full ticket id)

Comment 4

[Jan-Erik Rediger [:janerik]]

Oh right, this is a user tool, so a bit of fuzzy-matching goes a long way.

Comment 5

[Alessio Placitelli [:Dexter]]

(In reply to Jan-Erik Rediger [:janerik] from comment #2)

As I understand it glean_parser should pick out metrics by their associated tracking bugs. A pure number could mean our bugzilla, anything else needs to be the full URL (or we demand the full URL always)

Yes, that's what I understood as well: which would mean sliding towards Bugzilla a bit...

Comment 6

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean_parser/pull/344

Comment 7

[Chris H-C :chutten]

chutten merged PR #344: "bug 1704541 - Add command to generate data review request" in 8650385.

Now to file a follow-up to adding a mach subcommand for this that reads the metrics_index for you so you only need to specify the bug number.