Closed Bug 1686301 Opened 7 months ago Closed 6 months ago

Add error telemetry for skeleton UI

Categories

(Firefox :: General, enhancement)

enhancement

Tracking

()

RESOLVED FIXED
87 Branch
Tracking Status
firefox87 --- fixed

People

(Reporter: dthayer, Assigned: dthayer)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

We'd like to know how frequently the skeleton UI is enabled, and get visibility into why it may have been disabled for a particular user.

We need this in the skeleton UI stuff in mozglue and including
WinHeaderOnlyUtils breaks spidermonkey builds.

Attachment #9197602 - Attachment is obsolete: true
Attachment #9197603 - Attachment description: Bug 1686301 - Report errors in skeleton UI via telemetry → Bug 1686301 - Report errors in skeleton UI via telemetry r=mconley! r=emalysz!
Blocks: 1687561
Attachment #9197603 - Attachment description: Bug 1686301 - Report errors in skeleton UI via telemetry r=mconley! r=emalysz! → Bug 1686301 - Report errors in skeleton UI via telemetry r=mconley!,emalysz!

Comment on attachment 9197603 [details]
Bug 1686301 - Report errors in skeleton UI via telemetry r=mconley!,emalysz!

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
    How many users are having the skeleton UI disabled for them automatically, and why?

  2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses
    Ensure that a feature which we will likely ship is actually consistently enabled for users.

  3. What alternative methods did you consider to answer these questions? Why were they not sufficient?
    Local testing. It's infeasible to replicate the variability of the wild locally.

  4. Can current instrumentation answer these questions?
    No.

  5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.
    Measurement Description: startup.skeleton_ui_disabled_reason - the reason why the skeleton UI was not shown to the user.
    Data Collection Category: Either 1 or 2. Some limited interaction could conceivably be inferred from some values we give, such as if the skeleton UI has been disabled by the user.
    Tracking bug #: 1686301

  6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
    This collection is documented in its definitions file s Histograms.json, Scalars.yaml, and/or Events.yaml and in the Probe Dictionary at https://probes.telemetry.mozilla.org.

  7. How long will this data be collected? Choose one of the following:
    I want this data to be collected for 6 months initially (potentially renewable).

  8. What populations will you measure?

  • Which release channels?
    All.

  • Which countries?
    All.

  • Which locales?
    All.

  • Any other filters? Please describe in detail below.

  1. If this data collection is default on, what is the opt-out mechanism for users?
    Standard Firefox telemetry settings.

  2. Please provide a general description of how you will analyze this data.
    Explore the values reported via glam/tmo Measurement Dashboard/stmo.

  3. Where do you intend to share the results of your analysis?
    Likely here or in Matrix chats.

  4. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection? If so:
    No.

Additional note: There is also an additional value for this probe added in bug 1687561, queued up behind this one. It felt like a separate bug so I put it in as such, but it will report a value indicating that the skeleton UI crashed once if that ever happens for a user.

Attachment #9197603 - Flags: data-review?(chutten)
Attachment #9197603 - Flags: data-review?(chutten) → data-review?(nshadowen)

Comment on attachment 9197603 [details]
Bug 1686301 - Report errors in skeleton UI via telemetry r=mconley!,emalysz!

DATA COLLECTION REVIEW RESPONSE:

 Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file Scalars.yaml and the Probe Dictionary.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

The request is for an initial 6 month data collection with the potential to renew. :dthayer will be responsible for monitoring the data over time.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction. Measurement description: startup.skeleton_ui_disabled_reason - the reason why the skeleton UI was not shown to the user.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

Yes, in 6 months. :dthayer will be responsible for discontinuing or submitting of new data review request if extension is needed. (please set a to-do reminder or file a bug if appropriate)

Does the data collection use a third-party collection tool? 

No.

Result: datareview+

Attachment #9197603 - Flags: data-review?(nshadowen) → data-review+
Pushed by dothayer@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3dd80b8c8e03
Report errors in skeleton UI via telemetry r=mconley,emalysz
Status: ASSIGNED → RESOLVED
Closed: 6 months ago
Resolution: --- → FIXED
Target Milestone: --- → 87 Branch
See Also: → 1714266
You need to log in before you can comment on or make changes to this bug.