1421688 - Understand main crashes better in main pings

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Chris H-C :chutten]

We don't have a good count of main crashes from main pings for a variety of reasons. We do have SHUTDOWN_OK which is mostly fine, but isn't collected on release.

Let's expand SHUTDOWN_OK's purview to include release so we have main crash numbers (minus startup crashes that happen before the session file was read) from the whole population.

Or, if someone has a clever idea about how to do this in another way, this would be a good place to discuss it.

Comment 1

[Chris H-C :chutten]

Attachment: bug 1421688 - Allow SHUTDOWN_OK to be recorded on release as well data-r?rweiss

Also, while I'm here, add in some now-required fields to modernize the
probe definition.

Review commit: https://reviewboard.mozilla.org/r/204004/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/204004/

Comment 2

[Chris H-C :chutten]

Comment on attachment 8933006 [details]
bug 1421688 - Allow SHUTDOWN_OK to be recorded on release as well data-r?rweiss

Data Collection R?

This extends the recording of SHUTDOWN_OK, a boolean probe that never expires, to release channel collection. This probe's been around since at least bug 781531 when Histograms.json was first introduced, and records `true` if the user's previous session ended normally and `false` if it ended in a crash.

Comment 3

[Alessio Placitelli [:Dexter]]

Comment on attachment 8933006 [details]
bug 1421688 - Allow SHUTDOWN_OK to be recorded on release as well data-r?rweiss

https://reviewboard.mozilla.org/r/204004/#review209670

Comment 4

[Chris H-C :chutten]

>    What questions will you answer with this data?

How many browser-process crashes happen to release users?

>    Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

Crashes are an important measure of Firefox quality.

>    What alternative methods did you consider to answer these questions? Why were they not sufficient?

Using main process crash pings. They don't include the full spread of data that main pings do without joining on client_id. Also, crash pings are only one source of truth... it would be beneficial to have something to compare the counts against.

>    Can current instrumentation answer these questions?

Yes, but slowly.

>    List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the found on the Mozilla wiki.

SHUTDOWN_OK, Category 1


>    How long will this data be collected? Choose one of the following:

Ongoing.

>    What populations will you measure?

Adding release channel to the pre-release channels already collected from.

>    Please provide a general description of how you will analyze this data.

The usual suspects: re:dash, ipynbs. TMO, if we add a sampled release-channel mechanism next year.

>    Where do you intend to share the results of your analysis?

reports.tmo if ipynb, Slack#datahype, maybe a blog post if I find something cool.

Comment 5

[[e-mail]]

Comment on attachment 8933006 [details]
bug 1421688 - Allow SHUTDOWN_OK to be recorded on release as well data-r?rweiss

https://reviewboard.mozilla.org/r/204004/#review216976

1. Standard telemetry, and so these measurements will have standard telemetry documentation.
2. Yes, data preferences for telemetry will be obeyed.
3. Permanent ('ongoing') data collection.  I assume :chutten is point for this measurement moving forward.
4. Type 1.
5. Default on for crashes in release (currently default on for prerelease)
6. No new identifiers
7. Within policy
8. No further check-in

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b3beb896ac26
Allow SHUTDOWN_OK to be recorded on release as well data-r?rweiss r=Dexter,rweiss+418169

Comment 7

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/b3beb896ac26