915438 - Create a long-term graph of crash rates

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Robert Kaiser]

[I'm filing this in the Socorro product for now, but it might actually belong in a metrics component or such, depending on who will work on it.]

To see how the stability program is doing and how our product stability evolves, and to make that visible to both the stability group and management, we should have a long-term graph (over at least a year or two) of crash rates.

For the first iteration we should have this for the Firefox release channel, as a stacked area graph with series for browser crashes, plugin crashes and plugin hangs (for now - content crashes may be added later but we don't have those yet).

We already have this data in Socorro (we are using it for the front page graph there), it's just a matter of getting it out of the database and creating the graph.


One thing to think about there though is how we can smoothen out the variations we get at the start of every release, as we want an ongoing graph over a longer term. We could do the whole release channel, the recent two releases, all releases within the last x weeks (8? 12?) or something convoluted like "add up recent releases until we cross x% of uptake on the channel" (50%? 60%? 75%?). Some help and ideas there would be appreciated.


Once we have this for Firefox release, I'd like to see it for mobile ("FennecAndroid") and also for the beta channel as well.

Comment 1

[Robert Kaiser]

One things we should put on this graph as well, right away or as a followup, is this note from the stability week:
-----------------------------
Annotate/show events like Firefox or MS releases on the crash charts? SUMO already has the ability to note "events" on their graphs. They use it for releases as well. Maybe we can steal or learn.
-----------------------------

Comment 3

[Robert Kaiser]

I have gathered some data for this recently and made some graphs like this for myself. This showed me well where the value of this is and I'm looking forward to having this is a dynamic and better accessible fashion.

Comment 4

[Robert Kaiser]

FWIW, I ended up fixing this myself as well, as I couldn't wait for it and this seems to not be of high priority to anyone else. See https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/ for my current solution.