1521645 - Add OOM crashes to webcompat status dashboard

Status: RESOLVED FIXED

(Data Science :: Dashboard, task, P2)

Description

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Users can ask the crash reporter to include the URL loaded in the active tab when a crash occurs. The frequency of OOM crashes on top sites may represent Firefox's performance. We should track OOM crashes on top sites over time and report them on the dashboard at https://metrics.mozilla.com/tdsmith/webcompat/.

Comment 1

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

The rules describing the fraction of crash reports that are accepted are here: https://github.com/mozilla-services/antenna/blob/79954910207f53e6dc8ecdf72ef67eff33a60064/antenna/throttler.py#L270

tl;dr:

• crash reports manually submitted from about:crashes are accepted
• crash reports with a comment are accepted
• crash reports with an email address are accepted
• prerelease and ESR crash reports are accepted
• 10% of other crash reports from release are accepted.

Before applying a straight 10x multiplier to OOM crashes it would be good to estimate how many crash reports fall into one of the 100% buckets.

Comment 2

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Ah! Only about half of OOM reports are from release -- nearly a third are from ESR -- so a 10x multiplier is indeed a bad choice.

Comment 3

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Happily, the guardians of Socorro have anticipated my need, and have included a "Throttle rate" column; weighting reports as 1/(throttle rate) should do the needful.

Comment 4

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Attachment: Weighted OOM crashes per day, per site

The attached plot (daily small OOM crashes across the Firefox userbase, per site, per day) represents a very small fraction of the Firefox user base; I wonder whether it tells us anything about webcompat. Mike, do you think this is informative / is this something you'd like to see on the webcompat site dashboard?

Comment 5

[Mike Taylor [:miketaylr]]

I think so, yeah. Observing big spikes on top sites seems like a useful tool for Relman and other interested parties.

Comment 6

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

I discussed this with Mike yesterday; instead of continuing to push this onto the dashboard prototype, I'll call the current proof-of-concept done and work on transitioning it to the Webcompat/OI collaboration around dashboarding.

One thing to note is that we only have enough crash volume to report daily-crashes-per-site for the release channel -- something would have to be truly wrong for us to see a meaningful number of prerelease crashes per day. We could consider a per-build crash-rate aggregation instead, which would allow us to integrate over a longer period in beta, though it would be harder to guess from that plot whether changes might be associated with a Firefox release vs a change to the site.