1447645 - Investigate recent increase in crash pings on release

Status: RESOLVED INACTIVE

(Data Platform and Tools :: Monitoring & Alerting, enhancement)

Description

[Mark Reid [:mreid]]

Attachment: Snapshot of increasing crash rates

We've gotten alerts lately for an increase in crash pings on release per [1] (screenshot attached for posterity).

Let's do some investigation to determine if this needs further action.


[1] https://pipeline-cep.prod.mozaws.net/dashboard_output/graphs/analysis.moz_telemetry_doctype_monitor_crash.volume.html

Comment 1

[Chris H-C :chutten]

It's 59.0.1's content shutdown crash rate spiking.

Not sure what's making shutdowns take so long in 59.0.1, but we're having to kill content processes because of it.

Comment 2

[Chris H-C :chutten]

Right, forgot to show my work: https://sql.telemetry.mozilla.org/queries/52169/source#139160

Comment 3

[Chris H-C :chutten]

Sorry, it's the content shutdown crash ping count spiking. The count of content shutdown crashes is fairly level with older releases (see mission control: https://data-missioncontrol.dev.mozaws.net/#/release/windows/content_shutdown_crashes?timeInterval=259140&relative=0&percentile=99&normalized=0&disabledVersions=&versionGrouping=version&startTime=1521417600)

However, the number of crash pings denoting shutdown crashes has spiked. This inconsistency has been noted before (bug 1413172) but not fully explained. It was thought to be a constant factor, but that clearly is no longer the case.

Comment 4

[:shell escalante]

recent as of 4 years ago - closing for inactive.