1378435 - Evaluate spark implementation of MacroBase on `crash_rates` dataset

Status: RESOLVED WONTFIX

(Data Platform and Tools :: Monitoring & Alerting, enhancement, P3)

Description

[Anthony Miyaguchi [:amiyaguchi]]

Crash rates are a good first use case of MacroBase on real telemetry data.[1] They are a well defined measure used in the wild and useful for identifying anomalous sub-populations. Crashes are often associated with particular build, operating systems, or hardware configuration.

Crash rates are defined as the total crashes per thousand usage hours. Clients aggregate measurements over the course of a session to avoid constant pinging to the telemetry ingestion endpoint. Thus, it is important to normalize by usage hours to take advantage of the classification and explanation pipeline to avoid individual client skew.

There are e10s experiments that can be used to verify the classification and explanation pipeline across experimental and control groups. The crash rates dashboard also provides a view to aid an intuition about the data.

The report will cover data cleaning, ETL, exploration, evaluation of the classification and explaination pipeline, and an overview of what this might look like in the context of Spark Streaming.

[1] https://chutten.github.io/telemetry-dashboard/crashes/

Comment 1

[Anthony Miyaguchi [:amiyaguchi]]

Attachment: [WIP] MacroBase and Crash Rates

This is a work in progress of the report that will be submitted to RTMO. I am trying to implement the smallest usable subset of the data and MacroBase pipeline. Batched explanations consistently kill the spark stage running on a 3x c3.4xlarge cluster.

Comment 2

[Anthony Miyaguchi [:amiyaguchi]]

Comment on attachment 8883630 [details]
[WIP] MacroBase and Crash Rates

URL to WIP can be found at the following gist.

https://gist.github.com/acmiyaguchi/30a4aafef6e2c4061767b8da90b6eba0

Comment 3

[Anthony Miyaguchi [:amiyaguchi]]

Bug 1379806 blocks the completion of this notebook since no meaningful comparison can be made without a reference.