1594108 - Create a bigquery view to replace error_aggregates dataset

Status: RESOLVED FIXED

(Data Platform and Tools :: General, task)

Description

[William Lachance (:wlach)]

Based on discussions with :akomar and :bewu, it looks like the easiest way forward to replace Mission Control v1's athena usage is to create a bigquery view based on the live ping tables for "core", "main", and "crash" that mission control can query in much the same way as it currently queries error_aggregates.

You can see an example of mission control's usage of error aggregates here:

https://github.com/mozilla/missioncontrol/blob/99e8415ac3e428df5ac9060951efd7138e8e69f3/missioncontrol/etl/measure.py#L93

A list of "measures" we track is defined here:

https://github.com/mozilla/missioncontrol/blob/99e8415ac3e428df5ac9060951efd7138e8e69f3/missioncontrol/base/management/commands/load_initial_data.py#L16

We can just skip any "measure" which isn't related to crashes.

The business logic to create error aggregates is here:

https://github.com/mozilla/telemetry-streaming/blob/master/src/main/scala/com/mozilla/telemetry/streaming/ErrorAggregatorBase.scala