Get rid of crash_aggregates in favor of error_aggregates

NEW
Unassigned

Status

Data Platform and Tools
Datasets: General
P3
normal
9 months ago
4 days ago

People

(Reporter: mdoglio, Unassigned)

Tracking

(Depends on: 1 bug)

Details

(Reporter)

Description

9 months ago
The Mission Control dataset (error_aggregates) schema is a superset of what crash_aggregates offers. Also, its latency is <1h vs up to 24h. We should get rid of crash_aggregates and save both machine and human time.

Comment 1

9 months ago
Is that (error_aggregates) coming from main pings, or is it using crash pings? I'm wondering if we should focus on using error_aggregates for everything dealing with crashes or if we should be creating a new crash_aggregates based on crash pings.
(Reporter)

Comment 2

9 months ago
It's using both, like crash_aggregates does. I like the idea of having a crash ping-only aggregates dataset. I would be happy to work on that if it becomes a priority.

Comment 3

9 months ago
With 55 in release, we finally have the full capability to correlate crash reports with crash pings (if applicable). So if we could get this dataset up and running, we could start looking at ping-only data on all of release, even segmented by process type. I don't know who would determine the priority of this, but it would be very useful to get dashboards hooked up to this data as we lead up to 57.
(In reply to Mauro Doglio [:mdoglio] from comment #2)
> It's using both, like crash_aggregates does. I like the idea of having a
> crash ping-only aggregates dataset. I would be happy to work on that if it
> becomes a priority.

We could use direct-to-parquet for this dataset. It doesn't require parsing the entire payload, you can have it output a subset of fields to parquet, which in this case would probably just be that dimensional information (version, app, build, etc.).
(Reporter)

Comment 5

8 months ago
It sounds like we can:
1- get rid of crash_aggregates
2- use crash_summary (which is already in parquet) to derive new datasets (like the one for crash reports correlations that :ddurst is talking about).

Updated

8 months ago
Depends on: 1382236

Updated

5 months ago
Priority: -- → P3
Moved, per https://bugzilla.mozilla.org/show_bug.cgi?id=1453996
Component: Datasets: Crash Aggregates → Datasets: General
You need to log in before you can comment on or make changes to this bug.