Closed Bug 1575239 Opened 5 years ago Closed 5 years ago

Develop GCP-native error_aggregates replacement

Categories

(Data Platform and Tools Graveyard :: Datasets: Error Aggregates, task)

Product:
Data Platform and Tools Graveyard
Component:
Datasets: Error Aggregates
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1594108

People

(Reporter: akomar, Unassigned)

Details

Error aggregates dataset is currently produced in Spark streaming job. It seems that converting it to a SQL query, scheduled every hour would be the most straightforward migration path.
That would also require some minor changes in Mission Control to make it read from BigQuery instead of Athena.

Will: can you confirm if it would be acceptable to have a 1 hour delay of data for Mission Control?

Flags: needinfo?(wlachance)

Confirmed with Ritu and Phillip at channel meeting that a one hour (or even several hour) delay in receiving/processing the data is acceptable.

I am going to CC them (as well as Marcia) on this bug just to give them a heads up that the backend data source feeding the old mission control dashboard will be changing in the next few months. This is most likely going to be an interim solution while we work on a new dashboard with data science.

Flags: needinfo?(wlachance)
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE
Product: Data Platform and Tools → Data Platform and Tools Graveyard
You need to log in before you can comment on or make changes to this bug.