Closed Bug 1286862 Opened 8 years ago Closed 7 years ago

telemetry-aggregator job should depend on past

Categories

(Data Platform and Tools :: General, defect, P2)

Product:
Data Platform and Tools
Component:
General
Type:
defect
Points:
1

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rvitillo, Unassigned)

References

Details

User Story

By setting depend_on_past = True in the job's configuration for Airflow we ensure that only a single job at the time will be run during backfill, which is what we want to avoid DDoSiNg the database.
      No description provided.
Points: --- → 1
Priority: -- → P2
Component: Metrics: Pipeline → Datasets: Telemetry Aggregates
Product: Cloud Services → Data Platform and Tools
The airflow job is already setup using depend_on_past [0]. Thought we can possibly consider removing it since we now have a read-replica; heavy writing won't affect users anymore.

[0] https://github.com/mozilla/telemetry-airflow/blob/master/dags/telemetry_aggregates.py#L7
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Component: Datasets: Telemetry Aggregates → General
You need to log in before you can comment on or make changes to this bug.