By setting depend_on_past = True in the job's configuration for Airflow we ensure that only a single job at the time will be run during backfill, which is what we want to avoid DDoSiNg the database.
Component: Metrics: Pipeline → Datasets: Telemetry Aggregates
Product: Cloud Services → Data Platform and Tools
The airflow job is already setup using depend_on_past . Thought we can possibly consider removing it since we now have a read-replica; heavy writing won't affect users anymore.  https://github.com/mozilla/telemetry-airflow/blob/master/dags/telemetry_aggregates.py#L7
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.