Status

Data Platform and Tools
Scheduling
P3
normal
2 years ago
9 months ago

People

(Reporter: mreid, Unassigned)

Tracking

(Depends on: 9 bugs)

Details

(Reporter)

Description

2 years ago
The current mechanism for scheduling tasks on analysis.telemetry.mozilla.org is very simple and as our needs become more complex, we're rapidly outgrowing its capabilities.

Specifically, we need the following:
- Dependencies between jobs
- Improved monitoring (see bug 1232642)
- More flexible scheduling options
- Support for backfill
- Ability to launch one-off jobs
(Reporter)

Updated

2 years ago
Blocks: 1248688
Depends on: 1232642

Updated

2 years ago
Priority: -- → P3
Summary: [meta] Improve scheduling on analysis.telemetry.mozilla.org → [meta] Deploy Airflow
Summary: [meta] Deploy Airflow → [meta] Airflow
I have changed the title as Airflow has been deployed. The remaining work involves slowly moving existing scheduled jobs to Airflow.
No longer blocks: 1248688
(Reporter)

Updated

2 years ago
No longer depends on: 1274984
Depends on: 1309352
Depends on: 1310486
Depends on: 1309622
Depends on: 1306664
(Reporter)

Updated

a year ago
Depends on: 1316669
Depends on: 1321424
Depends on: 1323321
(Reporter)

Updated

a year ago
Depends on: 1324805
Depends on: 1336975
Depends on: 1339495
Depends on: 1353097

Updated

10 months ago
Component: Metrics: Pipeline → Scheduling
Product: Cloud Services → Data Platform and Tools

Updated

10 months ago
Depends on: 1362227

Updated

10 months ago
Depends on: 1362229
Depends on: 1365363
You need to log in before you can comment on or make changes to this bug.