Closed Bug 1455730 Opened 8 years ago Closed 7 years ago

Upgrade Airflow to 1.9

Categories

(Data Platform and Tools :: General, defect, P1)

defect
Points:
2

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bugzilla, Assigned: hwoo)

References

Details

Attachments

(1 file)

Airflow 1.9 was released in December and includes many, many changes: https://github.com/apache/incubator-airflow/blob/master/CHANGELOG.txt More importantly for us at the moment, Airflow 1.8 is no longer compatible with pip 10.0 meaning we have to downgrade pip as a hack. We should come up with a test plan for major airflow changes and then upgrade ASAP.
Depends on: 1455732
Assignee: nobody → hwoo
Priority: P2 → P3
Priority: P3 → P2
Priority: P2 → P1
todo: - fix fernet issue - generate new mozlog.diff file for prod container build (patching step fails)
- Modify cloudops-deployment repo to change AIRFLOW_FERNET_KEY TO AIRFLOW__CORE__FERNET_KEY for all envs: e.g. https://github.com/mozilla-services/cloudops-deployment/blob/master/projects/data/puppet/yaml/app/data.prod.wtmo.yaml - Also modify this in sops for all envs
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Component: Scheduling → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: