Closed
Bug 1406972
Opened 8 years ago
Closed 8 years ago
Use EMR 5.8.0/Spark 2.2 as the default for all airflow tasks
Categories
(Data Platform and Tools :: General, enhancement, P2)
Data Platform and Tools
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: bugzilla, Unassigned)
Details
Attachments
(1 file)
Pinning to EMR 5.8.0 reduced the duration of the Experiment Aggregates job by nearly 1/3, and longitudinal runtimes also improved, so it seems it's stable and we might see significant improvement in the rest of our jobs. We should run trials for our p1 datasets (main_summary, crash_summary, churn, telemetry_aggregates are probably a good place to start?) and then change the default.
|
||
Comment 1•8 years ago
|
||
A little bit late to the party, but this PR bumped the version to 5.9.0.
|
|
||
Updated•8 years ago
|
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
|
Assignee | |
Updated•3 years ago
|
Component: Scheduling → General
You need to log in
before you can comment on or make changes to this bug.
Description
•