Closed
Bug 1312747
Opened 8 years ago
Closed 8 years ago
Refactor EMR scripts
Categories
(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)
Cloud Services Graveyard
Metrics: Pipeline
Tracking
(Not tracked)
RESOLVED
INVALID
People
(Reporter: rvitillo, Assigned: jezdez)
References
Details
User Story
Currently airflow and atmo are using two different EMR steps [1] [2] for almost the same logic. We should refactor those into a single script and add that directly to the telemetry-analysis-service repository so that we can have different steps in different environments, like staging and production. The bootstrap script [3] and the Spark configuration [4] should also be moved to telemetry-analysis-service. [1] https://github.com/mozilla/emr-bootstrap-spark/blob/master/ansible/files/batch.sh [2] https://github.com/mozilla/telemetry-airflow/blob/master/ansible/files/spark/airflow.sh [3] https://github.com/mozilla/emr-bootstrap-spark/blob/master/ansible/files/telemetry.sh [4] https://github.com/mozilla/emr-bootstrap-spark/blob/master/ansible/files/configuration.json
No description provided.
Updated•8 years ago
|
Points: --- → 2
Priority: -- → P2
Assignee | ||
Updated•8 years ago
|
Assignee: nobody → jezdez
Status: NEW → ASSIGNED
Priority: P2 → P1
Assignee | ||
Comment 1•8 years ago
|
||
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → INVALID
Updated•6 years ago
|
Product: Cloud Services → Cloud Services Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•