Closed Bug 1306228 Opened 8 years ago Closed 8 years ago

Airflow scheduled Jupyter notebooks are failing

Categories

(Cloud Services Graveyard :: Metrics: Pipeline, defect)

Product:
Cloud Services Graveyard
Component:
Metrics: Pipeline
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rvitillo, Assigned: mdoglio)

References

Details

User Story

This is likely due to the recent changes we landed to telemetry-airflow and or emr-bootstrap-spark.

Downloading 's3://telemetry-airflow/steps/airflow.sh' to '/mnt/var/lib/hadoop/steps/s-1QQVZXB6F797X/.'
Unable to parse whitelist (/home/hadoop/anaconda2/lib/python2.7/site-packages/moztelemetry/histogram-whitelists.json). Assuming all histograms are acceptable.
Running job for 20160928
[1;31m---------------------------------------------------------------------------[0m
[1;31mNameError[0m                                 Traceback (most recent call last)
[1;32m/mnt/analyses/telemetry_aggregator.py[0m in [0;36m<module>[1;34m()[0m
[0;32m      8[0m [0mdate[0m [1;33m=[0m [0menviron[0m[1;33m[[0m[1;34m'date'[0m[1;33m][0m[1;33m[0m[0m
[0;32m      9[0m [1;32mprint[0m [1;34m"Running job for {}"[0m[1;33m.[0m[0mformat[0m[1;33m([0m[0mdate[0m[1;33m)[0m[1;33m[0m[0m
[1;32m---> 10[1;33m [0maggregates[0m [1;33m=[0m [0maggregate_metrics[0m[1;33m([0m[0msc[0m[1;33m,[0m [1;33m([0m[1;34m"nightly"[0m[1;33m,[0m [1;34m"aurora"[0m[1;33m,[0m [1;34m"beta"[0m[1;33m,[0m [1;34m"release"[0m[1;33m)[0m[1;33m,[0m [0mdate[0m[1;33m)[0m[1;33m[0m[0m
[0m[0;32m     11[0m [1;32mprint[0m [1;34m"Number of build-id aggregates: {}"[0m[1;33m.[0m[0mformat[0m[1;33m([0m[0maggregates[0m[1;33m[[0m[1;36m0[0m[1;33m][0m[1;33m.[0m[0mcount[0m[1;33m([0m[1;33m)[0m[1;33m)[0m[1;33m[0m[0m
[0;32m     12[0m [1;32mprint[0m [1;34m"Number of submission date aggregates: {}"[0m[1;33m.[0m[0mformat[0m[1;33m([0m[0maggregates[0m[1;33m[[0m[1;36m1[0m[1;33m][0m[1;33m.[0m[0mcount[0m[1;33m([0m[1;33m)[0m[1;33m)[0m[1;33m[0m[0m

[1;31mNameError[0m: name 'sc' is not defined

Attachments

(1 file)

PR 47
52 bytes, text/x-github-pull-request
mdoglio
: review?
Details | Review
No description provided.
Assignee: nobody → mdoglio
User Story: (updated)
Attached file PR 47 —
Attachment #8796131 - Flags: review?
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: