If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Remove the "--alow-errors" flag from the airflow notebook runner

NEW
Unassigned

Status

Data Platform and Tools
Scheduling
P2
normal
11 months ago
5 months ago

People

(Reporter: mreid, Unassigned)

Tracking

(Blocks: 1 bug)

Details

(Reporter)

Description

11 months ago
When running a Jupyter notebook via airflow, we pass the "--allow-errors" argument to nbconvert[1]. 

This means that if an exception is raised in a cell, subsequent cells will continue to execute. This seems like a recipe for buggy behaviour.

If possible, we should remove this flag for running scheduled jobs.
(Reporter)

Comment 1

11 months ago
[1] https://github.com/mozilla/telemetry-airflow/blob/master/ansible/files/spark/airflow.sh#L90
Blocks: 1269754

Updated

10 months ago
Points: --- → 3
Priority: -- → P2
(Reporter)

Updated

5 months ago
Component: Metrics: Pipeline → Scheduling
Product: Cloud Services → Data Platform and Tools
You need to log in before you can comment on or make changes to this bug.