1709861 - Failure of stripe.stripe_import_events task run for 2021-05-05

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect, P1)

Description

[Jeff Klukas [:klukas] (UTC-4)]

The stripe.stripe_import_events task failed with a rather non-descriptive error message:

airflow.exceptions.AirflowException: Pod returned a failure: failed

This is causing the bqetl_mozilla_vpn DAG to send continual rescheduling alerts since the stripe_import_events is a dependency with a task sensor. I've toggled off that DAG for now to hopefully avoid sending more alerts until the upstream job is complete.

Comment 1

[Jeff Klukas [:klukas] (UTC-4)]

The bqetl_stripe DAG is also affected. I've marked the ExternalTaskSensor at the root of that DAG as failed for now, so that it stops sending alerts. That task will need to be cleared once stripe.stripe_import_events is ready.

Comment 2

[Daniel Thorn [:relud]]

https://github.com/mozilla/bigquery-etl/pull/2009

Comment 3

[Daniel Thorn [:relud]]

The airflow log line from the script itself was:

[2021-05-06 01:16:48,510] {pod_launcher.py:156} INFO - b'Error: event.data.dispute.evidence contained unexpected field: product_description\n'

I've opened a PR to add all missing fields to event.data.dispute.evidence, and I'm currently manually running the import to verify that it works before merging.