1905102 - Airflow task bqetl_mobile_kpi_metrics.fenix.checks__warn_fenix_derived__engagement__v1 failed for exec_date 2024-06-27

Status: RESOLVED WONTFIX

(Data Platform and Tools :: General, defect)

Description

[Mike Williams [:mwilliams]]

Airflow task bqetl_mobile_kpi_metrics.fenix.checks__warn_fenix_derived__engagement__v1 failed for exec_date 2024-06-27

Task link:
https://workflow.telemetry.mozilla.org/dags/bqetl_mobile_kpi_metrics/grid?dag_run_id=scheduled__2024-06-26T12%3A00%3A00%2B00%3A00&task_id=fenix.checks__warn_fenix_derived__engagement__v1&tab=logs

Log extract:

Error in query string: Error processing job 'moz-fx-data-shared- prod:bqjob_r55b4c6c3ee9e4c66_000001905995330b_1': Columns with NULL values: first_seen_date

Comment 1

[Krzysztof Ignasiak [:kik]]

This seems to be related to the following bug:

https://bugzilla.mozilla.org/show_bug.cgi?id=1900084

The decision with it was to wait a bit and see if this issue resolves itself.

---

More detailed Slack conversation related to this: https://mozilla.slack.com/archives/C01E8GDG80N/p1717172902167549

Comment 2

[Mike Williams [:mwilliams]]

Same error again today 2024-06-28.

And agreed, it does seem related to that linked bug.

Comment 3

[Katie Windau]

Attachment: DAG Screenshot 2024-07-01

Comment 4

[Katie Windau]

Error is continuing to today 2024-07-01 (see attached screenshot)

Comment 5

[Krzysztof Ignasiak [:kik]]

Disable not_null check for the engagement table until the upstream issue is resolved:
https://github.com/mozilla/bigquery-etl/pull/5874