Closed
Bug 1938404
Opened 2 months ago
Closed 2 months ago
Airflow task `bqetl_artifact_deployment.publish_new_tables` failed for exec_date 2024-12-19
Categories
(Data Platform and Tools :: General, defect)
Data Platform and Tools
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: srose, Assigned: jmoscon)
Details
(Whiteboard: [airflow-triage])
Log extract:
Failed deploy for sql/moz-fx-data-shared-prod/jira_service_desk_derived/user_v1/query.py: (Unable to create table moz-fx-data-shared-prod.jira_service_desk_derived.user_v1: 404 POST https://bigquery.googleapis.com/bigquery/v2/projects/moz-fx-data-shared-prod/datasets/jira_service_desk_derived/tables?prettyPrint=false: Not found: Dataset moz-fx-data-shared-prod:jira_service_desk_derived)
Comment 1•2 months ago
|
||
This should be fixed by https://github.com/mozilla/bigquery-etl/pull/6710. I happened to be looking at this because I was testing some performance improvements in https://mozilla-hub.atlassian.net/browse/DSRE-1773 and it looks like it took ~30m to go from https://github.com/mozilla/private-bigquery-etl/commit/f37d46d9a48438a38f4aa19eee34e542d911f3a7 to prod deployment. I will file a ticket to trigger airflow deploys from bqetl CI only if no dataset metadata is being added, and to make sure dataset deployment logic kicks off an artifact deployment job if a new dataset is added.
Reporter | ||
Updated•2 months ago
|
Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•