Closed Bug 1937819 Opened 2 months ago Closed 2 months ago

Airflow task `bqetl_artifact_deployment.publish_new_tables` failed for exec_date 2024-12-17

Categories

(Data Platform and Tools :: General, defect)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: srose, Assigned: kwindau)

Details

(Whiteboard: [airflow-triage])

Task link:
https://workflow.telemetry.mozilla.org/dags/bqetl_artifact_deployment/grid?dag_run_id=manual__2024-12-17T14%3A28%3A54.494485%2B00%3A00&task_id=publish_new_tables&tab=logs

Log extract:

Failed deploy for sql/moz-fx-data-shared-prod/telemetry_derived/fx_health_ind_clients_daily_by_os_version_v1/query.sql: (Unable to update table moz-fx-data-shared-prod.telemetry_derived.fx_health_ind_clients_daily_by_os_version_v1: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/moz-fx-data-shared-prod/datasets/telemetry_derived/tables/fx_health_ind_clients_daily_by_os_version_v1?prettyPrint=false: Provided Schema does not match Table moz-fx-data-shared-prod:telemetry_derived.fx_health_ind_clients_daily_by_os_version_v1. Field default_percent is missing in new schema)
Failed deploy for sql/moz-fx-data-shared-prod/telemetry_derived/fx_health_ind_clients_daily_by_os_v1/query.sql: (Unable to update table moz-fx-data-shared-prod.telemetry_derived.fx_health_ind_clients_daily_by_os_v1: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/moz-fx-data-shared-prod/datasets/telemetry_derived/tables/fx_health_ind_clients_daily_by_os_v1?prettyPrint=false: Provided Schema does not match Table moz-fx-data-shared-prod:telemetry_derived.fx_health_ind_clients_daily_by_os_v1. Field default_percent is missing in new schema)

I think this is just because the artifact deployment DAG was running really far behind, I had deployed a new table manually with new columns and I think the old deployment run for a previous version of the table was confused why there are more columns now. Will keep an eye to make sure the next run works OK since that is the run where I add the new columns in production

Status: NEW → ASSIGNED
Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.