Closed Bug 1841267 Opened 2 years ago Closed 2 years ago

Airflow task bqetl_artifact_deployment.publish_views failed for exec_date 2023-06-30

Categories

(Data Platform and Tools :: General, defect)

Product:
Data Platform and Tools
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: lvargas, Assigned: ascholtz)

Details

(Whiteboard: [airflow-triage])

Airflow task bqetl_artifact_deployment.publish_views failed for exec_date 2023-06-30

Task link:
https://workflow.telemetry.mozilla.org/log?dag_id=bqetl_artifact_deployment&task_id=publish_views&execution_date=2023-06-29T05%3A30%3A00%2B00%3A00&map_index=-1

Log extract:

https://console.cloud.google.com/kubernetes/pod/us-west1/workloads-prod-v1/default/publish-views-uhvk089n/logs?project=moz-fx-data-airflow-gke-prod

Failed to publish view: moz-fx-data-shared-prod.mozilla_vpn.subscriptions
Failed to publish view: moz-fx-data-shared-prod.fenix.client_deduplication
Failed to publish view: moz-fx-data-shared-prod.monitoring.airflow_dag

These views are not sql generated.
For fenix.client_deduplication the logs indicate that field normalized_app_id already in schema.

The task is still failing for exec_date 2023-07-03

Assignee: nobody → ascholtz

Airflow task bqetl_artifact_deployment.publish_tables failed for since 2023-07-15:

google.api_core.exceptions.BadRequest: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/moz-fx-data-shared-prod/datasets/adjust_derived/tables/adjust_deliverables_v1?prettyPrint=false: Provided Schema does not match Table moz-fx-data-shared-prod:adjust_derived.adjust_deliverables_v1. Field app is missing in new schema

BadRequest: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/moz-fx-data-shared-prod/datasets/revenue_derived/tables/revenue_data_admarketplace_rpm_v1?prettyPrint=false: Provided Schema does not match Table moz-fx-data-shared-prod:revenue_derived.revenue_data_admarketplace_rpm_v1. Field revenue has changed type from INTEGER to NUMERIC

https://console.cloud.google.com/errors/detail/CNvymtqa_7mXlgE;version=publish-new-tables-qfoy8v31;time=P1D?project=moz-fx-data-airflow-gke-prod

Airflow task bqetl_artifact_deployment.publish_tables failed for exec_date 2023-07-17:

BadRequest: 400 Column 7 in UNION ALL has incompatible types: STRUCT<boolean STRUCT<browser_is_default BOOL, glean_core_migration_successful BOOL, glean_error_preinit_tasks_timeout BOOL, ...>, counter STRUCT<glean_error_io INT64, glean_error_preinit_tasks_overflow INT64, glean_time_invalid_timezone_offset INT64, ...>, datetime STRUCT<glean_validation_first_run_hour TIMESTAMP, raw_glean_validation_first_run_hour STRING>, ...>, STRUCT<boolean STRUCT<browser_is_default BOOL, glean_core_migration_successful BOOL, glean_error_preinit_tasks_timeout BOOL, ...>, counter STRUCT<glean_error_io INT64, glean_error_preinit_tasks_overflow INT64, glean_time_invalid_timezone_offset INT64, ...>, datetime STRUCT<glean_validation_first_run_hour TIMESTAMP, raw_glean_validation_first_run_hour STRING>, ...>, STRUCT<boolean STRUCT<browser_is_default BOOL, glean_core_migration_successful BOOL, glean_error_preinit_tasks_timeout BOOL, ...>, counter STRUCT<glean_error_io INT64, glean_error_preinit_tasks_overflow INT64, glean_time_invalid_timezone_offset INT64, ...>, datetime STRUCT<glean_valid... at [24:1]

google.api_core.exceptions.BadRequest: 400 CREATE VIEW has columns with duplicate name normalized_app_id at [5:1]

  • after some investigation it appears that the query that tried to generate moz-fx-data-shared-prod.fenix.client_deduplication contains -- Generated via ./bqetl generate glean_usage

https://console.cloud.google.com/errors/detail/CK-gutOClOGQXA;version=publish-views-k47f0fff;time=P1D?project=moz-fx-data-airflow-gke-prod

Airflow task bqetl_artifact_deployment.publish_tables failed for exec_date 2023-07-20:

google.api_core.exceptions.BadRequest: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/moz-fx-data-shared-prod/datasets/revenue_derived/tables/revenue_data_admarketplace_rpm_v1?prettyPrint=false: Provided Schema does not match Table moz-fx-data-shared-prod:revenue_derived.revenue_data_admarketplace_rpm_v1. Field revenue has changed type from INTEGER to NUMERIC

google.api_core.exceptions.BadRequest: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/moz-fx-data-shared-prod/datasets/telemetry_derived/tables/histogram_percentiles_v1?prettyPrint=false: Field aggregates.KEY already exists in schema

google.api_core.exceptions.BadRequest: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/moz-fx-data-shared-prod/datasets/google_ads_derived/tables/daily_ad_group_stats_v1?prettyPrint=false: Provided Schema does not match Table moz-fx-data-shared-prod:google_ads_derived.daily_ad_group_stats_v1. Field ad_group_id has changed type from STRING to INTEGER

https://console.cloud.google.com/errors/detail/CNvymtqa_7mXlgE;version=publish-new-tables-tkkttsqk;time=P1D?project=moz-fx-data-airflow-gke-prod

This is now running successfully

Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.