Closed Bug 1589133 Opened 5 years ago Closed 5 years ago

Latest date for Experiments dataset in bigquery is 10/6

Categories

(Data Platform and Tools :: General, defect)

Product:
Data Platform and Tools
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: bugzilla, Assigned: jason)

Details

The max date I see for the BQ version of the experiments dataset is 2019-10-06 while the athena dataset shows a max date of 2019-10-14. Airflow reports that the load job succeeded, so it looks like we'll need to investigate what's going on further.

https://github.com/mozilla/telemetry-airflow/pull/638

More details are in the issue. Once this change lands we will need to backfill the missing days. Another outstanding issue is to make sure parquet2bigquery returns failures correctly so that airflow can mark the dag task as failed.

https://github.com/mozilla/parquet2bigquery/pull/38 for exception handling issues.

Fixes landed, dates backfilled and verified all dates.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Component: Datasets: Experiments → General
You need to log in before you can comment on or make changes to this bug.