Closed Bug 1899799 Opened 9 months ago Closed 9 months ago

bqetl_glean_usage.checks__warn_org_mozilla_firefox_derived__baseline_clients_last_seen__v1 failed for exec_date 5/29

Categories

(Data Platform and Tools :: General, defect)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: Alekhya, Unassigned)

References

Details

(Whiteboard: [airflow-triage])

Columns with NULL values: first_seen_date

Blocks: 1899800

This is still ongoing.

Just a quick update here, there was a discussion related to this few days back. We decided for now to wait until the next Shredder run to see if this gets resolved.

Full conversation can be found here: https://mozilla.slack.com/archives/C01E8GDG80N/p1717172902167549

These NULL values occurred after a backfill of org_mozilla_firefox_derived.baseline_clients_daily_v1 and org_mozilla_firefox_derived.baseline_clients_last_seen_v1, but not org_mozilla_firefox_derived.baseline_clients_first_seen_v1. Since baseline_clients_first_seen_v1 was already shredded some clients re-appeared in baseline_clients_daily_v1 without a first_seen_date.

I quick fix was to remove these clients from org_mozilla_firefox_derived.baseline_clients_daily_v1 and org_mozilla_firefox_derived.baseline_clients_last_seen_v1 for the most recent partition. They will get removed from the other partitions once shredder picks them up

Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED

I also removed clients without first_seen_date values from the org_mozilla_firefox_derived.baseline_clients_last_seen_v1 table's 2024-06-03 partition so I could rerun the 2024-06-04 ETL for that table (which selects from the previous day's data) to make sure this fix works as intended.

You need to log in before you can comment on or make changes to this bug.