bqetl_glean_usage.checks__warn_org_mozilla_firefox_derived__baseline_clients_last_seen__v1 failed for exec_date 5/29
Categories
(Data Platform and Tools :: General, defect)
Tracking
(Not tracked)
People
(Reporter: Alekhya, Unassigned)
References
Details
(Whiteboard: [airflow-triage])
Columns with NULL values: first_seen_date
Comment 1•9 months ago
|
||
This is still ongoing.
Comment 2•9 months ago
|
||
Just a quick update here, there was a discussion related to this few days back. We decided for now to wait until the next Shredder run to see if this gets resolved.
Full conversation can be found here: https://mozilla.slack.com/archives/C01E8GDG80N/p1717172902167549
Comment 3•9 months ago
|
||
These NULL values occurred after a backfill of org_mozilla_firefox_derived.baseline_clients_daily_v1
and org_mozilla_firefox_derived.baseline_clients_last_seen_v1
, but not org_mozilla_firefox_derived.baseline_clients_first_seen_v1
. Since baseline_clients_first_seen_v1
was already shredded some clients re-appeared in baseline_clients_daily_v1
without a first_seen_date
.
I quick fix was to remove these clients from org_mozilla_firefox_derived.baseline_clients_daily_v1
and org_mozilla_firefox_derived.baseline_clients_last_seen_v1
for the most recent partition. They will get removed from the other partitions once shredder picks them up
Comment 4•9 months ago
|
||
I also removed clients without first_seen_date
values from the org_mozilla_firefox_derived.baseline_clients_last_seen_v1
table's 2024-06-03 partition so I could rerun the 2024-06-04 ETL for that table (which selects from the previous day's data) to make sure this fix works as intended.
Description
•