1874164 - Airflow task bqetl_glean_usage.checks__warn_org_mozilla_ios_tiktok_reporter_derived__baseline_clients_last_seen__v1 failed for exec_date 2024-01-10

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect)

Description

[Krzysztof Ignasiak [:kik]]

Airflow task bqetl_glean_usage.checks__warn_org_mozilla_ios_tiktok_reporter_derived__baseline_clients_last_seen__v1 failed for exec_date 2024-01-10

Task link:
https://workflow.telemetry.mozilla.org/dags/bqetl_glean_usage/grid?dag_run_id=scheduled__2024-01-10T02%3A00%3A00%2B00%3A00&task_id=tiktokreporter_ios.checks__warn_org_mozilla_ios_tiktok_reporter_derived__baseline_clients_last_seen__v1

Log extract:

[2024-01-11, 02:19:54 UTC] {pod_manager.py:437} INFO - [base] Error in query string: Error processing job 'moz-fx-data-shared- prod:bqjob_r327f759786286531_0000018cf65155d6_1': Min Row Count Error: 0 rows found, expected more than 1 rows
[2024-01-11, 02:19:54 UTC] {pod_manager.py:450} INFO - [base] Error in query string: Error processing job 'moz-fx-data-shared- prod:bqjob_r1ac1d5d105ab555e_0000018cf651804e_1': division by zero: 0 / 0

This appears due to the fact that no baseline pings have been delivered for the execution date.

Action required:

Confirm baseline pine is expected for this application
a) if yes, we need to understand why it's not being delivered and when we should expect them.
b) if not, we should add to the list of applications without a baseline ping here: https://github.com/mozilla/bigquery-etl/blob/18f6f229f9c3b3d4a6e6604552a20410525c0c8a/sql_generators/glean_usage/common.py#L25-L37

Comment 1

[Krzysztof Ignasiak [:kik]]

:travis_ it appears you added this application to the probe-scraper configuration via this commit: https://github.com/mozilla/probe-scraper/commit/2dde9099e549e097451b6a31a2c84a1bbf512891

Do you know if we expect to receive baseline pings for this specific application?

Comment 2

[Travis Long [:travis_]]

:kik, since these are using the standard Glean bindings for iOS and Android then, yes they will be sending baseline pings, event pings, and metrics pings if there is data on any of those, along with any custom pings they have defined. Any application that has glean (not glean.js) as a dependency should expect to ingest these internal Glean pings.

From what I last heard from Jesse, they are going to do some final QA and testing before this is released so we should start seeing some traffic from these applications soon-ish.

tl;dr, can we make this better? Certainly! Will the effort involved be worthwhile considering how infrequent we can expect this? Probably not

Comment 3

[Krzysztof Ignasiak [:kik]]

Hey :travis_ ,

BQ ETL Checks have been introduced to our workflows and run automatically after a query is executed. Glean apps are part of this workflow which whereby a number of checks are added to all baseline app tables. Currently, one of out workflows is getting marked as failed due to the reported error in this bug. We do have other cases where this is expected due to apps not sending this ping and therefore the corresponding tables do not contain any data. This is okay in itself, we have other cases where apps do not send baseline pings and therefore the checks fail on them. This is why we introduced an exclusion list to not auto-generate the baseline checks for those apps.

So the main question here is if this app should be added to the exclusion list or are we actually expecting to receive baseline pings for it but currently are not?

Comment 4

[Travis Long [:travis_]]

:kik this app should eventually generate baseline pings, it just hasn't yet because it isn't released. So, I don't think putting it on the exclusion list is appropriate, in this case.

Comment 5

[Sean Rose [:srose]]

This is still continuing to fail. Do we have any ETA for when it will be released?

Comment 6

[Katie Windau]

This is still continuing to fail, and it ran for 16 hours on 2/4/24. Error states expects 1 or more rows, received 0.

Comment 7

[Lucia Vargas]

Quick update, still in failing status for 2024-02-12.

Comment 8

[Eduardo Filho [:efilho]]

Still failing for 2024-03-01, 2024-03-02 and 2024-03-03 with the error:
Min Row Count Error: 0 rows found, expected more than 1 rows
which probably causes the next
division by zero: 0 / 0

Comment 9

[Eduardo Filho [:efilho]]

checks__warn_org_mozilla_ios_tiktok_reporter_tiktok_reportershare_derived__baseline_clients_last_seen__v1
failing for the same reason.

Comment 10

[Eduardo Filho [:efilho]]

Seems like :akomar will remove this check, because the data for this ETL is private. Arkadiusz, can you please confirm if that's accurate? If you also don't mind closing the ticket when it is done, please?