1863382 - Airflow task probe_scraper.mozilla_schema_generator failed for 2023-11-04

Status: NEW

(Data Platform and Tools :: General, defect)

Description

[Anna Scholtz [:ascholtz]]

Airflow task probe_scraper.mozilla_schema_generator failed since 2023-11-04 with incompatible changes

Task link:
https://workflow.telemetry.mozilla.org/dags/probe_scraper/grid?dag_run_id=scheduled__2023-11-04T00%3A00%3A00%2B00%3A00&task_id=mozilla_schema_generator&tab=logs

Log extract:

023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - --- /app/validate_schema_evolution/4fbe84acc9ab20365ee3c645e00dc2a80e24865d_base-revision/firefox-desktop.metrics.1.txt
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +++ /app/validate_schema_evolution/4866b23c29aa98b4d1a1b4a3094b8950ab678f21_generated-schemas/firefox-desktop.metrics.1.txt
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - @@ -290,2 +290,4 @@
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - root.metrics.labeled_counter.rtcrtpsender_setparameters_blame_stale_transactionid.[].value INT64
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +root.metrics.labeled_counter.search_service_initialization_status.[].key STRING
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +root.metrics.labeled_counter.search_service_initialization_status.[].value INT64
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - root.metrics.labeled_rate.[].key STRING
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - 
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - --- /app/validate_schema_evolution/4fbe84acc9ab20365ee3c645e00dc2a80e24865d_base-revision/pine.metrics.1.txt
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +++ /app/validate_schema_evolution/4866b23c29aa98b4d1a1b4a3094b8950ab678f21_generated-schemas/pine.metrics.1.txt
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - @@ -279,2 +279,4 @@
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - root.metrics.labeled_counter.rtcrtpsender_setparameters_blame_stale_transactionid.[].value INT64
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +root.metrics.labeled_counter.search_service_initialization_status.[].key STRING
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +root.metrics.labeled_counter.search_service_initialization_status.[].value INT64
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - root.metrics.memory_distribution.extensions_apis_dnr_startup_cache_read_size.count INT64
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - 
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - --- /app/validate_schema_evolution/4fbe84acc9ab20365ee3c645e00dc2a80e24865d_base-revision/org-mozilla-fenix-nightly.metrics.1.txt
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +++ /app/validate_schema_evolution/4866b23c29aa98b4d1a1b4a3094b8950ab678f21_generated-schemas/org-mozilla-fenix-nightly.metrics.1.txt
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - @@ -375,2 +375,4 @@
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - root.metrics.labeled_counter.fog_validation_gvsv_audio_stream_init_gecko.[].value INT64
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +root.metrics.labeled_counter.fog_validation_gvsv_audio_stream_init_gvst.[].key STRING
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - +root.metrics.labeled_counter.fog_validation_gvsv_audio_stream_init_gvst.[].value INT64
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - root.metrics.labeled_counter.gfx_content_frame_time_reason.[].key STRING
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - 
[2023-11-05, 03:21:57 UTC] {pod_manager.py:235} INFO - found incompatible changes

Comment 1

[Anna Scholtz [:ascholtz]]

It looks like the migration ping got removed for fenix applications:

[2023-11-04, 03:19:39 UTC] {pod_manager.py:235} INFO - items removed from the base
[2023-11-04, 03:19:39 UTC] {pod_manager.py:235} INFO - -org-mozilla-fennec-aurora.migration.1.txt
[2023-11-04, 03:19:39 UTC] {pod_manager.py:235} INFO - -org-mozilla-firefox-beta.migration.1.txt
[2023-11-04, 03:19:39 UTC] {pod_manager.py:235} INFO - -org-mozilla-fenix.migration.1.txt
[2023-11-04, 03:19:39 UTC] {pod_manager.py:235} INFO - -org-mozilla-fenix-nightly.migration.1.txt
[2023-11-04, 03:19:39 UTC] {pod_manager.py:235} INFO - -org-mozilla-firefox.migration.1.txt

Comment 2

[Anna Scholtz [:ascholtz]]

This happened due to the changes in https://github.com/mozilla/probe-scraper/commit/760f8d951a41ea840e39e320b75bf3aade70f104 that removed support-migration

Comment 3

[Anna Scholtz [:ascholtz]]

Attachment: GitHub Pull Request