1799725 - Backfill unified_metrics_v1

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement)

Description

[Alekhya Kommasani [:Alekhya]]

telemetry_derived.unified_metrics_v1 needs to be backfilled due to multiple changes:

1. the upstream mobile_search_clients_daily_v1 has been backfilled to add Focus Android Glean and Focus iOS Glean
2. Columns uri_count, is_default_browser, and isp were populated in upstream tables

The following are the backfill commands

./bqetl query backfill telemetry_derived.unified_metrics_v1 --project_id=moz-fx-data-shared-prod -s 2021-01-01 -e 2022-11-30 -n 0

As a downstream to unified_metrics_v1 active_users_aggregates_v1 table also needs backfill
./bqetl query backfill telemetry_derived.active_users_aggregates_v1 --project_id=moz-fx-data-shared-prod -s 2021-01-01 -e 2022-11-30 -n 0

Comment 1

[Anna Scholtz [:ascholtz]]

It looks like the is_default_browser field is used in a couple more downstream tables:

• telemetry_derived.active_users_aggregates_attribution_v1
• telemetry_derived.cohort_daily_statistics_v1
• telemetry_derived.rolling_cohorts_v1

Comment 2

[Alekhya Kommasani [:Alekhya]]

good catch!

I will add the below commands

./bqetl query backfill telemetry_derived.active_users_aggregates_attribution_v1 --project_id=moz-fx-data-shared-prod -s 2021-01-01 -e 2022-11-30 -n 0
./bqetl query backfill telemetry_derived.cohort_daily_statistics_v1 --project_id=moz-fx-data-shared-prod -s 2021-01-01 -e 2022-11-30 -n 0
./bqetl query backfill telemetry_derived.rolling_cohorts_v1 --project_id=moz-fx-data-shared-prod -s 2021-01-01 -e 2022-11-30 -n 0

Comment 3

[Anna Scholtz [:ascholtz]]

lgtm

Comment 4

[Lucia Vargas [:lucia-vargas-a]]

Also required:
./bqetl query backfill telemetry_derived.active_users_aggregates_device_v1 --project_id=moz-fx-data-shared-prod -s 2021-01-01 -e 2022-11-30 -n 0
./bqetl query backfill telemetry_derived.active_users_aggregates_v1 --project_id=moz-fx-data-shared-prod -s 2021-01-01 -e 2022-11-30 -n 0

Backfilling from 2021-01-01 to align with the upstream update.

Comment 5

[Lucia Vargas [:lucia-vargas-a]]

telemetry_derived.active_users_aggregates_attribution_v1 completed for 2021.

Comment 6

[Anna Scholtz [:ascholtz]]

:lvargas Has this backfill been completed? Can we close the ticket?

Comment 7

[Lucia Vargas [:lucia-vargas-a]]

We can close this ticket.
New backfill will be required by week of Apr 3rd as part of the move to use Glean data for Mobile.