1707965 - Decommission attitudes_daily

Status: RESOLVED FIXED

(Data Platform and Tools :: General, task, P2)

Description

[Jeff Klukas [:klukas] (UTC-4)]

The attitudes_daily pipeline is no longer being actively used, and after discussion with Roseanne Scholl we're okayed to tear it down. Note that Mikal and Selena are interested in similar areas of investigation, so User Research will likely be building a new survey structure in the same vein later this year. For that effort, we can hopefully build out a more general Alchemer->BigQuery integration, guided by the attitudes_daily implementation.

We'll remove the code, but document the deletion in https://github.com/mozilla/bigquery-etl/blob/main/GRAVEYARD.md so that we can easily refer back to it for future survey integrations.

Comment 1

[Jeff Klukas [:klukas] (UTC-4)]

I've toggled off the attitudes_daily DAG in Airflow: https://workflow.telemetry.mozilla.org/tree?dag_id=attitudes_daily

So the surveygizmo import and query will no longer run. I'll pursue retiring the code in coming days.

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/telemetry-airflow/pull/1299

Comment 3

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/bigquery-etl/pull/2003

Comment 4

[Jeff Klukas [:klukas] (UTC-4)]

PRs are linked here and open for review. Once those are both approved, I will merge them and then manually delete the attitudes_daily and surveygizmo_daily_attitudes tables.

Comment 5

[Jeff Klukas [:klukas] (UTC-4)]

PRs are merged and the tables have been removed. I also sent an announcement to the fx-data-dev list.