1345217 - [Churn] Separate csv uploads to s3 from churn job

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, enhancement, P1)

Description

[Anthony Miyaguchi [:amiyaguchi]]

The Churn notebook generates both parquet and csv data formats for consumption. However, csv will soon lose relevancy once data is consumed from telemetry-parquet.

Separating the logic between generating the data and creating csv files for consumption will make it easier to kill csv as a file format in the future. This also makes the dependencies more clear.

Comment 1

[Anthony Miyaguchi [:amiyaguchi]]

:mreid, can you confirm that the two locations specified in the churn notebook are currently being used?

* net-mozaws-prod-us-west-2-pipeline-analysis/mreid/churn/
* net-mozaws-prod-metrics-data/telemetry-churn/

Comment 2

[Anthony Miyaguchi [:amiyaguchi]]

Attachment: Bug 1345217 - Separate csv uploads to s3 from churn job #36

Comment 3

[Mark Reid [:mreid]]

(In reply to Anthony Miyaguchi [:amiyaguchi] from comment #1)
> :mreid, can you confirm that the two locations specified in the churn
> notebook are currently being used?
> 
> * net-mozaws-prod-us-west-2-pipeline-analysis/mreid/churn/
Yes, this is the location from which data is fetched to be imported into Tableau.

> * net-mozaws-prod-metrics-data/telemetry-churn/
This is the location that automagically relays to the metrics.services.mozilla.com server. I don't think anything is using this data. Matt, can you confirm that nothing is hooked up to that dataset via metrics.s.m.c?

Comment 4

[Matt Pressman [:mpressman]]

Yep, I an confirm that there is not anything using net-mozaws-prod-metrics-data/telemetry-churn/

Comment 5

[Anthony Miyaguchi [:amiyaguchi]]

Attachment: Bug 1345217 - Add churn_to_csv downstream from churn #95

Pull request for mozilla/telemetry-airflow.