FxA Metrics ETL: CDP metrics daily export to BigQuery
Categories
(Data Platform and Tools :: General, task, P3)
Tracking
(Not tracked)
People
(Reporter: klukas, Unassigned)
References
Details
See overview of this work captured in https://docs.google.com/document/d/1UNulczGgBSepy0V1YISAfEuczbtvXaR59HB7xwTCFCk/edit#heading=h.kubxlc4i0f3e
See overview of FxA data need for CDP in https://bugzilla.mozilla.org/show_bug.cgi?id=1573359
Marking this as P3 for now as we try to understand more about the status of the CDP project and whether this work has sufficient ROI.
|
Reporter | |
Comment 1•5 years ago
|
||
There is an existing view on top of the raw Stackdriver BQ logs codified in cloudops-infra. This may be sufficient and not require any additional work.
Of note, that query is accessing data with unhashed IDs (which is necessary for CDP). We could potentially write a query in bigquery-etl that writes out to a separate CDP-specific project that regular analytics users would not have access to, but we would need to hash IDs for anything that we store within the main shared-prod project.
|
|
Reporter | |
Comment 2•5 years ago
|
||
I've been gathering context about CDP and my conclusion is that the CDP implementation that's being targeted here was intended as a proof of concept and it's unlikely that we will prioritize any work to improve or port over support.
I will do some investigation to see whether there is access logging that would be able to tell us whether the current implementation is even being used.
|
|
Reporter | |
Comment 3•5 years ago
|
||
Given discussion in https://bugzilla.mozilla.org/show_bug.cgi?id=1620321 the per_service_user_activity view is indeed being looked at currently.
|
|
Reporter | |
Comment 4•4 years ago
|
||
Closing as WONTFIX for now. We can reopen if the conversation around CDP's future indicates further work here is needed, but I find it likely that CDP will take some different shape that won't require this specific task.
Description
•