Closed Bug 1561043 Opened 6 years ago Closed 6 years ago

Port active_profiles dataset to use BigQuery source

Categories

(Data Platform and Tools :: General, defect, P3)

Product:
Data Platform and Tools
Component:
General
Type:
defect
Points:
2

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: amiyaguchi, Unassigned)

References

Details

The active_profiles dataset is generated by a job defined in wcbeard/bgbb_airflow. This job generates a frequency and recency table generated from clients_daily in wcbeard/bgbb_lib.

The frequency and recency table can be read from the clients_daily table in BigQuery. This would allow porting the entire airflow job to run within dataproc.

https://github.com/wcbeard/bgbb_lib/blob/master/bgbb/sql/sql_utils.py#L64-L157

Points: --- → 2
Priority: -- → P3
Blocks: 1596205
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Component: Datasets: General → General
You need to log in before you can comment on or make changes to this bug.