1362521 - Investigate and possibly decommission/replace push-load

Status: RESOLVED FIXED

(Data Platform and Tools Graveyard :: Operations, enhancement)

Description

[Wesley Dawson [:whd]]

Filing in the old component because I'm not sure where it maps to in the new components.

We've got an old push-load one-off :RaFromBRC and :relud set up that I believe populates a redshift database somewhere. It appears to run about 5 times a day. If this job is still running correctly and the redshift database is being used from re:dash, we should probably get it migrated to logging 2.0 / parquet. If the data is not being used or the job is failing, we should decommission it. Last I was aware this was supposed to run daily, but perhaps we bumped the frequency to keep up with load.

Comment 1

[Daniel Thorn [:relud]]

Once we are outputting to parquet in logging 2.0, and bbangert has moved his needed dashboards to rely on

Comment 2

[Daniel Thorn [:relud]]

on the parquet outputs*

Comment 3

[Daniel Thorn [:relud]]

:benbangert does the quicksight stuff you have now sufficiently replace the prod push redshift cluster, and logging 1.0 job to load it?

Comment 4

[Ben Bangert [:benbangert]]

We have sufficient info to replace this now, it can be dropped. Thanks!