1592879 - Allow access to Tiles Redshift instance from snippets admin instances

Status: RESOLVED WONTFIX

(Data Platform and Tools Graveyard :: Operations, enhancement)

Description

[Josh Mize [:jgmize]]

We recently started pulling snippets performance metrics from in the Tiles Redshift instance via redash[0], but now that we will soon be pulling in more data[1], it may make more sense to pull it from Redshift directly, similar to the original solution for bug 1272388.

[0] https://github.com/mozmeao/snippets-service/pull/1174
[1] https://github.com/mozmeao/snippets-service/pull/1215

Comment 1

[Jeff Klukas [:klukas] (UTC-4)]

I spoke with :jgmize and :nanj about this plan, so I can give some more context here.

I had previously spoken with :nanj about what we thought was a need to have an ongoing import from the Tiles Redshift database to BigQuery, but further conversation made it clear that Pocket doesn't need that data in BQ and the only identified need left was for snippets data. :jgmize proposed a solution where the snippet admin instances are responsible for pulling the necessary data from BQ and from Tiles Redshift, obviating the need for a Redshift -> BQ pipeline.

This plan sounds great to me and :nanj and will save work for data ops / data platform.

I think Data Ops might not be the right place for this particular request about accessing Tiles Redshift. I asked :jgmize to follow up with :jbuck about who needs to be involved to provision that access. I'm going to file an additional bug, though, for BQ creds for the snippets admin instances.

Comment 2

[Jon Buckley [:jbuck]]

This data lives in Athena now, and can also be accessed with STMO.