1250013 - create production sync metrics redshift instance

Status: RESOLVED INVALID

(Firefox :: Sync, defect, P2)

Description

[Danny Coates [:dcoates]]

Create or find a Redshift instance to use for beta testing sync metrics in production.

Comment 1

[Danny Coates [:dcoates]]

An estimate on cluster size...

Given the flexibility with resizing I'm comfortable starting with a super small cluster.

https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html#cluster-resize-intro

Its hard to estimate upfront a lower bound for how much storage we'll need given redshift's compression, but the upper bound will likely be in the mid 10s of GB per day for full production traffic. A dc1.large has 160GB of storage, so at 40GB per day we'd need 8 nodes for 30 days of data.

To beta test against 1 sync node a single dc1.large should be sufficient and will give us a better idea of what to expect from the full cluster.

:bobm :whd f? next steps?

Comment 2

[Bob Micheletto [:bobm]]

I believe this falls under WHD's purview, so I'm clearing my needinfo.  I will monitor and assist as required.

Comment 3

[Wesley Dawson [:whd]]

I need to parameterize the provisioning logic we use for redshift clusters for this and bug #1251097, which I will do next week. Is the plan here to instrument a single sync production node for this test? If so this is going to require bobm to do the instrumentation.

Comment 4

[Danny Coates [:dcoates]]

> I need to parameterize the provisioning logic we use for redshift clusters for this and bug #1251097, which I will do next week.

Great! I won't need an instance until after fxa train-57 is in prod (at least 2 weeks)

> Is the plan here to instrument a single sync production node for this test?

yes. I'll start another bug for wiring it up.

Comment 5

[Danny Coates [:dcoates]]

superseded by bug 1291340