1566204 - Change treeherder-stage to use Provisioned IOPS storage

Status: RESOLVED WONTFIX

(Tree Management :: Treeherder: Infrastructure, task, P1)

Description

[Armen [:armenzg]]

Same idea as with bug 1565042.
There were no fall outs from it.

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla-platform-ops/devservices-aws/pull/107

Comment 2

[Kendall Libby [:fubar] (he/him)]

What's the cost difference for changing all of these to PIOPS?

Comment 3

[Armen [:armenzg]]

I'm excluding the instance costs and focusing on the storage type
General SSD Multi AZ

• Storage rate $0.23 per GB-month -> 1000 * .23 = $230

Provisioned IOPS Multi-AZ

• Storage rate $0.25 per GB-month -> 1000 * .25 = $250
• IOPS Rate $0.2 per IOPS-month -> 6000 * .2 = $1200

I'm fine with later reducing the beefiness of stage and prototype (which I was planning to), however, having this extra capacity on production will actually be very helpful.

I have plans to keep looking at reducing the impact on the DB from the app perspective which would allow us to keep reducing the IOPS.

Alternatively to this idea is to bump the storage size from 1Tb to 2Tb which would move us from 3,000 IOPS to 6,000 IOPS and would only cost $230/month [1].

Would this alternative approach work for you?

Baseline I/O performance for General Purpose SSD storage is 3 IOPS for each GiB. This relationship means that larger volumes have better performance. For example, baseline performance for a 100-GiB volume is 300 IOPS. Baseline performance for a 1-TiB volume is 3,000 IOPS. And baseline performance for a 5.34-TiB volume is 16,000 IOPS.

Comment 4

[Kendall Libby [:fubar] (he/him)]

Bumping them up to 2tb sounds preferable. Thanks!

Comment 5

[Armen [:armenzg]]

Sounds good! My apologies as I did not realize how much more expensive it is.
Let's try that in bug 1567257.