1373190 - Do budget estimate for health ping

Status: RESOLVED WORKSFORME

(Toolkit :: Telemetry, enhancement, P1)

Description

[Georg Fritzsche [:gfritzsche]]

We need to do a budget estimate for the health ping before landing it.

This will involve:
- doing an upper-bound estimate of the full health ping from [1]
- getting numbers from re:dash [2] on how often existing errors occur in main pings now
- using this to project a rough estimate of what ping volume & storage size we can expect

1: https://docs.google.com/document/d/15__ODWaL_W5-qRgeUzua4OZzYRo3ZQJ-qdpN58hNfhc/edit#
2: https://sql.telemetry.mozilla.org/

Comment 1

[Kate Ustiuzhanina]

See this https://docs.google.com/document/d/14JY5k1FDS5o3-Z3jLRRXOXnZOaJuwoSCn0fk2DIjMNw document.

Comment 2

[Georg Fritzsche [:gfritzsche]]

(In reply to Kate Ustiuzhanina from comment #1)
> See this
> https://docs.google.com/document/d/14JY5k1FDS5o3-
> Z3jLRRXOXnZOaJuwoSCn0fk2DIjMNw document.

Chris, can you review this and the linked gist?

Comment 3

[Chris H-C :chutten]

Looks correct and straightforward. I'm not sure if the compression ratio is valid, but even uncompressed its value to us likely outstrips the cost.

As always, ask :mreid for budget OK.

Comment 4

[Georg Fritzsche [:gfritzsche]]

Mark, here is a budget heads-up.

Summary for budget impact:
- Uncompressed storage volume: ~31 gb/month
- Compressed (assuming compression ratio as 1:3): ~10 gb/month
- Compressed cost: ~3$/month

Comment 5

[Mark Reid [:mreid]]

If we're accumulating 10GB of data per month, we need to multiply by our retention period (currently 12 months), which gives us the total amount we'll be billed for each month.

So that brings it up to $36/mo, which still seems totally fine to me :)

Comment 6

[Georg Fritzsche [:gfritzsche]]

Great, sounds like we're done here.