1527442 - Create cost estimate for Normandy negative match events

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P1)

Description

[bugzilla]

• Use uptake telemetry to estimate how often normandy runs per day
• Estimate number of additional event pings a "no match" event for each recipe would generate

Comment 1

[bugzilla]

Assuming each Normandy run would generate one extra event ping, and subtracting existing event pings that contain Normandy enrollment events and multiplying by the average event ping size, we're looking at a ~25% increase in total telemetry ping count and a ~8% increase in total bytes pushed through telemetry (using current average event ping size for the estimate).

The cost benefit here doesn't seem to add up. We could consider sampling the no-match events at 1% as we discussed and I believe that cost would be more in line with the benefits we'd get out of it, and would in almost all circumstances be more than enough data to diagnose issues.

Comment 2

[bugzilla]

Here's the notebook where I compiled this data -- it's not really formatted for other people to read but it does show where I got these numbers
https://dbc-caf9527b-e073.cloud.databricks.com/#notebook/83087/command/83170