1555352 - Validate data after clearing metrics on disabling lands

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, task, P1)

Description

[Michael Droettboom [:mdroettboom]]

We should validate the incoming data looking for any weird behaviors due to landing the feature that clears metrics when they are disabled.

Things to check include:

• Pings with the c0ffee client_id.

• Pings sent without required application lifetime metrics (which will probably be rejected by the JSON schema)

:chutten has more background in this and could probably suggest more good things to check.

Comment 1

[Chris H-C :chutten]

I'm afraid I'm missing context on what it means that metrics are cleared when reporting is disabled. Which metrics? Cleared how?

Comment 2

[Michael Droettboom [:mdroettboom]]

When the user disables (opts-out) of telemetry, most metrics persisted to disk and in memory at that point are cleared, and the client_id is set to the infamous "c0ffee" value. More details here.

chutten: I have needinfo'd you thinking you might have experience verifying such things on desktop so could fill in some more things we should look at. If you have nothing to add, no worries :)

Comment 3

[Chris H-C :chutten]

For just the "clearing things out" part, I think you have it covered: Check for the canary, Check that things sent after opting back in have the newly-created data...

Might be worth double-checking that it doesn't get tripped too frequently. If you see the client count inflate when this lands that could be evidence that the client id is being reset too often, for example. This would also exhibit as a left-shift in the distribution of seq numbers and sequence lengths overall.