1499753 - Add canary client ID alert

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Jan-Erik Rediger [:janerik]]

Now that the canary client ID bugfix is in and soon rolling out to stable, we want to add an automatic alert to ensure:

1) the rate of pings with canary client ID goes down
2) we don't see pings with canary client ID from builds with a buildid after 20180917220115 (the buildid the fix went in)

Comment 1

[Jan-Erik Rediger [:janerik]]

Mike, could you help in getting this monitor set up?

Comment 2

[Mike Trinkala [:trink]]

This specification needs to be more precise, in a quick sample (less than a minute) here is what would have alerted (normalized channel, buildId, canary count):
nightly
    20181021100613 = 44
    20181004100221 = 1
    20181017101852 = 9
    20181018103737 = 14
    20181019100100 = 3
    20181010100126 = 1
    20180918100025 = 1
    20181017101626 = 1
    20181020102546 = 22
    20181022100054 = 32
    20181009183348 = 1
    20181008100120 = 3
    20181014100136 = 6
    20181016100107 = 10
beta
    20180924202103 = 18
    20181015152921 = 456
    20181008155858 = 62
    20181001131022 = 57
release
    20181018182531 = 12

Comment 3

[Jan-Erik Rediger [:janerik]]

Oh weird.
I took a look at these pings using hindsight and it looks those are all `saved-session` pings from Fennec (Firefox for Android), thus it seems like the problem there persists.
I'll look into the problem with Android.
If it proves true I'll give you more detailed info for the monitor to capture desktop for now.

Comment 4

[Chris H-C :chutten]

Marking as P1 since it is currently being worked on.

Comment 5

[Jan-Erik Rediger [:janerik]]

I now have a working monitor (jrediger_mozilla_com.canary_monitor_desktop), based on :trink's previous code, updated with the correct build IDs and alerting to me (for now).