1295651 - Add alert emails for accessibility telemetry probe(s)

Status: NEW

(Core :: Disability Access APIs, defect, P3)

Description

[David Bolter [:davidb] (NeedInfo me for attention)]

Thanks for raising this chutten!

In here: https://dxr.mozilla.org/mozilla-central/source/toolkit/components/telemetry/Histograms.json

Current probes are:
A11Y_INSTANTIATED_FLAG -has accessibility support been instantiated
A11Y_CONSUMERS - Accessibility client by enum id
A11Y_ISIMPLEDOM_USAGE_FLAG - have the ISimpleDOM* accessibility interfaces been used
A11Y_IATABLE_USAGE_FLAG - has the IAccessibleTable accessibility interface been used
A11Y_UPDATE_TIME - time spent updating accessibility (ms)

Comment 1

[David Bolter [:davidb] (NeedInfo me for attention)]

Chris how are alerts triggered?

Comment 2

[Chris H-C :chutten]

Changes in Nightly histogram distributions are recognized by cerberus [1]. medusa [2] then takes care of sending out the alerts in a timely (and, in the case of expiring probes, recurring) fashion.

In short: There's some fancy stats we throw at a computing cluster every day that looks at what the Nightly histograms looked like two days ago, and what they looked like the seven days previous. If there's a marked difference, cerberus records it and medusa alerts on it.

This is where the humans take over, as the change might be legit, might be an improvement, or might just be noise (though it's been a while since it's been noise). In trying to find humans to figure this out, it alerts everyone in the alert_emails field of Histograms.json (which it requests a new copy of every day) plus the dev-telemetry-alerts mailing list. 

Triaging the dev-telemetry-alerts mailing list is a human (me) who looks at the changes coming in at some random interval, trying to find owners and explanations for all the alerts that no one's replied to on-list.

[1]: https://github.com/mozilla/cerberus
[2]: https://github.com/mozilla/medusa