1972339 - cert.validation_success_by_ca bin collision between CAs and unknown entries in RootCertificateTelemetryUtils.h

Status: RESOLVED FIXED

(Core :: Security: PSM, defect, P1)

Description

[Matthew McPherrin]

Recently I was looking at the cert.validation_success_by_ca metric in Glam.

I noticed that in the histogram data, Bin 1 had a lot of entries which seemed much higher than I'd expect for the CA assigned that bin (Kamu SM).

In https://github.com/mozilla-firefox/firefox/blame/main/security/manager/ssl/RootCertificateTelemetryUtils.h
that's ROOT_CERTIFICATE_SOFTOKEN

These were split out of the unknown bin 0 in https://bugzilla.mozilla.org/show_bug.cgi?id=1576755

Bins 1, 2, 3 were assigned to active CAs in https://bugzilla.mozilla.org/show_bug.cgi?id=1909978 and https://bugzilla.mozilla.org/show_bug.cgi?id=1915285 so now it looks like both real CAs and the special cases

As well, that conflicts with the comment at the top of
https://github.com/mozilla-firefox/firefox/blob/main/security/manager/tools/KnownRootHashes.json

So I think three real CAs and the various unknown cases are overlapping in the data.

Comment 1

[Dana Keeler (she/her) [:keeler]]

Attachment: Bug 1972339 - convert CA telemetry ID generation to python script r?jschanck

This patch replaces the xpcshell script that generated CA telemetry IDs with
an implementation in python that directly reads the certdata.txt file that
provides the data for the builtin roots module.

This patch also revs the telemetry metrics cert.validation_success_by_ca,
cert_pinning.failures_by_ca, and ssl.ct_policy_non_compliant_connections_by_ca,
replacing them each with ..._by_ca_2 versions.

Additionally, this patch retires the legacy histograms previously mirrored by
the replaced metrics.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/6fb22513f4fa
https://hg.mozilla.org/integration/autoland/rev/871352e1a376
convert CA telemetry ID generation to python script r=jschanck,frontend-codestyle-reviewers

Comment 3

[amarc]

https://hg.mozilla.org/mozilla-central/rev/871352e1a376