1174565 - [EME] Add telemetry to report which EME files are missing

Status: RESOLVED FIXED

(Core :: Audio/Video, defect, P1)

Description

[Chris Pearce [:cpearce (Not reading bugmail)]]

We have telemetry to report when GMP DLL files disappear, but we don't know report when the plugin DLL's voucher file disappears. We've had reports from the field that the voucher file can disappear but the DLL and info files are still present. We'll want to drive down this number over time, so we need telemetry to report when they disappear.

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1174565-plugin-missing-files-histograms.patch

Follow-up to bug 1156566.
Adding telemetry histograms to track exactly which GMP files are missing. I kept the previous coarse-grained histograms, so we can see the overall trend (and also because I was not sure I could just remove them while they may still be used by previous versions).

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1174565-plugin-missing-files-telemetry.patch

Adding telemetry reporting about which GMP files are missing.

Comment 3

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8627538 [details] [diff] [review]
1174565-plugin-missing-files-histograms.patch

Just thought of something. Delaying review for now.

Comment 4

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8627539 [details] [diff] [review]
1174565-plugin-missing-files-telemetry.patch

Just thought of something. Delaying review for now.

Comment 5

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1174565-plugin-missing-files-histograms.patch

Follow-up to bug 1156566.
Adding telemetry histogram to track exactly which GMP files are missing. I kept the previous coarse-grained histograms, so we can see the overall trend (and also because I was not sure I could just remove them while they may still be used by previous versions).

Comment 6

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1174565-plugin-missing-files-telemetry.patch

Adding telemetry reporting about which GMP files are missing.

Comment 7

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 8627668 [details] [diff] [review]
1174565-plugin-missing-files-telemetry.patch

Review of attachment 8627668 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM, spohl should also review this, he maintains GMPProvider.

Comment 8

[Vladan Djeric (:vladan)]

Comment on attachment 8627666 [details] [diff] [review]
1174565-plugin-missing-files-histograms.patch

Review of attachment 8627666 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/components/telemetry/Histograms.json
@@ +7833,5 @@
> +  "VIDEO_ADOBE_GMP_MISSING_FILES": {
> +    "alert_emails": ["[e-mail]"],
> +    "expires_in_version": "42",
> +    "kind": "enumerated",
> +    "n_values" : 8,

you might want to to declare "n_values" larger than you need it to future-proof it.. it's not possible to re-declare this histogram property later

@@ +7834,5 @@
> +    "alert_emails": ["[e-mail]"],
> +    "expires_in_version": "42",
> +    "kind": "enumerated",
> +    "n_values" : 8,
> +    "description": "Adobe EME GMP files missing (sum of: 1:library, 2:info, 4:voucher)",

will you log a "0" a value when no files are missing? it could provide a denominator for the error rate

@@ +7848,5 @@
> +  "VIDEO_OPENH264_GMP_MISSING_FILES": {
> +    "alert_emails": ["[e-mail]", "[e-mail]"],
> +    "expires_in_version": "42",
> +    "kind": "enumerated",
> +    "n_values" : 4,

ditto

Comment 9

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8627668 [details] [diff] [review]
1174565-plugin-missing-files-telemetry.patch

Following Vladan's comments, I'll change the code a bit to have "0" values; I'll also split this reporting into separate platforms.

Comment 10

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1174565-plugin-missing-files-histograms.patch

(In reply to Vladan Djeric (:vladan) -- please needinfo! from comment #8)
> Comment on attachment 8627666 [details] [diff] [review]
> 1174565-plugin-missing-files-histograms.patch
> 
> Review of attachment 8627666 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: toolkit/components/telemetry/Histograms.json
> @@ +7833,5 @@
> > +    "n_values" : 8,
> you might want to to declare "n_values" larger than you need it to
> future-proof it.. it's not possible to re-declare this histogram property
> later

After discussing it with :cpearce, we're confident this number won't grow larger, as the number of files in plugin most likely won't change.

> @@ +7834,5 @@
> > +    "description": "Adobe EME GMP files missing (sum of: 1:library, 2:info, 4:voucher)",
> 
> will you log a "0" a value when no files are missing? it could provide a
> denominator for the error rate

Good idea! Added "0" value.

We've also split histograms between Windows/Mac/Linux/Other systems.

Comment 11

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1174565-plugin-missing-files-telemetry.patch

Reporting GMP missing files, or 0 for no missing files.
Split telemetry between Windows/Mac/Linux/Other systems.

Comment 12

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 8628055 [details] [diff] [review]
1174565-plugin-missing-files-telemetry.patch

Review of attachment 8628055 [details] [diff] [review]:
-----------------------------------------------------------------

Make sure to check `./mach xpcshell-test toolkit/mozapps/extensions/test/xpcshell/test_gmpProvider.js`.

Comment 13

[Vladan Djeric (:vladan)]

(In reply to Gerald Squelart [:gerald] from comment #10)
> After discussing it with :cpearce, we're confident this number won't grow
> larger, as the number of files in plugin most likely won't change.

That's fine then

> We've also split histograms between Windows/Mac/Linux/Other systems.

You don't need to do that, the Telemetry ping already reports OS details (including OS version) and the Telemetry dash shows the distributions separated by OS, e.g. GC_MAX_PAUSE_MS for Windows systems only http://mzl.la/1T4010p

Comment 14

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8628055 [details] [diff] [review]
1174565-plugin-missing-files-telemetry.patch

Will need another round of changes, as telemetry is already split by OSes.

Comment 15

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1174565-plugin-missing-files-histograms.patch

Adding histograms to track which Adobe/OpenH264 GMP files are missing.
(As per comment 13, no need to manually split by OS.)

Comment 16

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1174565-plugin-missing-files-telemetry.patch

Reporting telemetry for GMP missing files, or 0 for no missing files.
Updated xpcshell tests.

Comment 17

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fffd2c774cb6

Comment 18

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/64ed41dd62fe
https://hg.mozilla.org/integration/mozilla-inbound/rev/edb7fc17176f

Comment 19

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/64ed41dd62fe
https://hg.mozilla.org/mozilla-central/rev/edb7fc17176f