1362187 - Add Telemetry for MathML

Status: RESOLVED FIXED

(Core :: MathML, enhancement, P5)

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Tom:
I'm starting in Layout, and it was suggested I could possibly tackle adding telemetry for MathML (having some experience with telemetry, but none yet with layout&MathML); I was about to open a bug, and I noticed yours here.

Have you done any work on it yet?

If not, is that ok if I take over this bug?
In which case: Anything in particular you had in mind? I'm thinking of at least having some sort of ping every time a MathML expression is used; please let me know if you have suggestions.

Comment 2

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

(In reply to Gerald Squelart [:gerald] from comment #1)
> Tom:
> I'm starting in Layout, and it was suggested I could possibly tackle adding
> telemetry for MathML (having some experience with telemetry, but none yet
> with layout&MathML); I was about to open a bug, and I noticed yours here.
> 
> Have you done any work on it yet?

No.

> If not, is that ok if I take over this bug?

Yes!

> In which case: Anything in particular you had in mind? I'm thinking of at
> least having some sort of ping every time a MathML expression is used;
> please let me know if you have suggestions.

You'll need to get a Data Steward review for adding any Telemetry probes, and they can guide you in this.  They will ask what the goal is. I believe the goal is "Determine what percentage of page loads use MathML"

My suggestion is to use a scalar (https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/scalars.html) and increment it for every page that has a MathML equation (that is, even if the page contains 50 expressions, increment it only once). This count can be divided by another probe (I forget what the example given to me was) that triggers once per page load to get a percentage.  

But again, a Data Steward will be able to guide you a little bit better on the specifics.

Comment 3

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1362187 - DOC_HAD_MATHML telemetry -

Every time a document is destroyed, we record whether MathML was enabled,
which is set to true when a MathML element was bound to the document.

Review commit: https://reviewboard.mozilla.org/r/192300/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/192300/

Comment 4

[François Marier [:francois]]

Comment on attachment 8921279 [details]
Bug 1362187 - DOC_HAD_MATHML telemetry -

https://reviewboard.mozilla.org/r/192300/#review197452

This is Category 1 data.

datareview+

Comment 5

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Thank you for the suggestion Tom.
I went for just recording whether each document had a MathML element, which is very easy (thanks to the existing nsIDocument::GetMathMLEnabled()), and should be good enough to give us an initial indication of the level of usage of MathML in the field.

Thank you Francois for the quick data review!

Comment 6

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Comment on attachment 8921279 [details]
Bug 1362187 - DOC_HAD_MATHML telemetry -

https://reviewboard.mozilla.org/r/192300/#review197754

::: dom/base/nsDocument.cpp:1775
(Diff revision 1)
>        }
>        if (mHasUnsafeEvalCSP) {
>          Accumulate(Telemetry::CSP_UNSAFE_EVAL_DOCUMENTS_COUNT, 1);
>        }
> +
> +      Accumulate(Telemetry::DOC_HAD_MATHML, GetMathMLEnabled());

This approach is 'ok' but it could be made more efficient.

Here we'll be incrementing and reporting data about MathML on every page destruction. That's a lot. The advantage of this is when we go to t.m.o we'll see a very straightforward chart: X% has it and 100-X% doesn't.

We can make this more efficient by only reporting when MathML is used. (And instead of using a boolean, using a scalar.) Then when we go to t.m.o we'll see a number like "5000" uses of MathML. That's the uses, the numerator, but we don't have a denominator!  However, we happen to have the denominator in another probe nearby: MIXED_CONTENT_PAGE_LOAD which _is_ accumulated on every page destruction.

So I think we should change this to be a scalar and only accumulate if GetMathMLEnabled() is true.

The disadvantage of this is our graph in t.m.o won't give us data at a glance and we'll need to do some custom analysis. However, as chutten told me: "That being said, you probably want to identify maximal usage, not aggregate usage, and to figure out how many users are using mathml at those frequent levels... which requires custom analysis anyway"

So since custom analysis is a given, let's made the code just a wee bit more streamlined. If you want to be fancy you could probably use MOZ_UNLIKELY() in the branch.

::: toolkit/components/telemetry/Histograms.json:10312
(Diff revision 1)
>      "kind": "enumerated",
>      "n_values": 10,
>      "description": "How often does a request result in HSTS priming? (0=Sent HSTS priming, 1=No priming, 2=Priming skipped due to cached HSTS, 3=Priming skipped due to cached NO HSTS, 4=Priming failed (request error), 5=Priming skipped (missing load info), 6=Priming skipped (already upgraded)"
>    },
> +  "DOC_HAD_MATHML": {
> +    "record_in_processes": ["main", "content"],

You should be able to remove 'main' from here

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1362187 - mathml.doc_count scalar telemetry -

Every time a document is destroyed, we record whether MathML was enabled,
which is set to true when a MathML element was bound to the document.
'mathml.doc_count' will report how many such documents existed during a
session. It should be compared to MIXED_CONTENT_PAGE_LOAD, which counts the total number of all page loads.

Review commit: https://reviewboard.mozilla.org/r/192734/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/192734/

Comment 8

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Comment on attachment 8921729 [details]
Bug 1362187 - mathml.doc_count scalar telemetry -

https://reviewboard.mozilla.org/r/192734/#review197912

Comment 9

[François Marier [:francois]]

Comment on attachment 8921729 [details]
Bug 1362187 - mathml.doc_count scalar telemetry -

https://reviewboard.mozilla.org/r/192734/#review198240

Category 1 data.

datareview+

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/43ee744b0a2d
mathml.doc_count scalar telemetry - r=francois,tjr

Comment 11

[Attila Craciun [:arny]]

https://hg.mozilla.org/mozilla-central/rev/43ee744b0a2d