1281536 - Un-expire or remove reader mode histogram probes

Status: RESOLVED FIXED

(Toolkit :: Reader Mode, defect, P1)

Description

[:Margaret Leibovic]

The follow probes are expiring in Firefox 50:

* READER_MODE_DOWNLOAD_MS expires in version 50.0a1 (watched by [e-mail]) - Time (ms) to download a document to show in reader mode
* READER_MODE_DOWNLOAD_RESULT expires in version 50.0a1 (watched by [e-mail]) - The result of trying to download a document to show in reader view (0=Success, 1=Error XHR, 2=Error no document)
* READER_MODE_PARSE_RESULT expires in version 50.0a1 (watched by [e-mail]) - The result of trying to parse a document to show in reader view (0=Success, 1=Error too many elements, 2=Error in worker, 3=Error no article)
* READER_MODE_SERIALIZE_DOM_MS expires in version 50.0a1 (watched by [e-mail]) - Time (ms) to serialize a DOM to send to the reader worker
* READER_MODE_WORKER_PARSE_MS expires in version 50.0a1 (watched by [e-mail]) - Time (ms) for the reader worker to parse a document

In an email thread, Gijs proposed removing all of them except READER_MODE_PARSE_RESULT. That sounds fine to me.

Comment 1

[Justin Dolske [:Dolske]]

I don't have a strong opinion here, but I'd suggest keeping both READER_MODE_PARSE_RESULT and READER_MODE_DOWNLOAD_RESULT as general quality/problem indicators. The time probes seem fine to let expire.

Comment 2

[Barbara Bermes [:barbara] - NI please!]

(In reply to Justin Dolske [:Dolske] from comment #1)
> I don't have a strong opinion here, but I'd suggest keeping both
> READER_MODE_PARSE_RESULT and READER_MODE_DOWNLOAD_RESULT as general
> quality/problem indicators. The time probes seem fine to let expire.

+1

Comment 3

[:Gijs (he/him)]

Attachment: Bug 1281536 - remove reader mode telemetry that's out of date, extend two existing probes,

Review commit: https://reviewboard.mozilla.org/r/60544/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/60544/

Comment 4

[:Margaret Leibovic]

Comment on attachment 8764973 [details]
Bug 1281536 - remove reader mode telemetry that's out of date, extend two existing probes,

https://reviewboard.mozilla.org/r/60544/#review57680

::: toolkit/components/telemetry/Histograms.json:9596
(Diff revision 1)
> -    "n_buckets": 60,
> -    "description": "Time (ms) to download a document to show in reader mode"
> -  },
>    "READER_MODE_PARSE_RESULT" : {
> -    "expires_in_version": "50",
> +    "expires_in_version": "54",
>      "alert_emails": ["[e-mail]"],

I think we should update these emails to alert firefox-dev instead of my personal email account :)

Comment 5

[Benjamin Smedberg]

Comment on attachment 8764973 [details]
Bug 1281536 - remove reader mode telemetry that's out of date, extend two existing probes,

https://reviewboard.mozilla.org/r/60544/#review59428

::: toolkit/components/telemetry/Histograms.json:9602
(Diff revision 1)
>      "kind": "enumerated",
>      "n_values": 5,
>      "description": "The result of trying to parse a document to show in reader view (0=Success, 1=Error too many elements, 2=Error in worker, 3=Error no article)"
>    },
>    "READER_MODE_DOWNLOAD_RESULT" : {
> -    "expires_in_version": "50",
> +    "expires_in_version": "54",

Responding to the bug comment: there should be somebody in particular who is responsible for these probes, not just a mailng list. It's fine to also add the list to the email alerts.

Be aware, that the alerting mechanism only works properly for scalar values, so you should not expect alerts for changes here: if this is in fact valuable data, somebody should either be responsible for monitoring it on a regular (weekly?) basis, or developing a real alerting mechanism.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/fx-team/rev/ec92630e4c63
remove reader mode telemetry that's out of date, extend two existing probes, r=margaret,data-r=bsmedberg

Comment 7

[:Gijs (he/him)]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #5)
> Comment on attachment 8764973 [details]
> Bug 1281536 - remove reader mode telemetry that's out of date, extend two
> existing probes,
> 
> https://reviewboard.mozilla.org/r/60544/#review59428
> 
> ::: toolkit/components/telemetry/Histograms.json:9602
> (Diff revision 1)
> >      "kind": "enumerated",
> >      "n_values": 5,
> >      "description": "The result of trying to parse a document to show in reader view (0=Success, 1=Error too many elements, 2=Error in worker, 3=Error no article)"
> >    },
> >    "READER_MODE_DOWNLOAD_RESULT" : {
> > -    "expires_in_version": "50",
> > +    "expires_in_version": "54",
> 
> Responding to the bug comment: there should be somebody in particular who is
> responsible for these probes, not just a mailng list. It's fine to also add
> the list to the email alerts.

Swapped to gijs@ + firefox-dev.

> Be aware, that the alerting mechanism only works properly for scalar values,
> so you should not expect alerts for changes here: if this is in fact
> valuable data, somebody should either be responsible for monitoring it on a
> regular (weekly?) basis, or developing a real alerting mechanism.

Is there a bug tracking making this work in general? Seems like generally, changes in distribution should trigger alerts, not just for scalars. If there is no prospect of that happening I can look at doing something myself or finding a PM who can own looking at this every week, but it'd be nice if we can delegate this to machines in general...

Comment 8

[Benjamin Smedberg]

Better alerting is on the longer-term roadmap for the data engineering team, but it's probably going to be at least three months; there's a bunch of core scheduling and validation work that needs to happen first to support a general solution.

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/ec92630e4c63