1513473 - Re-enable the CHARSET_OVERRIDE_USED and CHARSET_OVERRIDE_SITUATION telemetry probes on the release channel

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[Henri Sivonen (:hsivonen)]

At some point, old probes like CHARSET_OVERRIDE_USED and CHARSET_OVERRIDE_SITUATION were excluded from participating in release channel telemetry collection.

There is reason to believe that the beta channel usage distribution by locale may differ significantly from release channel usage per locale (this is a guess; I haven't seen data) and there is a reason to believe that users on different release channels have different levels of awareness of how to operate the feature.

Concretely, I'd like to understand the success of bug 977540 and bug 1071816 by observing the release-channel 65 to 66 difference in these probes and I'd like to use data from 66 on the release channel to compare the results for Ukrainian and Russian locales on one hand with the rest of the Cyrillic locales on the other hand to infer whether the Russian and Ukrainian encoding detectors are hurting more than they help (bug 845791).

Comment 1

[Henri Sivonen (:hsivonen)]

Attachment: Data review form

Comment 2

[Henri Sivonen (:hsivonen)]

Attachment: Bug 1513473 - Re-enable the Text Encoding menu telemetry probes on the release channel. datareview=chutten.

Comment 3

[Henri Sivonen (:hsivonen)]

Attachment: Data review form, v2 (suggesting to increase expiry every 6 months)

Revised the form to say that while I want to monitor these probes until we get to remove the Text Encoding menu, I'm OK with having to adjust expiry every 6 months instead of requesting never-expiring probes.

Comment 4

[Chris H-C :chutten]

Comment on attachment 9031030 [details]
Data review form, v2 (suggesting to increase expiry every 6 months)

Ack, sorry for the delay. (mozflu)

It seems as though you may have missed question 3 "What alternative methods did you consider to answer these questions? Why were they not sufficient?". We're looking for something along the lines of "We can't survey the web, surveying users individually is more privacy-invading (and prone to error), this is the only way we thought of that can measure the usage of the feature in Firefox."

I can't grant data review+ without answers to all of the questions. (Though your form looks peachy otherwise)

Comment 5

[Henri Sivonen (:hsivonen)]

Attachment: Data review form, v3 (with answer to the question I missed)

Comment 6

[Chris H-C :chutten]

Comment on attachment 9031924 [details]
Data review form, v3 (with answer to the question I missed)

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate? 

Yes. These collections are Telemetry so are documented in their definitions files (Histograms.json, Scalars.yaml), the Probe Dictionary, and on telemetry.mozilla.org's Measurement Dashboards.

    Is there a control mechanism that allows the user to turn the data collection on and off? 

Yes. These collections are Telemetry so can be controlled by Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

The collections will expire in Firefox 69.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Type 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

    Is the data collection covered by the existing Firefox privacy notice? 

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data? 

Yes. :hsivonen is responsible for renewing or removing the collections before they expire in Firefox 69.

---
Result: datareview+

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ae0f6d5379fc
Re-enable the Text Encoding menu telemetry probes on the release channel. datareview=chutten. r=janerik

Comment 8

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/ae0f6d5379fc

Comment 9

[Henri Sivonen (:hsivonen)]

Comment on attachment 9030654 [details]
Bug 1513473 - Re-enable the Text Encoding menu telemetry probes on the release channel. datareview=chutten.

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: None

User impact if declined: More difficult to assess the impact of bug 977540 and bug 1071816, which makes it harder to know if things improved for users.

Is this code covered by automated tests?: No

Has the fix been verified in Nightly?: No

Needs manual test from QE?: No

If yes, steps to reproduce: 

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): Only changes telemetry probes

String changes made/needed: None

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9030654 [details]
Bug 1513473 - Re-enable the Text Encoding menu telemetry probes on the release channel. datareview=chutten.

[Triage Comment]
Updates some probes and enables them on release in addition to Beta. Approved for 65.0b6.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/2f5faf2206df