1022528 - Add telemetry probe for usage of High Contrast OS themes

Status: RESOLVED FIXED

(Firefox :: General, defect)

Description

[Blair McBride [:Unfocused] (UNAVAILABLE)]

It'd be useful to know how often High Contrast themes are used. This could be expanded to measure usage of important theme distinctions such as Windows Classic also.


According to bug 582951, on Windows, High Contrast themes can be detected via the registry entry:

[HKEY_CURRENT_USER\Control Panel\Accessibility\HighContrast]
"Flags"="127"

Comment 1

[Dão Gottwald [:dao]]

(In reply to Blair McBride [:Unfocused] from comment #0)
> It'd be useful to know how often High Contrast themes are used.

What would it be useful for? Are you concerned that they may be close to unused? They're not used a lot compared to other themes, that's for sure; they are a minority feature by nature. Accessibility is a core value for Mozilla (mentioned in the manifesto, dunno if this is formalized in some other place), making it a priority regardless of popularity.

Comment 2

[Justin Dolske [:Dolske]]

I think we generally just want to understand how many users this impacts, and track any changes over time.

I agree that as a core Mozilla principle we can not say "oh, this is only used by just a few people so who cares." But it can help us with prioritizing work, and the extent of that work. EG, the breakdown bug this blocks (bug 1016556) has a range of things from between "terribly broken and should fix ASAP" and "would be nice to have."

Comment 3

[Eitan Isaacson [:eeejay]]

Attachment: Bug 1022528 - Add probe for high contrast theme.

Comment 4

[Eitan Isaacson [:eeejay]]

Comment on attachment 9050099 [details]
Bug 1022528 - Add probe for high contrast theme.

Before I ping the peers on this module, I just want to get input to see if i am telemetrying right.

The goal is to gather a boolean flag of "is high contrast mode turned on?". This results are broken up into three buckets for the "use document colors" pref. This is so we use both axis of data with one probe.

Comment 5

[Jan-Erik Rediger [:janerik]]

Comment on attachment 9050099 [details]
Bug 1022528 - Add probe for high contrast theme.

In general the telemetry code looks good.
I figure that PreferenceSheet::Initialize() is called exactly once per session. If so, that value is reported once after start, but not for successive subsessions (see the Telemetry (sub)session concept). Is this enough to answer your question?

You should think about whether this really needs to be a never-expiring probe or can go with an expire in half a year or so and then to reevaluate whether this collection still makes sense (we prefer expiring probes over non-expiring to keep our data small and relevant)

Comment 6

[Jan-Erik Rediger [:janerik]]

Oh, one thing I forgot: this will require data-review before it is landed.

Comment 7

[Yura Zenevich [:yzen] (please use "needinfo")]

Comment on attachment 9050099 [details]
Bug 1022528 - Add probe for high contrast theme.

I think you should probably set to expire in +5 release and open a bug to re-visit then.

Same concern: do you care about dynamic setting of this property? Will the pref change dynamically? In that case you would need to update the telemetry value appropriately when that happens.

Comment 8

[Eitan Isaacson [:eeejay]]

(In reply to Jan-Erik Rediger [:janerik] from comment #5)

You should think about whether this really needs to be a never-expiring probe or can go with an expire in half a year or so and then to reevaluate whether this collection still makes sense (we prefer expiring probes over non-expiring to keep our data small and relevant)

(In reply to Yura Zenevich [:yzen] from comment #7)

Same concern: do you care about dynamic setting of this property? Will the pref change dynamically? In that case you would need to update the telemetry value appropriately when that happens.

I think once per startup session is fine. This gives us a simple 1:1 picture of how high contrast is generally used. There may be a few outliers there that switch the pref back and fourth, but if we record that it would fudge the data about how many systems are configured to hc.

Comment 9

[Eitan Isaacson [:eeejay]]

Request for data collection review form

1. What questions will you answer with this data?

This will tell us:

• what percent of our users use a platform accessibility theme (aka high contrast)
• what percent of our users override the styling of websites with our color settings.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

Establish baseline for our current "high contrast" support as we further develop it and track its usage over time.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

We got some public information from Microsoft about the percentage of users who enable high contrast themes. But it is not enough.
We want to further understand if we see a similar ratio in our user population. If we see a smaller number it can inform us that high-contrast users are not well accommodated in Firefox, for example. By examining who overrides document colors in Firefox explicitly (or who turns that feature off explicitly), we can further learn about the favorability of our current high-contrast support.

4. Can current instrumentation answer these questions?

No.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the Mozilla wiki.

Measurement Description	Data Collection Category	Tracking Bug #
Does the user have a platform high contrast theme set?	2	Bug 1022528
Did the user override the document color settings?	2	Bug 1022528

6. How long will this data be collected? Choose one of the following:

This is scoped to a time-limited experiment/project until version 73.

7. What populations will you measure?

• Which release channels?

All

• Which countries?

All

• Which locales?

All

8. If this data collection is default on, what is the opt-out mechanism for users?

The regular Firefox telemetry opt out option, I assume?

9. Please provide a general description of how you will analyze this data.

We are interested in proportional numbers of user for each flag/setting.

10. Where do you intend to share the results of your analysis?

I don't think this is sensitive data, and I can forsee sharing this publicly, but it doesn't have to be.

11. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?

No.

Comment 10

[Jan-Erik Rediger [:janerik]]

You will need to explicitely request data-review on the form. If you attach the form as a file you will have the option to request data-review there.

Comment 11

[Eitan Isaacson [:eeejay]]

Attachment: Request for data collection review form

Comment 12

[Jan-Erik Rediger [:janerik]]

Unfortunately I am not a data steward, so can't do a review (see https://wiki.mozilla.org/Firefox/Data_Collection). With :chutten out, the next best person would be :liuche. I'm going ahead and move the flag.

Comment 13

[Chenxia Liu [:liuche]]

Comment on attachment 9051048 [details]
Request for data collection review form

data-review+

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?
Yes, in Scalars.yaml

2) Is there a control mechanism that allows the user to turn the data collection on and off?
Yes, Scalars.yaml is controlled by Firefox-level data controls

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**
Expires in 73

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **
Type 2, user selected settings

5) Is the data collection request for default-on or default-off?
default on

6) Does the instrumentation include the addition of **any *new* identifiers**?
No

7) Is the data collection covered by the existing Firefox privacy notice? 
Yes

8) Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No) (If yes, set a todo reminder or file a bug if appropriate)**
No, expires

9) Does the data collection use a third-party collection tool? 
No

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/70a566b541e0
Add probe for high contrast theme. r=emilio

Comment 15

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/70a566b541e0