1698100 - Record usage of compact mode in telemetry

Status: RESOLVED FIXED

(Firefox :: Toolbars and Customization, task)

Description

[Dave Townsend [:mossop]]

To get a better understanding of the usage of compact mode in the wild.

Comment 1

[Dave Townsend [:mossop]]

Attachment: Data collection request.

Comment 2

[Dave Townsend [:mossop]]

Attachment: Bug 1698100: Record the state of the toolbar density in telemetry. r=chutten

Comment 3

[Chris H-C :chutten]

Comment on attachment 9208726 [details]
Data collection request.

PRELIMINARY NOTES:

ni? to remember to please file a follow-up bug for removing the instrumentation during Firefox 90. userPrefs don't have built-in expiry notifications or mechanisms, so we don't want to forget.

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

No. This collection will expire in Firefox 90.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :Mossop is responsible for renewing or removing the collection before it expires in Firefox 90.

---

Result: datareview+

Comment 4

[Dave Townsend [:mossop]]

Filed bug 1698142 to track the removal.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a162279140c6
Record the state of the toolbar density in telemetry. r=chutten

Comment 6

[Dave Townsend [:mossop]]

Comment on attachment 9208728 [details]
Bug 1698100: Record the state of the toolbar density in telemetry. r=chutten

Beta/Release Uplift Approval Request

• User impact if declined: We are using this data to help make a product call, the more users we can gather this data from the better informed we will be so while there is no direct impact it may affect the availability of compact mode to users in the future.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This just adds two preferences to the already well tested preference collection in the telemetry environment.
• String changes made/needed: None

Comment 7

[Will]

Great to see this but I'm still concerned because there might be a flawed decision in progress: this is testing usage of compact mode for the current UI, correct? But Proton's UI is significantly taller than the current UI. The current "Normal" density is dense enough that I could see a lot of people not being bothered by it enough to change to Compact. But in my opinion, Proton's "Normal" density passes this threshold and a lot of users will want something slimmer/more dense once it rolls out. Basically, if you're collecting data on the current UI, it's not relevant data. We need data on Proton UI (Proton compact vs Proton normal). So if UX/Product Management is hell bent on getting rid of compact density, at least do this properly and wait until it rolls out to stable. If after lets say, 2 months, only 4% of users have opted to use Proton's compact density, you would indeed have grounds for removing it.

Comment 8

[Jonathan Kew [:jfkthame]]

Given that I think we can safely assume that a majority of users just use whatever we ship, unaware there might be other choices, there's a powerful inherent bias here. Do we have telemetry that tells us what proportion of users ever open the Customize UI?

Perhaps the right way forward would be to default-enable Compact mode in Proton for a significant test population. Then have telemetry that will tell us whether there's any difference in retention between users with Normal vs Compact mode, and what proportion of users explicitly choose to change the setting (in either direction).

Comment 9

[Chris H-C :chutten]

(In reply to Jonathan Kew (:jfkthame) from comment #8)

Do we have telemetry that tells us what proportion of users ever open the Customize UI?

I rely on the Probe Dictionary for questions like that. It didn't come up with anything using keywords like "custom" and "ui" that I could recognize, but I don't know all the ways we refer to customizable ui, so don't treat my scan as definitive.

Comment 10

[Dave Townsend [:mossop]]

Discussion about whether to remove compact mode or not belongs somewhere else. This bug is just about gathering some additional data to help with that decision. If there is other data needed then please file a new bug.

Comment 11

[Will]

(In reply to Dave Townsend [:mossop] from comment #10)

Discussion about whether to remove compact mode or not belongs somewhere else. This bug is just about gathering some additional data to help with that decision. If there is other data needed then please file a new bug.

Right but does this telemetry probe include Proton UI? And is there a distinction because current UI and Proton UI? Current UI and Proton UI are significantly different in height

Comment 12

[Will]

*between current UI and Proton UI

(sorry for the typo)

Comment 13

[Jonathan Kew [:jfkthame]]

(In reply to Dave Townsend [:mossop] from comment #10)

Discussion about whether to remove compact mode or not belongs somewhere else. This bug is just about gathering some additional data to help with that decision. If there is other data needed then please file a new bug.

(The point is that the data being gathered here may not be the appropriate data to properly answer that question.)

OK, I filed bug 1698171.

Comment 14

[Dave Townsend [:mossop]]

(In reply to Will from comment #11)

(In reply to Dave Townsend [:mossop] from comment #10)

Discussion about whether to remove compact mode or not belongs somewhere else. This bug is just about gathering some additional data to help with that decision. If there is other data needed then please file a new bug.

Right but does this telemetry probe include Proton UI? And is there a distinction because current UI and Proton UI? Current UI and Proton UI are significantly different in height

This probe just registers the current setting of the UI density.

Comment 15

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/a162279140c6

Comment 16

[Julien Cristau [:jcristau] (back August 18)]

Comment on attachment 9208728 [details]
Bug 1698100: Record the state of the toolbar density in telemetry. r=chutten

approved for 87 rc1

Comment 17

[Julien Cristau [:jcristau] (back August 18)]

https://hg.mozilla.org/releases/mozilla-release/rev/830bd9ef72a8

Comment 18

[kuro68k]

A telemetry probe might be missing a lot of usage of compact mode. The kinds of people who fiddle with the settings to get them just the way they like them are probably also more inclined to disable telemetry.

Comment 19

[Sören Hentzschel]

people who fiddle with the settings to get them just the way they like them are probably also more inclined to disable telemetry.

Well, people who disable telemetry don't want to be considered. That's the reason why they disabled telemetry.

Comment 20

[kuro68k]

(In reply to Sören Hentzschel from comment #19)

people who fiddle with the settings to get them just the way they like them are probably also more inclined to disable telemetry.

Well, people who disable telemetry don't want to be considered. That's the reason why they disabled telemetry.

Maybe the reason is that they don't like telemetry or don't have the time/energy to work out what information is included and if they are happy with it. I have seen some of them complaining that their views are ignored because every decision is driven by telemetry.