1620358 - Record the current toolbar state in telemetry

Status: RESOLVED FIXED

(Firefox :: Toolbars and Customization, enhancement)

Comment 1

[Su-Young Hong]

design and requirement docs for this telemetry:

https://docs.google.com/document/d/14eo5m89sb82v1HAE7wxvBBc26vMaEoBtXdCHGfpdzsI/edit
https://docs.google.com/document/d/1FKT-PtaV9GD5-6GNLZmGjh13Hd5HR7ZEFfDO42CD098/edit
https://docs.google.com/document/d/1DsqdUuBdMXLvibE4SXzGpPf9zvMNMdzAe4dflg1PnFs/edit

Comment 2

[Su-Young Hong]

Attachment: Data Review Request UI Telemetry.txt

adding data steward review form:

https://docs.google.com/document/d/1TzMSoPYBn3mzH7G5ria2K4zAsQG2dzPVbKkAvdTDjUE/edit

Comment 3

[Dave Townsend [:mossop]]

Attachment: Bug 1620358: Gather telemetry on toolbar state. r=jaws!,Gijs!,aswan!,mixedpuppy!

Comment 4

[Chris H-C :chutten]

Comment on attachment 9153557 [details]
Data Review Request UI Telemetry.txt

For Q5 please provide a list of all data collections. They're all Scalars, correct? Please provide their names, descriptions, and their respective categories.

Reading the Scalars.yaml it appears as though you'll be adding an identifier "Item ID". Could you please expand on what an "item id" can contain? (are all the item ids known at compile time? Can addons supply ids? Can they change over time? Is there a chance a user might be able to write a custom one?)

Comment 5

[Dave Townsend [:mossop]]

• browser.ui.toolbar_widgets

  Keyed boolean scalar, category 2.
  Records the state of the users toolbars at startup. Keys are of the form <widget id>_pinned_<area>
  where widget id is a unique identifier for the toolbar item and area is a unique identifier for
  the area the item is on the toolbar. Add-on provided items use an id of the form addon<number>
  where the number is generated on startup currently based on the ordering of add-on items in the
  toolbar but there is no significance to this. The number remains the same for the add-on for the
  Firefox session and may be different in subsequent sessions. Other than add-on items the identifiers
  are baked into the product however the set changes as UI elements are added and removed from the
  product so it doesn't seem reasonable to define the set ahead of time and expect it to remain
  valid.

• browser.ui.customized_widgets

  Keyed integer scalar, category 2.
  Counts when a user moves a toolbar item. Keys are of the form
  <widget id>_<add/remove/move>_<old>_<new>_<reason>. Widget id are as above, old and new are
  the identifier for the toolbar area and reason is a short string identifying what caused the
  change to occur. When the operation is add (a toolbar item moved into the toolbars) the old
  value will be na. Similarly for remove. The scalar is incremented each time the action is
  performed.

• browser.ui.interaction.*
  (keyboard, menu_bar, tabs_bar, nav_bar, bookmarks_bar, app_menu, tabs_context, content_context, overflow_menu, pinned_overflow_menu, pageaction_urlbar, pageaction_panel)

  Keyed integer scalar, category 2.
  Counts when a user interacts with something in the main browser UI. The suffix of the scalar is
  the area of the UI that was interacted with. The key is an identifier for the thing that was
  interacted with. Given the large number of buttons in the UI and that that list changes over
  time it isn't reasonable to define this list beforehand. As above add-on supplied items have
  their ID masked.

• browser.ui.interaction.*
  (preferences_paneHome, preferences_paneGeneral, preferences_panePrivacy, preferences_paneSearch, preferences_paneSearchResults, preferences_paneSync, preferences_paneContainers)

  Keyed integer scalar, category 2.
  As above but records interactions in about:preferences.

Additional notes can be found in the documentation added in the patch.

Comment 6

[Dave Townsend [:mossop]]

Comment on attachment 9153557 [details]
Data Review Request UI Telemetry.txt

Details on the added scalars provided in comment 5.

Comment 7

[Chris H-C :chutten]

Comment on attachment 9153557 [details]
Data Review Request UI Telemetry.txt

Thank you for the clarification. IDs can be tricky beasts.

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file [Scalars.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Scalars.yaml) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :Mossop and :shong are responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

Yes, it includes identifiers of toolbar items. These ids are either session-local or are baked into Firefox.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+

Comment 8

[Dave Townsend [:mossop]]

(In reply to Dave Townsend [:mossop] (he/him) from comment #5)

• browser.ui.toolbar_widgets
• browser.ui.customized_widgets

One amendment with these two scalars. We would like to report add-on provided widgets using their identifiers so we know what add-ons a user has on their toolbars and when they move those items. We will still be obscuring the identifier when a user clicks on these items. We already know what add-ons a user has installed in the environment telemetry however one thought that occurs to me is that if a user only had one add-on item in their toolbars, and then they click on it even though we obscure the click in the interaction scalar we would be able to figure out the add-on since there is only one option.

Does this alter the data classification or date-review decision?

Comment 9

[Chris H-C :chutten]

No. Addon ids are Category 1 data, so the interaction data Category 2 carries. The data review is valid for this data collection. Thank you for including this update.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a94462f7697a
Gather telemetry on toolbar state. r=Gijs,zombie

Comment 11

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/a94462f7697a

Comment 12

[Dave Townsend [:mossop]]

Comment on attachment 9154365 [details]
Bug 1620358: Gather telemetry on toolbar state. r=jaws!,Gijs!,aswan!,mixedpuppy!

Beta/Release Uplift Approval Request

• User impact if declined: We delay getting valuable insights into a user's use of the UI. So no user impact immediately but it will slow us down.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The code is adding new telemetry data. It gets called from a few different places, automated tests exercise most of that and if there was any problems it would have shown up in Nighty very quickly. We are seeing telemetry data come in. Largest risk is that the telemetry is incorrect in some way but that just impacts us, not users.
• String changes made/needed: None

Comment 13

[Julien Cristau [:jcristau]]

Comment on attachment 9154365 [details]
Bug 1620358: Gather telemetry on toolbar state. r=jaws!,Gijs!,aswan!,mixedpuppy!

this is coming in really really late...

approved for 78 rc1

Comment 14

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/releases/mozilla-release/rev/02ebf5816506