Closed Bug 945347 Opened 11 years ago Closed 2 years ago

BrowserUITelemetry for measuring Australis impact on user behaviour

Categories

(Privacy Graveyard :: Product Review, task, P1)

x86
All

Tracking

(Not tracked)

RESOLVED INCOMPLETE

People

(Reporter: mconley, Assigned: smartin, NeedInfo)

References

Details

(Whiteboard: [Australis:P-])

Bug 935093 is a meta bug for a whole series of bugs that add telemetry probes to Firefox for the express purpose of measuring the impact of Australis on user behaviour.

We're currently landing probes on Holly (the Australis-backout branch that will land on mozilla-aurora in ~1 week) for our control data. We will land similar probes on mozilla-central after that, and compare / contrast the data.

Note that the UX team is fully aware of the inherent selection bias that comes with Telemetry, along with the fact that Telemetry is per-session as opposed to per-user. This data will be calibrated with data that we get from FHR in order to weight its findings as accurately as possible, and will be one of many inputs that the UX team uses to measure the impact of Australis.

I'm requesting privacy review because after a conversation with StrangeCharm in IRC, I think we need the Privacy Team's OK to go ahead with this data collection.

The list of things we're attempting to collect can be found at the bottom of this Etherpad - from lines 100 onwards: https://firefox-ux.etherpad.mozilla.org/measuringaustralis

Here's how we're collecting it:

A module named UITelemetry.jsm landed in mozilla-central, Holly and mozilla-aurora last week. This module is used to collect both simple measurements and events in the browser session.

Simple measurements could be things like a list of default toolbar items that have been put into the palette.

The events are being used to count clicks on things like the Firefox menu button. Example: https://bug942280.bugzilla.mozilla.org/attachment.cgi?id=8337146

Note that bug 944116 was filed to remove the absolute timestamps from the events.

I just wanted to make sure this project wasn't treading Privacy-NOPE territory, and if it is, what to do about it. I'm happy to answer any and all questions about the project. I've Cc'd Blake Winton, the UX team representative for the project.
IMO, most of the stuff in the etherpad looks really standard and non-personal.  It's mainly "count the number of times widget x was used": feature usage.  Doesn't seem like anything is reporting what sites or types of sites users visit, or how they interact with the web (it's limited to how they interact with Firefox).  But it's a lot of stuff.  Lots of stuff.

I'm curious about a few things:

    To answer "What are people adding, moving and removing from their UI?"
    [Bug 943606] Dump the list of default items that are not in their default toolbars
    [Bug 943606] Dump the list of default items that are in the palette
    [Bug 943606] Dump the list of non-default items that are not in the palette

What are default vs. non-default items?  Are these things made up by users?
(In reply to Sid Stamm [:geekboy or :sstamm] from comment #1)
> 
> What are default vs. non-default items?  Are these things made up by users?

In this context, by "default", I mean items that stock Firefox puts into the toolbars. "Non-default" items are items that stock Firefox puts into the palette. It does not include things like items provided by add-ons - those are ignored.

In practice, I have a default and non-default whitelist of item IDs that I check each item against.
Whiteboard: [Australis:P-]
Hey Alex,

Sid Stamm referred me to you on this. Basically, I want to ensure that the UITelemetry we're gathering for this project jives with our privacy policy.

comment 0 explains at a high level what we're collecting and how we're collecting it. I should also add that all of this UITelemetry is shoved into the "simpleMeasures" object of the TelemetryPing, meaning that it's not split apart and bucketed like other things in Telemetry - all of the information kind of sticks together.

Here's an example of the UITelemetry object inside of the TelemetryPing: http://pastebin.mozilla.org/3797692

I think Sid was a little worried that the toolbar content data was a bit fingerprinty, but wasn't 100% sure. I'll note that those Arrays are not verbatim snapshots of the contents of a users toolbars, but just lists of items that:

1) Are in their default location
2) Have been moved from their default location
3) Have been removed from their default location

Only whitelisted items (items that are built-in) are counted - the rest are skipped.

Is this, or anything else in this blob of concern to you wrt our Privacy Policy?
Flags: needinfo?(afowler)
Flags: needinfo?(ahua)
Priority: -- → P1
Flags: needinfo?(jjensen)
Hi Stacy, apologies if I am being dense, but what did you need me to comment on?
Flags: needinfo?(jjensen)

This bug lies at rest in the graveyard.

Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.