1175977 - Telemetry probes for tracking protection

Status: RESOLVED WORKSFORME

(Firefox for Android Graveyard :: Settings and Preferences, defect)

Description

[:Margaret Leibovic]

A few ideas for probes:

* What % of users have disabled tracking protection?
* How often do people hit "Learn more" links?
* How often do people actually hit the shield icon?

We should look into updating the existing tracking protection histogram probes we have:
http://telemetry.mozilla.org/#filter=nightly%2F40%2FTRACKING_PROTECTION_ENABLED%2Fsaved_session%2FFirefox&aggregates=Submissions&evoOver=Builds&locked=true&sanitize=true&renderhistogram=Graph

Comment 1

[Barbara Bermes [:barbara] - NI please!]

More ideas for future

- How often do people run "take the tour"?
- How often do people abandon the tour and at what step?
- How often do people repeat "take the tour" (if that's even a planned feature at the moment)?

More broadly to prove the hypothesis of TP in general:
- Do we see an increased usage of the privacy settings in general after introducing this (i.e. apologies, I still don't have my Android to compare what it is currently available or not)?
- Question: can we time / get a sense of the average time between a user switching on and off the feature, that might give us further insights of users understand the feature, assuming they keep switching back and forth on/off.

- Comparison between mobile and desktop usage and adoption of this feature? Do we see some patterns (I will need to check with Javaun, I guess, how and what they track for Desktop)? I assume that can be done via Telemetry as well.

Comment 2

[:Margaret Leibovic]

See also: bug 1179441.

We should coordinate with the desktop team to make sure the shared probes are updated consistently.

Comment 3

[Martyn Haigh (:mhaigh)]

margaret: Who in the desktop should we be coordinating with?

Comment 4

[:Margaret Leibovic]

(In reply to Martyn Haigh (:mhaigh) from comment #3)
> margaret: Who in the desktop should we be coordinating with?

bgrins filed bug 1179441, so I would coordinate with him. ttaubert should also know what's going on.

Comment 5

[Martyn Haigh (:mhaigh)]

Hey - was wondering if you were able to help with creating a list of probes we should implement for TP on Android?

Cheers

Comment 6

[Brian Grinstead [:bgrins]]

(In reply to Martyn Haigh (:mhaigh) from comment #5)
> Hey - was wondering if you were able to help with creating a list of probes
> we should implement for TP on Android?

Hi Martyn, we are awaiting a response from Javaun about what data we want to collect in https://bugzilla.mozilla.org/show_bug.cgi?id=1179441#c3.  Once we figure out exactly to collect, let's coordinate to make sure we have the same probes in mobile/desktop.

Comment 7

[:Margaret Leibovic]

I know we're trying to stay coordinated with desktop here, and in bug 1112189, I did implement mobile support for the existing desktop histograms. But thinking about this more, I don't want us to block on desktop to start collecting some useful data.

I think it would be worthwhile to stop using these existing histograms, and just create our own fennec-specific histograms. Yes, let's coordinate with desktop to make sure we're collecting the same useful data, but we don't need to coordinate on the implementation.

Martyn or Chenxia, can one of you pick this up? I think we should try to get Nightly telemetry data sooner rather than later.

Comment 8

[Brian Grinstead [:bgrins]]

(In reply to :Margaret Leibovic from comment #7)
> I know we're trying to stay coordinated with desktop here, and in bug
> 1112189, I did implement mobile support for the existing desktop histograms.
> But thinking about this more, I don't want us to block on desktop to start
> collecting some useful data.
> 
> I think it would be worthwhile to stop using these existing histograms, and
> just create our own fennec-specific histograms. Yes, let's coordinate with
> desktop to make sure we're collecting the same useful data, but we don't
> need to coordinate on the implementation.
> 
> Martyn or Chenxia, can one of you pick this up? I think we should try to get
> Nightly telemetry data sooner rather than later.

From what I understand, we may not be wanting to collect any telemetry data while the user is in private browsing mode (at least on desktop), so we may actually end up removing some of the probes.  I'm not sure if that decision has been finalized though - ni? Javaun just to make sure everyone is on the same page.

Comment 9

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Brian Grinstead [:bgrins] from comment #8)

> From what I understand, we may not be wanting to collect any telemetry data
> while the user is in private browsing mode (at least on desktop), so we may
> actually end up removing some of the probes.  I'm not sure if that decision
> has been finalized though - ni? Javaun just to make sure everyone is on the
> same page.

What?! Why would we not be collecting anonymous, opt-in data?

Comment 10

[Javaun Moradi [:javaun]]

We're still having conversations about what data we actually need, and whether a non-representative sample from PBM could even help us answer that beyond what we could see from a non-representative sample in normal mode.

And then there's the question on sanctity of PBM, since we would technically be storing data on the HD about the user's experience there (that they'd used PBM) opt-in or no.

Comment 11

[Javaun Moradi [:javaun]]

Margaret's first two are good additions. Since these are particular to UI Preferences (and not PBM in particular) I think we can instrument them non-controversially.

* What % of users have disabled tracking protection?
* How often do people hit "Learn more" links?

Comment 12

[:Margaret Leibovic]

Until we have tracking protection enabled in non-private tabs, there's not much we can do here.

I'm closing this bug out. We can revisit adding more probes as we build more functionality.