Closed Bug 1179441 Opened 9 years ago Closed 8 years ago

Change telemetry measurements for Tracking Protection to account for PB Mode

Categories

(Firefox :: General, defect, P4)

defect

Tracking

()

RESOLVED DUPLICATE of bug 1231376
Tracking Status
firefox42 --- affected

People

(Reporter: bgrins, Unassigned)

References

Details

(Whiteboard: [fxprivacy] [blocked on decision])

As mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=1175327#c12, we need to make sure that the TP section in the control center shows up when privacy.trackingprotection.pbmode.enabled is set to true (doing that in Bug 1178985).

When "privacy.trackingprotection.pbmode.enabled" is set to true, it will change the meaning of the existing telemetry measurements.  So we should make sure they are measuring what we want them to.

TRACKING_PROTECTION_SHIELD is currently measured when TP is enabled but the shield isn't shown.  In this case when pbmode is on, we probably still want to measure the same thing so may not need any changes to it.

TRACKING_PROTECTION_ENABLED is measuring if the feature is turned on (via the "privacy.trackingprotection.enabled" pref).  Maybe we want to have a separate measurement or a different type of histogram where we could have multiple values:

1) Off
2) On b/c of "privacy.trackingprotection.enabled"
3) On b/c of "privacy.trackingprotection.pbmode.enabled"

These measurements are also used in android so we should make sure that the measurements mean the same thing across platforms:
https://dxr.mozilla.org/mozilla-central/search?q=TRACKING_PROTECTION_SHIELD&redirect=true
https://dxr.mozilla.org/mozilla-central/search?q=TRACKING_PROTECTION_ENABLED&redirect=true
Flags: firefox-backlog?
See Also: → 1175977
Hi Brian, do you have a sense of the point estimate?  If not, no worries, we can estimate it later on.  Also, should this bug be marked as + or - for verification?
Rank: 1
Flags: qe-verify?
Flags: needinfo?(bgrinstead)
Flags: firefox-backlog?
Flags: firefox-backlog+
Priority: -- → P1
(In reply to Marco Mucci [:MarcoM] from comment #1)
> Hi Brian, do you have a sense of the point estimate?  If not, no worries, we
> can estimate it later on.  Also, should this bug be marked as + or - for
> verification?

I think we will need to define exactly what data should be collected about TP in PBM before it's ready to estimate.  Not sure who the best person to consult with about that is, maybe Javaun?

If we went with the original suggestion of leaving TRACKING_PROTECTION_SHIELD as-is and adding a new measurement for TRACKING_PROTECTION_ENABLED to take into account three different states it seems like it'd be pretty easy, but I haven't worked with telemetry enough to give it a more detailed estimate.  It won't need verification.
Flags: qe-verify?
Flags: qe-verify-
Flags: needinfo?(bgrinstead)
(In reply to Brian Grinstead [:bgrins] from comment #2)
> (In reply to Marco Mucci [:MarcoM] from comment #1)
> > Hi Brian, do you have a sense of the point estimate?  If not, no worries, we
> > can estimate it later on.  Also, should this bug be marked as + or - for
> > verification?
> 
> I think we will need to define exactly what data should be collected about
> TP in PBM before it's ready to estimate.  Not sure who the best person to
> consult with about that is, maybe Javaun?
> 
> If we went with the original suggestion of leaving
> TRACKING_PROTECTION_SHIELD as-is and adding a new measurement for
> TRACKING_PROTECTION_ENABLED to take into account three different states it
> seems like it'd be pretty easy, but I haven't worked with telemetry enough
> to give it a more detailed estimate.  It won't need verification.

Thanks Brian, the estimation can wait until we've defined what data to collect.
Flags: needinfo?(jmoradi)
Some more background on what exactly is being collected right now:

TRACKING_PROTECTION_ENABLED is stored once per-window based on whether the feature is 'enabled' or not.  There are two different cases where the feature is enabled: privacy.trackingprotection.enabled is true or privacy.trackingprotection.pbmode.enabled is true AND we are in a private window.  See 'enabledHistogram' in https://dxr.mozilla.org/mozilla-central/source/browser/base/content/browser-trackingprotection.js#33

TRACKING_PROTECTION_SHIELD is logged if the feature is enabled but the shield did NOT show up.  So the naming seems to be backwards here.  See "TRACKING_PROTECTION_SHIELD" in https://dxr.mozilla.org/mozilla-central/source/browser/base/content/browser.js#6796.

TRACKING_PROTECTION_EVENTS is logged on every page when the feature is enabled.  It logs 0 to indicate any state of blocking (tracking detected and blocked, tracking detected and not blocked, tracking not detected).  It logs 1 one time when TP is been disabled for the current page.  It logs 2 one time when TP is enabled for the current page.  See 'eventsHistogram' here: https://dxr.mozilla.org/mozilla-central/source/browser/base/content/browser-trackingprotection.js#36.
A few possible changes given the current state (Comment 4):

For TRACKING_PROTECTION_ENABLED we probably want to separately know whether it's on because of pbmode.   We may want to measure the following states instead of a simple on/off:

0: Off
1: On b/c of "privacy.trackingprotection.enabled"
2: On b/c of "privacy.trackingprotection.pbmode.enabled"

The TRACKING_PROTECTION_SHIELD name is the opposite of what it's collecting.  We should rename it or change the measurement to log when the shield is shown.

TRACKING_PROTECTION_EVENTS is gathering two different data sets: the state of blocking and user enabling/disabling on a site.  It would be interesting to see the first data set broken into a separate probe that stores:

0: tracking detected and blocked
1: tracking detected and not blocked
2: tracking not detected
I'm setting up a meeting w/ metrics/UAG to do a full audit of our needs and of telemetry probe coverage. Will rope everyone in shortly.
Flags: needinfo?(jmoradi)
Whiteboard: [fxprivacy] → [fxprivacy] [blocked on decision]
Whiteboard: [fxprivacy] [blocked on decision] → [fxprivacy] [campaign] [blocked on decision]
Javaun confirmed this bug be de-prioritized to 'P2' as a decision is pending.
Priority: P1 → P2
Whiteboard: [fxprivacy] [campaign] [blocked on decision] → [fxprivacy] [blocked on decision]
Depends on: 1189928
Blocks: 1188565
Rank: 1
Priority: P2 → P3
Priority: P3 → P4
Blocks: 1216897
No longer blocks: 1188565
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.