1513344 - telemetry for incognito mode

Status: VERIFIED FIXED

(WebExtensions :: General, enhancement, P1)

Description

[Shane Caraveo (:mixedpuppy)]

We want to know if (and which perhaps) extensions are enabled by the user for private browsing

Comment 1

[Shane Caraveo (:mixedpuppy)]

Data we need:

• how many users in proportion allow at least one extension to run in private windows.
• how many users revoke the permissions (so allow first and then go back to not allow)
• how many extensions they allow in private windows

Can either of you fit this in?

Comment 2

[Luca Greco [:rpl] [:luca] [:lgreco]]

I'm adding Bug 1500147 as a dependency for this issue, as it is going to provide an initial set of telemetry events for the user interactions (actions and links) on about:addons and so we should be able to more easily add the telemetry data for the interactions described in comment 1 on top of the changes Bug 1500147 is going to introduce.

I'm reviewing mstriemer's patches from Bug 1500147, and so I can definitely pick this up (as I think that soon Mark may be likely busy with the work on the new about:addons page).

Comment 3

[emanuela [ux]]

Hey Luca,

can we add this metric:

• how many users interact with the doorhanger in private windows

thank you.

Comment 4

[Luca Greco [:rpl] [:luca] [:lgreco]]

(In reply to emanuela [ux] from comment #3)

Hey Luca,

can we add this metric:

• how many users interact with the doorhanger in private windows

I just added all the metrics mentioned in comment 1 and comment 3 to the user story field.

Comment 5

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: request-data-review-bug-1513344-extensions-private-browsing.md

Hi :chutten,
the attached file contains the data review request related to a couple of small additions to the existing telemetry event definitions (in particular the "addonsManager.action" telemetry event, recently added in Bug 1500147) to record some telemetry related to the user interactions with this new feature, which is going to let users "grant or prevent" to an extension the ability of interacting/running in private browsing windows.

Comment 6

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: Bug 1513344 - Record addonsManager.action telemetry event on extension private browsering access. r?mstriemer!,janerik!

Comment 7

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: Bug 1513344 - Record addonsManager.action telemetry event on interaction with private browsing doorhanger. r?mstriemer!,janerik!

Depends on D19664

Comment 8

[Chris H-C :chutten]

Comment on attachment 9043614 [details]
request-data-review-bug-1513344-extensions-private-browsing.md

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file ([Events.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Events.yaml)) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A, this collection expires in Firefox 73.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default-on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :rpl is responsible for removing or renewing the collection before it expires in Firefox 73.

---
Result: datareview+

Comment 9

[Shane Caraveo (:mixedpuppy)]

(In reply to emanuela [ux] from comment #3)

Hey Luca,

can we add this metric:

• how many users interact with the doorhanger in private windows

Which doorhanger?

There is the private window doorhanger, which only ever appears once to a user. I'm not clear how useful telemetry is on that. What are we trying to find out from this.

If it's the post-install doorhanger, that may be more useful, and we could track the new checkbox for enabling private permission.

But I thought we didn't do telemetry in private windows (I could easily be wrong here).

Comment 10

[emanuela [ux]]

(In reply to Shane Caraveo (:mixedpuppy) from comment #9)

(In reply to emanuela [ux] from comment #3)

Hey Luca,

can we add this metric:

• how many users interact with the doorhanger in private windows

Which doorhanger?

There is the private window doorhanger, which only ever appears once to a user. I'm not clear how useful telemetry is on that. What are we trying to find out from this.

But I thought we didn't do telemetry in private windows (I could easily be wrong here).

It will help us to understand if the messaging via doorhanger in this context is effective or not. Of course, if we're not able to get this metric because we are in private browsing mode -- well, we are not able, no further discussion needed :)

If it's the post-install doorhanger, that may be more useful, and we could track the new checkbox for enabling private permission.

The post-install doorhanger is new that's why we didn't have any request for telemetry, and yes! I'd like to track how many people are actually going to enable a new extension right after install.

Comment 11

[Luca Greco [:rpl] [:luca] [:lgreco]]

Hey :chutten,
sorry to bother you, but even if it was something I mentioned in the data review request, I would like to double-check explicitly with you the following doubt from Shane:

But I thought we didn't do telemetry in private windows (I could easily be wrong here).

Technically the telemetry is about the doorhanger (the related patch is attachment 9043616 [details]) and we are not tracking any details about what the user may be doing in the private window itself, but we do record telemetry about the interaction with that doorhanger (if it has been dismissed it or if we ended up opening about:addons).

Comment 12

[Chris H-C :chutten]

No worries.

Our current data collection practices around Private Browsing were clarified by Ehsan last September. In short, you can collect Private Browsing-related data so long as it doesn't affect the core premise of PBM: you can't tell what was browsed in it.

(This would be Category 3 or 4 data anyway, and subject to privacy protections and default-off behaviour in release (and additional scrutiny))

I presume this doorhanger isn't tied to web activity (because if it were, you would need to update your Data Review Request to up the Category of collection and re-request review), so it is acceptable collection within or outside of PBM.

Comment 13

[Luca Greco [:rpl] [:luca] [:lgreco]]

Thanks a lot :chutten comment 12 was extremely helpful.

I confirm that the telemetry from the attached phabricator revision isn't tied to the web activity, and so it is Category 2 as per data review request.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/aa5b7b1b30af
Record addonsManager.action telemetry event on extension private browsering access. r=mstriemer,janerik
https://hg.mozilla.org/integration/autoland/rev/1b826d8ae899
Record addonsManager.action telemetry event on interaction with private browsing doorhanger. r=mstriemer,janerik

Comment 15

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/aa5b7b1b30af
https://hg.mozilla.org/mozilla-central/rev/1b826d8ae899

Comment 16

[CosminB]

This issue is verified as fixed on Firefox 67.0a1 (20190304215035) under Win 7 64-bit and Mac OS X 10.14.1.

“Okay, Got It”
313640 addonsManager action doorhanger null {"action": "dismiss", "view": "privateBrowsing"}

“Manage Extensions”
62213 addonsManager action doorhanger null {"action": "manage", "view": "privateBrowsing"}

“Allow” from extension’s detailed page
62307 addonsManager action aboutAddons on {"action": "privateBrowsingAllowed", "view": "detail", "addonId": "{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}", "type": "extension"}

“Don’t Allow” from extension’s detailed page
193691 addonsManager action aboutAddons off {"action": "privateBrowsingAllowed", "view": "detail", "addonId": "{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}", "type": "extension"}

“Allow this extension to run in Private Windows” checkbox - has no event yet, but it will be fixed in Bug1529347.