1204995 - Telemetry probe to know what % of people have "open external links in private tabs" enabled

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Settings and Preferences, defect)

Description

[:Margaret Leibovic]

We made this feature, let's know how many people actually use it.

Comment 1

[:Margaret Leibovic]

Attachment: MozReview Request: Bug 1204995 - Update FENNEC_OPEN_URLS_IN_PRIVATE expires_in_version value to actually collect data. r=mfinkle

Bug 1204995 - Add telemetry histogram probe to know what % of people have "open external links in private tabs" enabled. r?mcomella

Comment 2

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Comment on attachment 8661775 [details]
MozReview Request: Bug 1204995 - Update FENNEC_OPEN_URLS_IN_PRIVATE expires_in_version value to actually collect data. r=mfinkle

https://reviewboard.mozilla.org/r/19445/#review17479

::: toolkit/components/telemetry/Histograms.json:8828
(Diff revision 1)
> +    "description": "Reports the state of the open external links in private tabs preference"

You should add an `alert_email` here. See bug 1198955 where I remove such a probe.

Comment 3

[Mark Finkle (:mfinkle) (use needinfo?)]

https://reviewboard.mozilla.org/r/19445/#review17485

::: mobile/android/base/BrowserApp.java:1868
(Diff revision 1)
> +            Telemetry.addToHistogram("FENNEC_OPEN_URLS_IN_PRIVATE", (openPrivate ? 1 : 0));

This probably works well enough, and we use "boolean" for other similar probes. I wonder if this overcounts though. We wanted to add "flag" support too. Food for thought.

::: toolkit/components/telemetry/Histograms.json:8825
(Diff revision 1)
> +  "FENNEC_OPEN_URLS_IN_PRIVATE": {

Technically, this needs a p? from a privacy/telemetry person. Ally or Vladan would be fine and should be very responsive. Ping on IRC if needed.

Comment 4

[Mark Finkle (:mfinkle) (use needinfo?)]

Also, does this probe pass/fail the test (do we have a test?) for tracking Private Browsing behavior? Technically, we are not tracking when the user opens a PB tab with this probe, just whether the preference is set.

Comment 5

[:Margaret Leibovic]

Comment on attachment 8661775 [details]
MozReview Request: Bug 1204995 - Update FENNEC_OPEN_URLS_IN_PRIVATE expires_in_version value to actually collect data. r=mfinkle

(In reply to Mark Finkle (:mfinkle) from comment #4)
> Also, does this probe pass/fail the test (do we have a test?) for tracking
> Private Browsing behavior? Technically, we are not tracking when the user
> opens a PB tab with this probe, just whether the preference is set.

I assumed it would be okay because it is just measuring whether or not a preference is set, but let's check with bsmedberg to be thorough. I also still need to request a telemetry review from a data steward on this.

This feature is only on Nightly, and I'm going to have the probe set to expire in a release, so that we collect the minimum amount of data necessary to understand whether or not people are using this feature. I am prepared to argue that this probe is important for helping us understand how people are using the new privacy features we're trying to develop.

Comment 6

[Benjamin Smedberg]

Comment on attachment 8661775 [details]
MozReview Request: Bug 1204995 - Update FENNEC_OPEN_URLS_IN_PRIVATE expires_in_version value to actually collect data. r=mfinkle

This is fine, because it's a setting that doesn't indicate that you've used PB, and it's already recorded on the user's disk.

Comment 7

[:Margaret Leibovic]

(In reply to Mark Finkle (:mfinkle) from comment #3)
> https://reviewboard.mozilla.org/r/19445/#review17485
> 
> ::: mobile/android/base/BrowserApp.java:1868
> (Diff revision 1)
> > +            Telemetry.addToHistogram("FENNEC_OPEN_URLS_IN_PRIVATE", (openPrivate ? 1 : 0));
> 
> This probably works well enough, and we use "boolean" for other similar
> probes. I wonder if this overcounts though. We wanted to add "flag" support
> too. Food for thought.

I thought we had problems with trying to use "flag" in the past... but it looks like this might answer my question:
https://bugzilla.mozilla.org/show_bug.cgi?id=1170137#c2

Comment 8

[:Margaret Leibovic]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9a184e2c9790

Comment 9

[:Margaret Leibovic]

Updated try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=aaa79329a816

I updated this to use a flag, but want to make sure there isn't anything else causing bug 1170137. I also set the probe to expire in 44, since I probably won't get around to landing this until next week.

Comment 10

[Mark Finkle (:mfinkle) (use needinfo?)]

NI for landing

Comment 11

[:Margaret Leibovic]

(In reply to Mark Finkle (:mfinkle) from comment #10)
> NI for landing

My try push is pretty orange... it seems unrelated, but I'm going to make another one to be sure, since retriggers don't seem to be resolving it.

Comment 12

[:Margaret Leibovic]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=13778cb960d6

Comment 13

[:Margaret Leibovic]

(In reply to :Margaret Leibovic from comment #11)
> (In reply to Mark Finkle (:mfinkle) from comment #10)
> > NI for landing
> 
> My try push is pretty orange... it seems unrelated, but I'm going to make
> another one to be sure, since retriggers don't seem to be resolving it.

Actually, I was confused by the way treeherder now collpases passing tests... there are passing test runs, and it seems like the failures really are just awful intermittents related to service workers.

Comment 14

[:Margaret Leibovic]

https://hg.mozilla.org/integration/fx-team/rev/df9dfa4138ed4c7864fe6e55f1ab41f10eb30c35
Bug 1204995 - Add telemetry histogram probe to know what % of people have "open external links in private tabs" enabled. r=mcomella r=bsmedberg

Comment 15

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/df9dfa4138ed

Comment 16

[:Margaret Leibovic]

Comment on attachment 8661775 [details]
MozReview Request: Bug 1204995 - Update FENNEC_OPEN_URLS_IN_PRIVATE expires_in_version value to actually collect data. r=mfinkle

Bug 1204995 - Update FENNEC_OPEN_URLS_IN_PRIVATE expires_in_version value to actually collect data. r=mfinkle

Comment 17

[Mark Finkle (:mfinkle) (use needinfo?)]

Comment on attachment 8661775 [details]
MozReview Request: Bug 1204995 - Update FENNEC_OPEN_URLS_IN_PRIVATE expires_in_version value to actually collect data. r=mfinkle

https://reviewboard.mozilla.org/r/19445/#review19121

Comment 18

[:Margaret Leibovic]

https://hg.mozilla.org/integration/fx-team/rev/106a6e4b877e66969129018e034a1c1067ee8aa1
Bug 1204995 - Update FENNEC_OPEN_URLS_IN_PRIVATE expires_in_version value to actually collect data. r=mfinkle

Comment 19

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/106a6e4b877e