1491047 - Add telemetry probe to measure number of private browsing sessions

Status: VERIFIED FIXED

(Firefox :: Private Browsing, enhancement)

Description

[Luke Crouch [:groovecoder]]

We would like to measure the high-level aggregate figure of how many private browsing sessions are created.

Comment 1

[Josh Matthews [:jdm]]

What does a session mean? Every time there's a private window opened when there was not one before?

Comment 2

[Luke Crouch [:groovecoder]]

As I understand it yes, but merwin should verify.

Comment 3

[Josh Matthews [:jdm]]

If that's true, then https://searchfox.org/mozilla-central/source/docshell/base/nsDocShell.cpp#280-291 is probably a sensible place to accumulate the telemetry values of gNumberOfPrivateDocShells changes from 0 to 1.

Comment 4

[(no longer active)]

What Marshall told me was that we would like to know whether any private sessions existed in a given browser session.

Comment 5

[(no longer active)]

Attachment: Data Review Request

Comment 6

[(no longer active)]

Attachment: Bug 1491047 - Add a telemetry probe for measuring how many browsing sessions have had a private window opened in them; data-review=francois

Comment 7

[Merwin]

Ehsan and I have been working together on this.  This is Category 2 data that falls within our policies, although it previously fell outside of a specific policy that governed private browsing instrumentation. Ehsan announced changes to that policy here, so it should be good to go.  https://groups.google.com/forum/#!topic/mozilla.governance/7UH_MngkDx0

Regarding the purpose of this data, we'd like to have a high level account of the size and percentage of our user base that uses private browsing. I've asked Saptarshi to take a look at this bug from a data scientist perspective to ensure it answers the questions we are after.

Comment 8

[Josh Matthews [:jdm]]

Comment on attachment 9009693 [details]
Bug 1491047 - Add a telemetry probe for measuring how many browsing sessions have had a private window opened in them;  data-review=francois

Josh Matthews [:jdm] has approved the revision.

Comment 9

[Chenxia Liu [:liuche]]

Comment on attachment 9009692 [details]
Data Review Request

Chutten can you review? I'm just headed on PTO.
Though now that I see it's about private browsing, I think it's a can of worms because we have extra strict guarantees in what can be collected about PB - including evidence that someone has ever used PB. I think :Ehsan is the module owner for PB.
https://wiki.mozilla.org/Private_Browsing

Comment 10

[Chris H-C :chutten]

Comment on attachment 9009692 [details]
Data Review Request

I provided technical review support for the patch, so I'll redirect to :francois

Ehsan is indeed in charge of PB, and he wrote a nice explanation email about the rationale for our change in posture to no longer attempt to disguise from a local adversary whether PB was used to the governance list: https://groups.google.com/forum/#!topic/mozilla.governance/7UH_MngkDx0

...which reminds me, I should probably bring this up in the next Data Stewards meeting.

Comment 11

[(no longer active)]

Yes, this patch indeed has been submitted in light of the recent change we made to our policy.  As you observed, Chenxia, under our previous policy, this patch wouldn't be acceptable.

Comment 12

[François Marier [:francois]]

Comment on attachment 9009692 [details]
Data Review Request

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in Scalars.yaml.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Yes, Marshall Erwin.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 2.

5) Is the data collection request for default-on or default-off?

Default ON, all channels.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes, based on comment 7.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, permanent.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7a7b481a00f4
Add a telemetry probe for measuring how many browsing sessions have had a private window opened in them;  data-review=francois r=jdm

Comment 14

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/7a7b481a00f4

Comment 15

[(no longer active)]

Comment on attachment 9009693 [details]
Bug 1491047 - Add a telemetry probe for measuring how many browsing sessions have had a private window opened in them;  data-review=francois

Approval Request Comment
[Feature/Bug causing the regression]: Not a regression.  This is a new telemetry probe, we'd like to start getting data as early as possible.
[User impact if declined]: We would get data later than possible.  :-)
[Is this code covered by automated tests?]: No.
[Has the fix been verified in Nightly?]: Can easily be verified in about:telemetry.
[Needs manual test from QE? If yes, steps to reproduce]:
1. In a new session, go to about:telemetry, select the Sacalars tab.
2. Verify that there is no entry called dom.parentprocess.private_window_used.
3. Open a new private window, and close it (or leave it open, doesn't matter).
4. Go back to about:telemetry and refresh.
5. Verify that now there is an entry called dom.parentprocess.private_window_used and that its value is true.
[List of other uplifts needed for the feature/fix]:
[Is the change risky?]: No.
[Why is the change risky/not risky?]: The code change is very small, it is effectively checking a boolean and setting a boolean based on that and setting a scalar telemetry ID.
[String changes made/needed]: None.

Comment 16

[Pascal Chevrel:pascalc]

Comment on attachment 9009693 [details]
Bug 1491047 - Add a telemetry probe for measuring how many browsing sessions have had a private window opened in them;  data-review=francois

Low risk, uplift approved for 63 beta 10, thanks.

Comment 17

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/releases/mozilla-beta/rev/45c93938ae14

Comment 18

["Saptarshi Guha[:joy]"]

Just to confirm, for a given profile and a subs-session, this variable is True if a private browsing mode window was opened at least once and False otherwise?

Comment 19

[(no longer active)]

(In reply to "Saptarshi Guha[:joy]" from comment #18)
> Just to confirm, for a given profile and a subs-session, this variable is
> True if a private browsing mode window was opened at least once and False
> otherwise?

It will be present and true if a private window was opened at least once in the session, and absent otherwise.  There should never be a ping with the scalar submitted as false.  (IOW the scalar can really take only one value.)

Comment 20

[Oana Botisan, Desktop QA]

I managed to reproduce the issue using an older version of Nightly (2018-09-13) and the steps from comment 15 on Windows 10 x64.
I retested everything using the latest Nightly 64.0a1 on Windows 10 x64, macOS 10.13 and Ubuntu 16.04 x64 and the bug is not reproducing anymore.
However, on beta 63.0b9 it's still reproducing. I think I have to wait until the beta 63.0b10 is up to verify the fix.

Comment 21

[Oana Botisan, Desktop QA]

I verified the fix on beta 63.0b10 using Windows 10 x64, Ubuntu 16.04 x64 and macOS 10.13. The bug is not reproducing anymore.