Closed Bug 1176391 Opened 9 years ago Closed 9 years ago

UI Telemetry should NOT report on private-browsing button usage

Categories

(Toolkit :: Telemetry, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla41
Tracking Status
firefox39 --- fixed
firefox40 --- fixed
firefox41 --- fixed

People

(Reporter: vladan, Assigned: mconley)

Details

Attachments

(1 file)

UI Telemetry is collecting information about the usage of the FIrefox UI and that includes usage of the private-browsing button and "open-in-private-window" context menu item:

http://hg.mozilla.org/mozilla-central/annotate/2694ff2ace6a/browser/modules/BrowserUITelemetry.jsm#l654

http://hg.mozilla.org/mozilla-central/diff/d549a25bc029/browser/modules/BrowserUITelemetry.jsm

We need to stop collecting this data ASAP. Private browsing habits need to be 100% private.

This will need to be uplifted to as many versions as possible.
Okay, I don't really have any objections to this, though I do want to point out:

1) We're counting the number of times a user submitting Telemetry has clicked on either of those two items. We're certainly not collecting information on what they're viewing while in PB mode.

2) We are counting clicks on the built-in UI for private browsing windows as well, but it's getting mixed in with the counts for non-private browsing windows too. I suspect this is not a concern?
Flags: needinfo?(vdjeric)
> 1) We're counting the number of times a user submitting Telemetry has
> clicked on either of those two items. We're certainly not collecting
> information on what they're viewing while in PB mode.

That's right. But even reporting on the fact that a user used private browsing mode is uncomfortable and needs to be removed. 

> 2) We are counting clicks on the built-in UI for private browsing windows as
> well, but it's getting mixed in with the counts for non-private browsing
> windows too. I suspect this is not a concern?

That's fine.
Flags: needinfo?(vdjeric)
Another small nuance: i think it's ok to report that the toolbar was customized to have the private-browsing button in it
In a related question, assuming we're interested in how many people are using private browsing mode (or how often it's being used), how do you suggest we get this data?  Could we collect it in a more anonymous/aggregated way somehow?
(In reply to Blake Winton (:bwinton) from comment #5)
> In a related question, assuming we're interested in how many people are
> using private browsing mode (or how often it's being used), how do you
> suggest we get this data?  Could we collect it in a more
> anonymous/aggregated way somehow?

It's a question for another bug & would need wider discussion. We might come to the conclusion that Telemetry is not be the right vehicle for collecting this information, or that it needs to be an experiment with a small # of anonymous users, or that compensated user studies are preferable to Telemetry etc
This should be pretty straight-forward.
Assignee: nobody → mconley
(Famous last words)
Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?bwinton
Attachment #8624934 - Flags: review?(bwinton)
Comment on attachment 8624934 [details]
MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs

Redirecting to (rolls his reviewer-dice…) Gijs!  ;)

(Sorry if this doesn't update in ReviewBoard.  I couldn't figure out how to change it there.)
Attachment #8624934 - Flags: review?(bwinton) → review?(gijskruitbosch+bugs)
Attachment #8624934 - Flags: review?(gijskruitbosch+bugs) → review+
Comment on attachment 8624934 [details]
MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs

https://reviewboard.mozilla.org/r/11767/#review10239

Ship It!

::: browser/modules/BrowserUITelemetry.jsm:145
(Diff revision 1)
> +  if (pbIndex != -1) {
> +    itemsForCounting.splice(pbIndex, 1);
> +  }

because the thing here is actually called ALL_BUILTIN_ITEMS, I would prefer if we handled this in the mouseup handler where we record things, in case we ever end up using it for something else...
Comment on attachment 8624934 [details]
MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs

Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r=Gijs.
Attachment #8624934 - Attachment description: MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?bwinton → MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r=Gijs.
Attachment #8624934 - Flags: review+
Comment on attachment 8624934 [details]
MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs

Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs
Attachment #8624934 - Attachment description: MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r=Gijs. → MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs
Attachment #8624934 - Flags: review?(gijskruitbosch+bugs)
https://reviewboard.mozilla.org/r/11765/#review10241

Just a small alteration - I figure we don't want to record _anything_ if the user chose the private browsing window option from the context menu - not even "otheritem".
Comment on attachment 8624934 [details]
MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs

https://reviewboard.mozilla.org/r/11767/#review10245

Ship It!
Attachment #8624934 - Flags: review?(gijskruitbosch+bugs) → review+
Comment on attachment 8624934 [details]
MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs

Approval Request Comment
[Feature/regressing bug #]:

Browser UI Telemetry.


[User impact if declined]:

Users submitting Telemetry will continue to send us counts of how many times they choose to open private browsing windows.


[Describe test coverage new/current, TreeHerder]:

Manually tested, as we don't have too many tests for BrowserUITelemetry. :( bug 944481.


[Risks and why]: 

This is extremely low risk - even if BrowserUITelemetry failed to work at all, user's not looking at about:telemetry would not know about it. We, however, would know, once we realized that we were getting no data in for it.

I have, however, tested that gathering click counts on other toolbar buttons continues to work.


[String/UUID change made/needed]:

None.
Attachment #8624934 - Flags: approval-mozilla-beta?
Attachment #8624934 - Flags: approval-mozilla-aurora?
Marking 39+ as affected.
Comment on attachment 8624934 [details]
MozReview Request: Bug 1176391 - Remove privatebrowser-button and context menu click counts from BrowserUITelemetry.jsm. r?Gijs

Feels like we've had some telemetry bugs that shouldn't affect anything else still cause problems.  But let's take this anyway. I don't like the idea of leaking private browsing info even if it's just # of windows opened.
Attachment #8624934 - Flags: approval-mozilla-beta?
Attachment #8624934 - Flags: approval-mozilla-beta+
Attachment #8624934 - Flags: approval-mozilla-aurora?
Attachment #8624934 - Flags: approval-mozilla-aurora+
This patch landed on mozilla-central this morning but didn't got marked for some reason.
https://hg.mozilla.org/mozilla-central/rev/fffa73480bd0
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla41
(In reply to Liz Henry (:lizzard) from comment #19)
> Feels like we've had some telemetry bugs that shouldn't affect anything else
> still cause problems.  But let's take this anyway. I don't like the idea of
> leaking private browsing info even if it's just # of windows opened.

I'm not sure what this refers to, can you expand on that?
Ally: what do you think about the Telemetry for the Firefox Forget feature? There's already UITelemetry for the button itself and there are also timings for how well it's performing. There's active perf work going on in bug 1158561 (note my team is peripherally involved in that)
Flags: needinfo?(ally)
(In reply to Vladan Djeric (:vladan) -- please needinfo! from comment #24)
> Ally: what do you think about the Telemetry for the Firefox Forget feature?
> There's already UITelemetry for the button itself and there are also timings
> for how well it's performing. There's active perf work going on in bug
> 1158561 (note my team is peripherally involved in that)

tl;dr Probes are use it or lose it.

I have found no evidence that this probe is being used or has resulted in material improvements to Firefox in the 9+ months it has been collecting data. The UI telemetry dashboard has been down for several months, and apparently no one cares about any of the UI telemetry data enough to have it fixed. I understand there have been no custom queries run by mreid regarding the data, and Philip Sackl was unaware of the probe's existence and thus could not have used it to make any ux/ui decisions.

Back it out due to lack of usage.
Flags: needinfo?(ally)
I'ld like to start by saying that I'm completely in agreement with your decision here, Allison, and by thanking you for the guidelines.  :)

(In reply to Allison Naaktgeboren please NEEDINFO? :ally from comment #26)
> The UI telemetry dashboard has been down for several months, and
> apparently no one cares about any of the UI telemetry data enough to have it
> fixed.

To be fair, this is due to a lack of resources, not a lack of interest or care.  It's a part-time project being worked on by an (in my opinion) understaffed team, who have to concentrate their efforts elsewhere.

I will likely run a quick report to summarize the data we've collected so far, which should give us a good basis to make decisions on when we later re-visit the Forget feature.
You need to log in before you can comment on or make changes to this bug.