1505310 - Enable certificate error page event telemetry on release to support a pref flip study

Status: RESOLVED FIXED

(Firefox :: Security, defect, P1)

Description

[Romain Testard [:RT]]

Attachment: request.txt

This request is related to the shield study we want to run in 64 that looks at new certificate error page impact on retention and engagement (see bug 1501004 and bug 1503572. We need additional telemetry enabled on release for this study and the request is to have the probes detailed on bug 1503572 enabled on release.

Comment 1

[Romain Testard [:RT]]

Hi Francois, this is a request to enable on release 64 the event telemetry probes you approved on pre-release on bug 1484255.
Could you please help us with a review on this?

Comment 2

[François Marier [:francois]]

Comment on attachment 9023199 [details]
request.txt

Actually, that's not a text file. It looks like the Apple Pages format. Can you please attach it as a text file since I don't have a Mac and can't open this?

Comment 3

[Johann Hofmann [:johannh]]

Attachment: Bug 1505310 - Enable certificate error page event telemetry on release. r=janerik

Comment 4

[Johann Hofmann [:johannh]]

Just a note because I know folks tend to get concerned about Event Telemetry volume:

While I don't expect these events to come in an overwhelming amount, even on release, the sending of this telemetry can be flexibly toggled via the security.certerrors.recordEventTelemetry pref. This allows us to either proactively or reactively limit the amount of users that send the events to support the shield study.

Let me know if this is something you'd want to do.

Comment 5

[Romain Testard [:RT]]

Attachment: request_plain_text.txt

(In reply to François Marier [:francois] from comment #2)
> Comment on attachment 9023199 [details]
> request.txt
> 
> Actually, that's not a text file. It looks like the Apple Pages format. Can
> you please attach it as a text file since I don't have a Mac and can't open
> this?

Apologies! Now attaching in plain text format...

Comment 6

[Chris H-C :chutten]

(In reply to Johann Hofmann [:johannh] from comment #4)
> Just a note because I know folks tend to get concerned about Event Telemetry
> volume:
> 
> While I don't expect these events to come in an overwhelming amount, even on
> release, the sending of this telemetry can be flexibly toggled via the
> security.certerrors.recordEventTelemetry pref. This allows us to either
> proactively or reactively limit the amount of users that send the events to
> support the shield study.
> 
> Let me know if this is something you'd want to do.

It's tied to user interaction (and with a small piece of the UI at that) so I am not too concerned about volume here. 

If it becomes a problem we also have the mechanism of disabling the event recording via "setEventRecordingEnabled" on the category of the voluminous events, delivered by addon to affected clients. We haven't had to do that to anyone yet, and I don't anticipate this being the first :)

Comment 7

[François Marier [:francois]]

Comment on attachment 9023591 [details]
request_plain_text.txt

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in Events.yaml.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Not permanent.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 2 at most.

5) Is the data collection request for default-on or default-off?

Default ON, all channels.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, telemetry alerts are sufficient.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4e91b23279a9
Enable certificate error page event telemetry on release. r=janerik

Comment 9

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/4e91b23279a9

Comment 10

[Johann Hofmann [:johannh]]

Comment on attachment 9023578 [details]
Bug 1505310 - Enable certificate error page event telemetry on release. r=janerik

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: None

User impact if declined: No user impact either way, we want to be able to measure certificate errors in release 64.

Is this code covered by automated tests?: Unknown

Has the fix been verified in Nightly?: No

Needs manual test from QE?: No

If yes, steps to reproduce: 

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): Just updating the opt-out status of a telemetry probe.

String changes made/needed: None

Comment 11

[Julien Cristau [:jcristau] (back August 5)]

Comment on attachment 9023578 [details]
Bug 1505310 - Enable certificate error page event telemetry on release. r=janerik

make telemetry for cert errors opt-out, approved for 64.0b11

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/467e62db7b14