Enable certificate error page event telemetry on release to support a pref flip study

RESOLVED FIXED in Firefox 64

Status

()

defect
P1
normal
RESOLVED FIXED
7 months ago
7 months ago

People

(Reporter: RT, Assigned: johannh)

Tracking

64 Branch
Firefox 65
Points:
---
Bug Flags:
qe-verify -

Firefox Tracking Flags

(firefox64 fixed, firefox65 fixed)

Details

Attachments

(2 attachments, 1 obsolete attachment)

Reporter

Description

7 months ago
Posted file request.txt (obsolete) —
This request is related to the shield study we want to run in 64 that looks at new certificate error page impact on retention and engagement (see bug 1501004 and bug 1503572. We need additional telemetry enabled on release for this study and the request is to have the probes detailed on bug 1503572 enabled on release.
Assignee

Updated

7 months ago
Assignee: nobody → jhofmann
Status: NEW → ASSIGNED
Priority: -- → P1
Reporter

Comment 1

7 months ago
Hi Francois, this is a request to enable on release 64 the event telemetry probes you approved on pre-release on bug 1484255.
Could you please help us with a review on this?
Flags: needinfo?(francois)
Attachment #9023199 - Attachment mime type: application/x-iwork-pages-sffpages → text/plain
Flags: needinfo?(francois)
Attachment #9023199 - Flags: review?(francois)
Comment on attachment 9023199 [details]
request.txt

Actually, that's not a text file. It looks like the Apple Pages format. Can you please attach it as a text file since I don't have a Mac and can't open this?
Flags: needinfo?(rtestard)
Attachment #9023199 - Flags: review?(francois)
Assignee

Comment 4

7 months ago
Just a note because I know folks tend to get concerned about Event Telemetry volume:

While I don't expect these events to come in an overwhelming amount, even on release, the sending of this telemetry can be flexibly toggled via the security.certerrors.recordEventTelemetry pref. This allows us to either proactively or reactively limit the amount of users that send the events to support the shield study.

Let me know if this is something you'd want to do.
Reporter

Comment 5

7 months ago
(In reply to François Marier [:francois] from comment #2)
> Comment on attachment 9023199 [details]
> request.txt
> 
> Actually, that's not a text file. It looks like the Apple Pages format. Can
> you please attach it as a text file since I don't have a Mac and can't open
> this?

Apologies! Now attaching in plain text format...
Flags: needinfo?(rtestard)

Comment 6

7 months ago
(In reply to Johann Hofmann [:johannh] from comment #4)
> Just a note because I know folks tend to get concerned about Event Telemetry
> volume:
> 
> While I don't expect these events to come in an overwhelming amount, even on
> release, the sending of this telemetry can be flexibly toggled via the
> security.certerrors.recordEventTelemetry pref. This allows us to either
> proactively or reactively limit the amount of users that send the events to
> support the shield study.
> 
> Let me know if this is something you'd want to do.

It's tied to user interaction (and with a small piece of the UI at that) so I am not too concerned about volume here. 

If it becomes a problem we also have the mechanism of disabling the event recording via "setEventRecordingEnabled" on the category of the voluminous events, delivered by addon to affected clients. We haven't had to do that to anyone yet, and I don't anticipate this being the first :)
Attachment #9023199 - Attachment is obsolete: true
Comment on attachment 9023591 [details]
request_plain_text.txt

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in Events.yaml.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Not permanent.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 2 at most.

5) Is the data collection request for default-on or default-off?

Default ON, all channels.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, telemetry alerts are sufficient.
Attachment #9023591 - Flags: review+

Comment 8

7 months ago
Pushed by jhofmann@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4e91b23279a9
Enable certificate error page event telemetry on release. r=janerik

Comment 9

7 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/4e91b23279a9
Status: ASSIGNED → RESOLVED
Closed: 7 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 65
Assignee

Comment 10

7 months ago
Comment on attachment 9023578 [details]
Bug 1505310 - Enable certificate error page event telemetry on release. r=janerik

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: None

User impact if declined: No user impact either way, we want to be able to measure certificate errors in release 64.

Is this code covered by automated tests?: Unknown

Has the fix been verified in Nightly?: No

Needs manual test from QE?: No

If yes, steps to reproduce: 

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): Just updating the opt-out status of a telemetry probe.

String changes made/needed: None
Attachment #9023578 - Flags: approval-mozilla-beta?
Comment on attachment 9023578 [details]
Bug 1505310 - Enable certificate error page event telemetry on release. r=janerik

make telemetry for cert errors opt-out, approved for 64.0b11
Attachment #9023578 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
You need to log in before you can comment on or make changes to this bug.