collect pinning telemetry in release for mozilla sites that are essential for the operation of firefox

RESOLVED FIXED in Firefox 67

Status

()

enhancement
P1
normal
RESOLVED FIXED
5 months ago
4 months ago

People

(Reporter: keeler, Assigned: keeler)

Tracking

(Blocks 1 bug)

unspecified
mozilla67
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox67 fixed)

Details

(Whiteboard: [psm-assigned])

Attachments

(2 attachments)

For context, this is blocking bug 1494431 because we currently lack the necessary data to move forward. Since we need to collect release data, and it will take a while to ride there, can I ask to prioritize this change now-ish so we can measure impact when 66 hits the release channel?

Assignee: nobody → dkeeler
Priority: -- → P1
Whiteboard: [psm-assigned]

The telemetry histograms CERT_PINNING_MOZ_TEST_RESULTS_BY_HOST and
CERT_PINNING_MOZ_RESULTS_BY_HOST collect information about whether or not key
pinning checks succeed in connections to Mozilla sites that affect the
functionality of Firefox. This patch changes these histograms so that we also
collect this data by default in release.

Comment on attachment 9038379 [details]
bug 1521940 - collect pinning telemetry in release for mozilla sites that are essential for the operation of firefox data-review=chutten

Rob - let me know if I should ask someone else for this data review, but basically we're asking to expand two currently-existing telemetry probes to be opt-out rather than opt-in on release. They're category 1 and 2 data: part technical details (was the client able to build the right certificate chain for connections to specific Mozilla hosts) and part interaction (in some cases users would only connect to these hosts if they're using specific features).

Attachment #9038379 - Flags: review?(rrayborn)

(re comment 3)

Flags: needinfo?(rrayborn)

Comment on attachment 9038379 [details]
bug 1521940 - collect pinning telemetry in release for mozilla sites that are essential for the operation of firefox data-review=chutten

I've been told this needs to go through a full review at this point.

Flags: needinfo?(rrayborn)
Attachment #9038379 - Flags: review?(rrayborn)

Comment 7

4 months ago
Comment on attachment 9040858 [details]
1521940-pinning-telemetry-data-review

Are either of these histograms accumulated to when the user navigates to websites?
Flags: needinfo?(dkeeler)

Yes, if they visit the following:

addons.mozilla.org
addons.mozilla.net
aus4.mozilla.org
aus5.mozilla.org
firefox.com
accounts.firefox.com
api.accounts.firefox.com
sync.services.mozilla.com
cdn.mozilla.net
cdn.mozilla.org
download.mozilla.org
services.mozilla.com
telemetry.mozilla.org
testpilot.firefox.com
crash-reports.mozilla.com
crash-reports-xpsp2.mozilla.com
crash-stats.mozilla.com

Flags: needinfo?(dkeeler)

Note that most of these are background services of Firefox. Users almost never "navigate" there, but all Firefox clients connect to them.

Comment on attachment 9040858 [details]
1521940-pinning-telemetry-data-review

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file ([Histograms.json](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Histograms.json)) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes. Dana Keeler is responsible for this collection.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 3, Web Activity.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Unclear.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview- due to collection of Category 3 data by default on release channel.

This collection may prove eligible for default-on collection in release channel if that the collection is limited to the list of properties in comment #8 is considered an adequate risk mitigation. (see https://wiki.mozilla.org/Firefox/Data_Collection#Eligibility_for_Default_on_Data_Collection)

ni?merwin for a decision: Is it okay that we are recording the number of (and some technical details about) connections (mostly infrastructure, but potentially identifiably web activity) to the mozilla-owned origins listed in comment #8?
Flags: needinfo?(merwin)
Attachment #9040858 - Flags: review?(chutten) → review-

Comment 11

4 months ago

I think that is acceptable for the reason Julien mentioned - the user isn't navigating to these and therefore this isn't revealing of user web browsing. This is essentially interaction data because it reveals that the user interacted with a particular service integrated into Fx.

Flags: needinfo?(merwin)
Comment on attachment 9040858 [details]
1521940-pinning-telemetry-data-review

That's very clear guidance, thank you Marshall. I'll rerun the review.

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file ([Histograms.json](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Histograms.json)) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes. Dana Keeler is responsible for this collection.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+
Attachment #9040858 - Flags: review- → review+
Attachment #9038379 - Attachment description: bug 1521940 - collect pinning telemetry in release for mozilla sites that are essential for the operation of firefox r?rrayborn → bug 1521940 - collect pinning telemetry in release for mozilla sites that are essential for the operation of firefox data-review=chutten

Comment 13

4 months ago
Pushed by dkeeler@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/906716408541
collect pinning telemetry in release for mozilla sites that are essential for the operation of firefox data-review=chutten r=chutten

Comment 14

4 months ago
bugherder
Status: NEW → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
You need to log in before you can comment on or make changes to this bug.