1773701 - Add Telemetry for the Email Tracking Protection

Status: RESOLVED FIXED

(Core :: Privacy: Anti-Tracking, task, P2)

Description

[Tim Huang[:timhuang]]

We would like to collect telemetry for the email tracking protection to answer following questions.

• How often the email trackers will be blocked(Also, which category)?
  • i.e. the ratio of trackers being blocked in emails
• Where do we see the email trackers, webmail clients or others?
• How many email trackers (aggregate by the tracking company) have been blocked per top-level site?

Comment 1

[Tim Huang[:timhuang]]

Some details about the telemetry that we want to collect in this bug

1. We will collect the number of classified email trackers. This will allow us to know how many email trackers have been blocked.
2. We will collect the category of the classified email trackers(base or content).
3. We will collect if the email tracker is loaded under a email webapp or not.
4. We would like to know how many email trackers have been loaded (aggregate by the site) per email webapp page. However, it needs to be done by using ContentBlockingLog, which requires Bug 1773703. So, I won't be implementing this in this bug. I will open another bug to implement this.

Comment 2

[Tim Huang[:timhuang]]

Attachment: Bug 1773701 - Part 1: Add email tracking data collection feature to SafeBrowsing.jsm. r?dimi

Comment 3

[Tim Huang[:timhuang]]

Attachment: Bug 1773701 - Part 2: Implement the email tracking data collection feature. r?dimi!

Depends on D151521

Comment 4

[Tim Huang[:timhuang]]

Attachment: Bug 1773701 - Part 3: Add email tracking telemetry probes. r?dimi

This patch adds two telemetry probes.

1. The counter of classified third-party email trackers in different
   categories.
2. The counter of whether the email tracker is loaded under a email
   webapp.
   We use a list of known email webapp domains to check if the top-level
   content is an email webapp.

Depends on D151522

Comment 5

[Tim Huang[:timhuang]]

Attachment: Bug 1773701 - Part 4: Add a test to verify the email tracking telemetry. r?dimi!

Depends on D151523

Comment 6

[Tim Huang[:timhuang]]

Attachment: data-request.md

Comment 7

[Chris H-C :chutten]

Comment on attachment 9285170 [details]
data-request.md

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Tim Huang is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

---

Result: datareview+

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fab41f0c857f
Part 1: Add email tracking data collection feature to SafeBrowsing.jsm. r=dimi
https://hg.mozilla.org/integration/autoland/rev/8d3727e19ade
Part 2: Implement the email tracking data collection feature. r=dimi
https://hg.mozilla.org/integration/autoland/rev/2d2f3f95544d
Part 3: Add email tracking telemetry probes. r=dimi
https://hg.mozilla.org/integration/autoland/rev/8ec2108b4a5b
Part 4: Add a test to verify the email tracking telemetry. r=dimi

Comment 9

[Cosmin Sabou [:CosminS]]

Backed out for causing hybrid build bustages.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception%2Cusercancel&revision=8ec2108b4a5b5be82ae0f5d1b0f1d1fcb4b4de0c&selectedTaskRun=ETMVdBtUTyC1nXWEepho1Q.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=384792244&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/afd5e4a481f7963f09f6bd0b7ac166faedf57863

Comment 10

[Cosmin Sabou [:CosminS]]

Please have a look over this kind of failures too: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=linux%2C18.04%2Cx64%2Cwebrender%2Cdebug%2Cxpcshell%2Ctests%2Ctest-linux1804-64-qr%2Fdebug-xpcshell%2Cx1&revision=8ec2108b4a5b5be82ae0f5d1b0f1d1fcb4b4de0c&selectedTaskRun=ClvLGcHDTkO6ip83ONH3hg.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=384793226&repo=autoland

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f3c636e27b0b
Part 1: Add email tracking data collection feature to SafeBrowsing.jsm. r=dimi
https://hg.mozilla.org/integration/autoland/rev/d393067941bd
Part 2: Implement the email tracking data collection feature. r=dimi
https://hg.mozilla.org/integration/autoland/rev/aecd7795068a
Part 3: Add email tracking telemetry probes. r=dimi
https://hg.mozilla.org/integration/autoland/rev/37b4d8b45886
Part 4: Add a test to verify the email tracking telemetry. r=dimi

Comment 12

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/f3c636e27b0b
https://hg.mozilla.org/mozilla-central/rev/d393067941bd
https://hg.mozilla.org/mozilla-central/rev/aecd7795068a
https://hg.mozilla.org/mozilla-central/rev/37b4d8b45886