1501974 - Add telemetry to know the remote lookup result for archive files

Status: RESOLVED FIXED

(Toolkit :: Safe Browsing, enhancement, P2)

Description

[Dimi Lee [:dimi]]

This could help us know the improvement after including archived files information in the remote lookup request.
I think this is important because we can reference the result here to guess the potential gain if we want to implement dmg/RAR extraction.


This should land earlier so we can get the result "before" having zip extraction feature.

Comment 1

[Dimi Lee [:dimi]]

The idea I have in mind is not limit the telemetry just for archived files. Maybe a telemetry with file extension as the key is more flexible and general.

Comment 2

[Dimi Lee [:dimi]]

Attachment: WIP Patch

Comment 3

[Dimi Lee [:dimi]]

Attachment: Bug 1501974 - Add keyed telemetry to know the download protection remote lookup result . r?francois

Add APPLICATION_REPUTATION_SERVER_VERDICT_2 telemetry with file extension as the key.
This can give us an idea about the current detecion rate for archived
files.

Comment 4

[Dimi Lee [:dimi]]

Attachment: data-review-request.txt

Comment 5

[Chris H-C :chutten]

Comment on attachment 9036852 [details]
data-review-request.txt

Preliminary note:

From your comments about the questions being answered and the analysis you plan on performing, it seems to me that this measurement might serve its purposes if it expired in, say, Firefox 71 instead of being permanent. If so, it would be serving the principles of Lean Data to do so. (and we could always renew it if it then proves useful to have on an ongoing basis).

To be clear it passes Data Collection Review whether it expires or is permanent.

Speaking of which, for permanent collections it is wise to have them covered by an automated test ensuring that your collection doesn't break in the future. (without the expiry/renewal cycle ensuring we take a look at them every so often, collections can break without notice unless covered by tests).

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file (Histograms.json), the Probe Dictionary, and on telemetry.mozilla.org's Measurement Dashboards.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes. Dimi Lee is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction. (Though the result of the reputation request is Category 1, the reputation requests map 1:1 onto user interactions.)

Is the data collection request for default-on or default-off?

Default on, all channels.

Does the instrumentation include the addition of any new identifiers?

No. It does add a string key but the key must be a member of a build-time list. The key is of limited domain and doesn't fit the definition of "identifier"

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---

Result: datareview+

Comment 6

[Dimi Lee [:dimi]]

(In reply to Chris H-C :chutten from comment #5)

Comment on attachment 9036852 [details]
data-review-request.txt

Preliminary note:

From your comments about the questions being answered and the analysis you plan on performing, it seems to me that this measurement might serve its purposes if it expired in, say, Firefox 71 instead of being permanent. If so, it would be serving the principles of Lean Data to do so. (and we could always renew it if it then proves useful to have on an ongoing basis).

To be clear it passes Data Collection Review whether it expires or is permanent.

Speaking of which, for permanent collections it is wise to have them covered by an automated test ensuring that your collection doesn't break in the future. (without the expiry/renewal cycle ensuring we take a look at them every so often, collections can break without notice unless covered by tests).

Thank you for your suggestion, I'll change it to expired in Firefox 71 :)

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f412108e7d77
Add keyed telemetry to know the download protection remote lookup result . r=francois

Comment 8

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/f412108e7d77