1662857 - Report via Telemetry when an add-on is blocked

Status: RESOLVED FIXED

(Toolkit :: Blocklist Implementation, enhancement, P1)

Description

[Jorge Villalobos [:jorgev] (he/him)]

We want to get some answers on data questions around the add-ons blocklist so we can better design the future of add-on signing. After doing some digging around, it looks like we don't report on telemetry when add-ons are blocked. Tim Smith made this query for me based on the site_blocked, but it's not clear if this refers to the blocklist of the third party site install warning.

What I think we need is to record an event for when (1) an extension is not installed due to it being blocked, and (2) when an extension that is installed is being disabled due to a blocklist update. I don't know if this requires sending new events of just adding more details to the existing installation and disabling events. As long as we can query them, either should be fine.

Comment 1

[Rob Wu [:robwu]]

(In reply to Jorge Villalobos [:jorgev] (he/him) from comment #0)

Tim Smith made this query for me based on the site_blocked, but it's not clear if this refers to the blocklist of the third party site install warning.

This indeed refers to the third-party site warning (mixed with being blocked by enterprise policy) (source).

Comment 2

[Rob Wu [:robwu]]

Attachment: data-review-request-blocklist-blocked-addons.md

Comment 3

[Rob Wu [:robwu]]

Attachment: Bug 1662857 - Record blocklist source in telemetry

Comment 4

[Rob Wu [:robwu]]

Attachment: Bug 1662857 - Put version check in getNewestCompatibleUpdate

AddonUpdateChecker.getNewestCompatibleUpdate has only one non-test
consumer in XPIInstall.jsm. That consumer filters out older versions
after finding an update.

In order to correctly report whether an update is blocked by the
blocklist, the version comparison has been moved inside the
getNewestCompatibleUpdate method, with no other changes.

Comment 5

[Rob Wu [:robwu]]

Attachment: Bug 1662857 - Avoid intermittent failures by AddonTestUtils.promiseCompleteInstall

Tests that rely on AddonTestUtils.promiseCompleteInstall to await the
completion of an update may encounter intermittent test failures,
because the test helper relies on onInstallEnded, which is sent before
all cleanup has run (removing temp xpi, removing staged directory).

The install() method returns a promise that is resolved right after
the cleanup of those temporary files. So await that.

Comment 6

[Rob Wu [:robwu]]

Attachment: Bug 1662857 - Report blocklist.addonBlockChange to telemetry

Report to telemetry when add-on is blocked by the blocklist, or when an
updated/installation resutls in the unblocking of an add-on.

Comment 7

[Rob Wu [:robwu]]

These patches implement only part of the data review request. There is another one that adds "blockedAddons" to the main ping (as described in the data review request), but I'd like to take another look before attaching that patch too.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f8428a15daea
Record blocklist source in telemetry r=Gijs
https://hg.mozilla.org/integration/autoland/rev/7fb58c40e64b
Put version check in getNewestCompatibleUpdate r=mixedpuppy
https://hg.mozilla.org/integration/autoland/rev/ba744d7a1e09
Avoid intermittent failures by AddonTestUtils.promiseCompleteInstall r=rpl
https://hg.mozilla.org/integration/autoland/rev/c7249f0b44dc
Report blocklist.addonBlockChange to telemetry r=rpl,Gijs

Comment 10

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/f8428a15daea
https://hg.mozilla.org/mozilla-central/rev/7fb58c40e64b
https://hg.mozilla.org/mozilla-central/rev/ba744d7a1e09
https://hg.mozilla.org/mozilla-central/rev/c7249f0b44dc

Comment 11

[Travis Long [:travis_]]

Comment on attachment 9206694 [details]
data-review-request-blocklist-blocked-addons.md

Data Review

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the Glean-Dictionary and the metric definition files

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the standard telemetry preference

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

Data to be collected for 6 months

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1 & 2

5. Is the data collection request for default-on or default-off?

Default-on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

7. Is the data collection covered by the existing Firefox privacy notice?

Yes

8. Does the data collection use a third-party collection tool?

No

Result

data-review+