1665938 - Add telemetry to report if a WebExtension engine is saved in the search settings but not in the add-on manager

Status: RESOLVED FIXED

(Firefox :: Search, task, P2)

Description

[Mark Banner (:standard8)]

Currently, we do not have any validation that a WebExtension referenced in the saved search settings is actually installed and active in the application.

Additionally, we may have cases where the saved information for the WebExtension is out of date.

As we are looking to improve both of these cases, we would like to deploy some telemetry so that we can check for any likely negative/unexpected effects and take those into account when activating this code.

The code implemented to generate the telemetry will be the code that will do the future validation & update of settings - just with the actual changes disabled until we've checked what the telemetry says.

Comment 1

[Mark Banner (:standard8)]

Attachment: Bug 1665938 - Add telemetry to report if a WebExtension engine is saved in the search settings but not in the add-on manager. r?daleharvey

Comment 2

[Mark Banner (:standard8)]

Attachment: Search Engine Webextension Telemetry Data Review

Comment 3

[Teon Brooks [:teon]]

Comment on attachment 9180776 [details]
Search Engine Webextension Telemetry Data Review

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file Scalars.yaml and the Probe Dictionary.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :standard8 is responsible.
No. This collection will expire in six months.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :standard8 is responsible for renewing or removing the collection before it expires in six months.

---

Result: datareview+

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ce37b9b99fd6
Add telemetry to report if a WebExtension engine is saved in the search settings but not in the add-on manager. r=daleharvey

Comment 5

[Sandor Molnar[:smolnar]]

Backed out for causing browser_urlbar_event_telemetry failures

Backout link: https://hg.mozilla.org/integration/autoland/rev/55c26a4f6bdc5d2e5a83f8b83b897730f868cf43

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&collapsedPushes=803163&resultStatus=testfailed%2Cbusted%2Cexception%2Crunning%2Cpending%2Crunnable&revision=ce37b9b99fd622db4506a63761988f84aa809707

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=318825154&repo=autoland&lineNumber=10827

INFO - TEST-PASS | browser/components/urlbar/tests/browser/browser_urlbar_event_telemetry.js | Expected key provider must be in actual extra. Actual keys: [elapsed,numChars,numWords,selIndex,selType,provider]. - true == true -
[task 2020-10-16T11:57:39.808Z] 11:57:39 INFO - Buffered messages finished
[task 2020-10-16T11:57:39.809Z] 11:57:39 INFO - TEST-UNEXPECTED-FAIL | browser/components/urlbar/tests/browser/browser_urlbar_event_telemetry.js | extra[provider] must match in event urlbar#engagement#click. - "HeuristicFallback" matches "Autofill" - JS frame :: resource://testing-common/TelemetryTestUtils.jsm :: assertEvents :: line 239
[task 2020-10-16T11:57:39.809Z] 11:57:39 INFO - Stack trace:
[task 2020-10-16T11:57:39.809Z] 11:57:39 INFO - resource://testing-common/TelemetryTestUtils.jsm:assertEvents:239
[task 2020-10-16T11:57:39.809Z] 11:57:39 INFO - chrome://mochitests/content/browser/browser/components/urlbar/tests/browser/browser_urlbar_event_telemetry.js:test:1583
[task 2020-10-16T11:57:39.810Z] 11:57:39 INFO - Running test at index 4
[task 2020-10-16T11:57:39.811Z] 11:57:39 INFO - Type something, click one-off and press enter.
[task 2020-10-16T11:57:39.811Z] 11:57:39 INFO - GECKO(1512) | [Parent 1512, Main Thread] WARNING: '!mOwner', file /builds/worker/checkouts/gecko/dom/events/Event.cpp:704
[task 2020-10-16T11:57:39.812Z] 11:57:39 INFO - Awaiting for the urlbar panel to open
[task 2020-10-16T11:57:39.827Z] 11:57:39 INFO - GECKO(1512) | [Parent 1512, Main Thread] WARNING: 'NS_FAILED(targetPrincipal->GetAsciiOrigin(targetOrigin))', file /builds/worker/checkouts/gecko/toolkit/components/antitracking/AntiTrackingUtils.cpp:347

Comment 6

[Mark Banner (:standard8)]

Issue was in bug 1671404.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/92b7fcf7b9c2
Add telemetry to report if a WebExtension engine is saved in the search settings but not in the add-on manager. r=daleharvey

Comment 8

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/92b7fcf7b9c2