1578225 - Perma test_ext_extension_content_telemetry.js | test_telemetry - [test_telemetry : 326] Extension left running at test shutdown - "running" == "unloaded" when Gecko version increases to 72 on 2019-10-21

Status: VERIFIED FIXED

(WebExtensions :: General, defect)

Description

[Raul Gurzau (:RaulG)]

[Tracking Requested - why for this release]:

Version increase simulation:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=264546324&repo=try&lineNumber=2819

Log link: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=264546324&repo=try&lineNumber=2979

Log snippet:
[task 2019-09-02T11:39:36.914Z] 11:39:36 INFO - PID 28585 | [Parent 28585, Main Thread] WARNING: '!workerClassifier', file /builds/worker/workspace/build/src/netwerk/url-classifier/AsyncUrlChannelClassifier.cpp, line 785
[task 2019-09-02T11:39:36.914Z] 11:39:36 INFO - PID 28585 | ++DOMWINDOW == 3 (0x7f026d63b000) [pid = 28627] [serial = 3] [outer = 0x7f026e556f20]
[task 2019-09-02T11:39:36.916Z] 11:39:36 INFO - PID 28585 | [28670, MainThread] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
[task 2019-09-02T11:39:36.916Z] 11:39:36 INFO - PID 28585 | [28670, MainThread] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /builds/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 198
[task 2019-09-02T11:39:36.917Z] 11:39:36 INFO - PID 28585 | Couldn't convert chrome URL: chrome://branding/locale/brand.properties
[task 2019-09-02T11:39:36.917Z] 11:39:36 INFO - Unexpected exception undefined - timed out after 50 tries.
[task 2019-09-02T11:39:36.917Z] 11:39:36 INFO - undefined
[task 2019-09-02T11:39:36.918Z] 11:39:36 INFO - exiting test
[task 2019-09-02T11:39:36.918Z] 11:39:36 WARNING - TEST-UNEXPECTED-FAIL | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_extension_content_telemetry.js | test_telemetry - [test_telemetry : 326] Extension left running at test shutdown - "running" == "unloaded"
[task 2019-09-02T11:39:36.919Z] 11:39:36 INFO - resource://testing-common/ExtensionXPCShellUtils.jsm:ExtensionWrapper/<:326
[task 2019-09-02T11:39:36.919Z] 11:39:36 INFO - /builds/worker/workspace/build/tests/xpcshell/head.js:_execute_test/<:636
[task 2019-09-02T11:39:36.920Z] 11:39:36 INFO - exiting test
[task 2019-09-02T11:39:36.920Z] 11:39:36 INFO - NS_ERROR_ABORT:
[task 2019-09-02T11:39:36.921Z] 11:39:36 INFO - _abort_failed_test@/builds/worker/workspace/build/tests/xpcshell/head.js:789:20
[task 2019-09-02T11:39:36.921Z] 11:39:36 INFO - do_report_result@/builds/worker/workspace/build/tests/xpcshell/head.js:890:5
[task 2019-09-02T11:39:36.921Z] 11:39:36 INFO - Assert<@/builds/worker/workspace/build/tests/xpcshell/head.js:67:21
[task 2019-09-02T11:39:36.922Z] 11:39:36 INFO - proto.report@resource://testing-common/Assert.jsm:233:10
[task 2019-09-02T11:39:36.922Z] 11:39:36 INFO - equal@resource://testing-common/Assert.jsm:275:8
[task 2019-09-02T11:39:36.923Z] 11:39:36 INFO - ExtensionWrapper/<@resource://testing-common/ExtensionXPCShellUtils.jsm:326:24
[task 2019-09-02T11:39:36.923Z] 11:39:36 INFO - _execute_test/<@/builds/worker/workspace/build/tests/xpcshell/head.js:636:28
[task 2019-09-02T11:39:36.923Z] 11:39:36 INFO - async*_execute_test@/builds/worker/workspace/build/tests/xpcshell/head.js:645:5
[task 2019-09-02T11:39:36.924Z] 11:39:36 INFO - @-e:1:1
[task 2019-09-02T11:39:36.926Z] 11:39:36 INFO - Error: Found unexpected files in temporary directory: generated-extension.xpi, generated-extension-1.xpi at resource://testing-common/AddonTestUtils.jsm:466
[task 2019-09-02T11:39:36.926Z] 11:39:36 INFO - init/<@resource://testing-common/AddonTestUtils.jsm:466:15
[task 2019-09-02T11:39:36.926Z] 11:39:36 INFO - _execute_test/<@/builds/worker/workspace/build/tests/xpcshell/head.js:636:28
[task 2019-09-02T11:39:36.927Z] 11:39:36 INFO - async*_execute_test@/builds/worker/workspace/build/tests/xpcshell/head.js:645:5
[task 2019-09-02T11:39:36.928Z] 11:39:36 INFO - @-e:1:1

Comment 1

[Luca Greco [:rpl] [:luca] [:lgreco]]

The failure is triggered by some of the WebExtensions-related telemetry histograms which are currently set to expire on Firefox 72.

Hey Ben,
before I proceed to prepare the data review request and the patch to update the expire version of these histograms,
I wanted to double-check with you if the histograms keyed by addon id are in used in some STMO dashboards or reports.

If these keyed histograms are actually in use I'm going to re-new them along with the non-keyed ones.

Comment 2

[Ben Miroglio [:bmiroglio]]

Hey Luca,

Thanks for reaching out. Yes, we use these histograms in production datasets, namely the addons_daily dataset. Please do renew them!

Comment 3

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: data-review-request-bug-1578225-renew-webext-histograms.md

Hi Chris,
the attached data review request is related to telemetry histograms recorded by the WebExtensions framework that are going to expire in Firefox 72.
This telemetry data is currently in use and we'd like to extend their expiry release (as also confirmed by Ben in Comment 2) of 5 additional releases (I've also added some notes about this usage in the data review request for future reference).

Comment 4

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: Bug 1578225 - Extended expiry version for the WebExtensions telemetry histograms. r?janerik!

Comment 5

[Chris H-C :chutten]

Comment on attachment 9094282 [details]
data-review-request-bug-1578225-renew-webext-histograms.md

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file [Histograms.json](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Histograms.json) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

No. This collection will expire in Firefox 77.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :rpl is responsible for renewing or removing the collection before it expires in Firefox 77.

---
Result: datareview+

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4ae54d71bad0
Extended expiry version for the WebExtensions telemetry histograms. r=janerik

Comment 7

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/4ae54d71bad0

Comment 8

[Alex Cornestean]

Hello,

Will this fix require manual validation? If yes, please provide some steps to reproduce in order to correctly test it and also, please set the "qe-verify+" flag. Otherwise, could the "qe-verify-" flag be added? Thanks!

Comment 9

[Luca Greco [:rpl] [:luca] [:lgreco]]

The fix should be verified by running a new version increase simulation (as the one mentioned in comment 0), to verify that the failure is not triggered anymore as expected.

Comment 10

[Luca Greco [:rpl] [:luca] [:lgreco]]

I did forgot to set qe-verify- (which is what I meant to say in comment 9, the issue will be verified when a Code Sheriff will run a new beta simulation, a build similar to the one we do on a beta merge, like the one that Raul mentioned in comment 0).

Comment 11

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Verified fixed with latest version increase simulation: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception&revision=b30d003a9e626cc9ace7f954c0a5eebbb2d5c9cd