1425957 - Various webextension tests are going to permafail when the Gecko version number is bumped to 60

Status: VERIFIED FIXED

(WebExtensions :: General, defect, P3)

Description

[Ryan VanderMeulen [:RyanVM]]

[Tracking Requested - why for this release]: Permafailing tests when the Gecko version number is bumped to 60.

Looks like the probes these tests depend on are set to expire when Gecko hits 60:
WEBEXT_BACKGROUND_PAGE_LOAD_MS
WEBEXT_CONTENT_SCRIPT_INJECTION_MS
WEBEXT_EXTENSION_STARTUP_MS
WEBEXT_STORAGE_LOCAL_GET_MS
WEBEXT_STORAGE_LOCAL_SET_MS

Looks like there's some other expiring probes as well, but those are the ones causing test failures anyway. How do you want to proceed, Bob?

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Also affected:
TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/mochitest/test_chrome_ext_contentscript_telemetry.html | No data recorded for histogram: WEBEXT_CONTENT_SCRIPT_INJECTION_MS
TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_browserAction_telemetry.js | application terminated with exit code 11

Add two more to the list:
WEBEXT_BROWSERACTION_POPUP_OPEN_MS
WEBEXT_BROWSERACTION_POPUP_PRELOAD_RESULT_COUNT

Comment 2

[Bob Silverberg [:bsilverberg]]

Attachment: Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

We are still using the data from these probes to help guide us in the development of
the APIs in question, so would like to keep them in place until version 63.

Review commit: https://reviewboard.mozilla.org/r/208474/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/208474/

Comment 3

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8937762 [details]
Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

Requesting data review for this patch which extends the expiry dates for a number of temporary telemetry probes whose data we are still consuming.

Comment 4

[Bob Silverberg [:bsilverberg]]

We plan to extend the probes, which should address the test issues.

Comment 5

[Andrew Swan [:aswan]]

Comment on attachment 8937762 [details]
Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

https://reviewboard.mozilla.org/r/208474/#review214562

This needs data review, not code review.

Comment 6

[Liz Henry (:lizzard) (relman/hg->git project)]

The early/first merge will be Jan. 11th. 
Tracking so we can make sure to check back in early January. 
Since don't have flags yet for 60, I'm just tracking this for 59.

Comment 7

[Liz Henry (:lizzard) (relman/hg->git project)]

Rebecca, can you cover data review? Or, if not, can you help find someone to cover that? Thanks.

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8937762 [details]
Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

Try confirms that this fixes all issues with these tests.

Comment 9

[Chenxia Liu [:liuche]]

Comment on attachment 8937762 [details]
Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

https://reviewboard.mozilla.org/r/208474/#review216884

Approving probe extensions.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c01ea7a84567
Extend webextension telemetry probes for another 3 releases, r=liuche

Comment 11

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/c01ea7a84567