Closed Bug 1425957 Opened 2 years ago Closed 2 years ago

Various webextension tests are going to permafail when the Gecko version number is bumped to 60

Categories

(WebExtensions :: General, defect, P3, major)

defect

Tracking

(firefox-esr52 unaffected, firefox57 unaffected, firefox58 unaffected, firefox59+ verified)

VERIFIED FIXED
mozilla59
Tracking Status
firefox-esr52 --- unaffected
firefox57 --- unaffected
firefox58 --- unaffected
firefox59 + verified

People

(Reporter: RyanVM, Assigned: bsilverberg)

References

Details

Attachments

(1 file)

[Tracking Requested - why for this release]: Permafailing tests when the Gecko version number is bumped to 60.

Looks like the probes these tests depend on are set to expire when Gecko hits 60:
WEBEXT_BACKGROUND_PAGE_LOAD_MS
WEBEXT_CONTENT_SCRIPT_INJECTION_MS
WEBEXT_EXTENSION_STARTUP_MS
WEBEXT_STORAGE_LOCAL_GET_MS
WEBEXT_STORAGE_LOCAL_SET_MS

Looks like there's some other expiring probes as well, but those are the ones causing test failures anyway. How do you want to proceed, Bob?
Flags: needinfo?(bob.silverberg)
Also affected:
TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/mochitest/test_chrome_ext_contentscript_telemetry.html | No data recorded for histogram: WEBEXT_CONTENT_SCRIPT_INJECTION_MS
TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/test-oop-extensions/browser_ext_browserAction_telemetry.js | application terminated with exit code 11

Add two more to the list:
WEBEXT_BROWSERACTION_POPUP_OPEN_MS
WEBEXT_BROWSERACTION_POPUP_PRELOAD_RESULT_COUNT
Blocks: 1297167
Summary: Various webextension xpcshell tests are going to permafail when the Gecko version number is bumped to 60 → Various webextension tests are going to permafail when the Gecko version number is bumped to 60
Comment on attachment 8937762 [details]
Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

Requesting data review for this patch which extends the expiry dates for a number of temporary telemetry probes whose data we are still consuming.
Attachment #8937762 - Flags: review?(rweiss)
We plan to extend the probes, which should address the test issues.
Flags: needinfo?(bob.silverberg)
Assignee: nobody → bob.silverberg
Comment on attachment 8937762 [details]
Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

https://reviewboard.mozilla.org/r/208474/#review214562

This needs data review, not code review.
Attachment #8937762 - Flags: review?(aswan)
The early/first merge will be Jan. 11th. 
Tracking so we can make sure to check back in early January. 
Since don't have flags yet for 60, I'm just tracking this for 59.
Rebecca, can you cover data review? Or, if not, can you help find someone to cover that? Thanks.
Flags: needinfo?(rweiss)
Priority: -- → P3
Comment on attachment 8937762 [details]
Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

Try confirms that this fixes all issues with these tests.
Attachment #8937762 - Flags: feedback+
Comment on attachment 8937762 [details]
Bug 1425957 - Extend webextension telemetry probes for another 3 releases,

https://reviewboard.mozilla.org/r/208474/#review216884

Approving probe extensions.
Attachment #8937762 - Flags: review+
Flags: needinfo?(rweiss)
Attachment #8937762 - Flags: review?(rweiss)
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/c01ea7a84567
Extend webextension telemetry probes for another 3 releases, r=liuche
https://hg.mozilla.org/mozilla-central/rev/c01ea7a84567
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
Status: RESOLVED → VERIFIED
Product: Toolkit → WebExtensions
You need to log in before you can comment on or make changes to this bug.