1499363 - Perma X failures in xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_userScripts.js when Gecko 64 merges to Beta on 2018-10-15

Status: VERIFIED FIXED

(WebExtensions :: General, defect)

Description

[Eliza Balazs [:ebalazs_]]

Central as Beta:
https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed,busted,exception,retry,runnable&revision=98cb87e22c2cefda4c6750727b5b00e2be35e8b6&selectedJob=205799375

Failure log:
https://treeherder.mozilla.org/logviewer.html#?job_id=205785981&repo=try&lineNumber=1670
https://treeherder.mozilla.org/logviewer.html#?job_id=205801414&repo=try&lineNumber=2373

robwu: Hi, This is a regression from https://bugzilla.mozilla.org/show_bug.cgi?id=1491397. Can you have a look? Thanks!

Comment 1

[Rob Wu [:robwu]]

Thanks for the report.

The test failure is because this feature is disabled by default on non-Nightly.
At the start of the test file, the preference is flipped to enable the feature.
At the end of the test (before my new test), the runWithPrefs method is used to temporarily toggle the pref. This helper function does however not account for the original value; it simply clears the preference instead of pushing/popping the stack of overridden prefs.

I intend to fix this by changing the the test and implementation of runWithPrefs to use SpecialPowers.pushPrefEnv and SpecialPowers.popPrefEnv instead of the current Services.prefs.setBoolPref + Services.prefs.clearUserPref.

Comment 2

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: Bug 1499363 - Fix userScript test failure on beta due to runWithPrefs not restoring the initial prefs values. r?robwu!

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/99d43f9d3918
Fix userScript test failure on beta due to runWithPrefs not restoring the initial prefs values. r=robwu

Comment 4

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/99d43f9d3918

Comment 5

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Verified fixed with today's beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Crunnable&revision=f7ac7e6ce3518f18d06ad07ee080d219dc16cf33&selectedJob=206668654

Comment 6

[Intermittent Failures Robot]

8 failures in 2759 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* try: 8

Platform breakdown:
* linux64-qr: 1
* linux64: 3
* macosx64-devedition: 1
* android-em-4-2-x86: 2
* linux32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1499363&startday=2018-10-15&endday=2018-10-21&tree=all