Perma beta browser_privacypane_2.js - deleteOnClose should not be disabled - Got true, expected false

VERIFIED FIXED in Firefox 65

Status

()

defect
P1
normal
VERIFIED FIXED
5 months ago
4 months ago

People

(Reporter: CosminS, Assigned: johannh)

Tracking

({intermittent-failure})

unspecified
Firefox 66
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(firefox-esr60 unaffected, firefox64 unaffected, firefox65 verified, firefox66+ verified)

Details

(Whiteboard: [stockwell disable-recommended])

Attachments

(1 attachment)

Reporter

Description

5 months ago
[Tracking Requested - why for this release]:

Central as late Beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception&revision=a30fb730b7143aecb52e1a2e4e237a04190cebbe&selectedJob=218761682&searchStr=browser%2Cchrome

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=218761682&repo=try&lineNumber=10430

[task 2018-12-25T12:25:03.875Z] 12:25:03     INFO - TEST-PASS | browser/components/preferences/in-content/tests/browser_privacypane_2.js | rememberForms should not be disabled - 
[task 2018-12-25T12:25:03.876Z] 12:25:03     INFO - Buffered messages finished
[task 2018-12-25T12:25:03.877Z] 12:25:03     INFO - TEST-UNEXPECTED-FAIL | browser/components/preferences/in-content/tests/browser_privacypane_2.js | deleteOnClose should not be disabled - Got true, expected false
[task 2018-12-25T12:25:03.878Z] 12:25:03     INFO - Stack trace:
[task 2018-12-25T12:25:03.879Z] 12:25:03     INFO - chrome://mochikit/content/browser-test.js:test_is:1316
[task 2018-12-25T12:25:03.880Z] 12:25:03     INFO - chrome://mochitests/content/browser/browser/components/preferences/in-content/tests/privacypane_tests_perwindow.js:expect_disabled/<:76
[task 2018-12-25T12:25:03.881Z] 12:25:03     INFO - chrome://mochitests/content/browser/browser/components/preferences/in-content/tests/privacypane_tests_perwindow.js:expect_disabled:75
[task 2018-12-25T12:25:03.882Z] 12:25:03     INFO - chrome://mochitests/content/browser/browser/components/preferences/in-content/tests/privacypane_tests_perwindow.js:test_dependent_elements:103
[task 2018-12-25T12:25:03.883Z] 12:25:03     INFO - chrome://mochitests/content/browser/browser/components/preferences/in-content/tests/privacypane_tests_perwindow.js:runTestOnPrivacyPrefPane:11

This happens only on late beta simulation and looks to be a regression from Bug 1466817. Paolo could you please take a look over this? Thank you.
Flags: needinfo?(paolo.mozmail)

Comment 1

5 months ago
The only test that was changed in bug 1466817 is browser_spotlight.js, and in the log it is executed _after_ the test that fails here. The production code change seems totally unrelated to what browser_privacypane_2.js is testing. There is a new attribute added in the privacy pane implementation file, but after a quick look it seems strange that this would affect the test.

Cosmin, can you verify that the regression bug you identified is correct?
Flags: needinfo?(paolo.mozmail) → needinfo?(csabou)
Backed out bug 1466817 in beta sim and the failure is not present anymore.

Central as beta simulation:
https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Crunnable&revision=bd4531e9dd487fc05d38aa62b04e0b1011a0bf67
Flags: needinfo?(csabou)

Comment 3

5 months ago
Thanks! I'll check this out in the next few days.
Flags: needinfo?(paolo.mozmail)
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)

Comment 6

5 months ago
Bug 1466817 isn't the correct regression range, as comment 1 originally suggested.

Comment 2 is irrelevant because the backout was done on an early beta simulation (66.0b1) instead of the original late beta (66.0b12), so this test would likely have succeeded even without the backout. I figured this out after spending quite some time for a full build of this changeset.

I've then built revision a30fb730b714 mentioned in comment 0 locally, and verified that browser_privacypane_2.js fails when run together with its entire test folder, but not individually:

  ./mach test browser/components/preferences/in-content/tests/

However, backing out bug 1466817 from that revision and running the tests in the folder again does not fix the failure.

You need to find the correct regression range here, and I suggest double checking the regressing bug with a cherry-picked backout and posting both the changeset that fails and the one that succeeds.
No longer blocks: 1466817
Flags: needinfo?(paolo.mozmail) → needinfo?(csabou)
Sorry for the time spent on this, Paolo.

Late beta simulation with a backout of https://hg.mozilla.org/mozilla-central/rev/724652219657 / bug 1514853 points to that bug (none of the oranges have a failure message for this test): https://treeherder.mozilla.org/#/jobs?repo=try&revision=a45702316af9a507640676f60b2847323afe2d6a

Ehsan, please take a look. This permafails on beta.
Blocks: 1514853
Flags: needinfo?(csabou) → needinfo?(ehsan)
Comment hidden (Intermittent Failures Robot)

Comment 9

5 months ago
Johann/Erica, can you please take this?  I don't have the bandwidth right now... Thanks!
Flags: needinfo?(jhofmann)
Flags: needinfo?(ewright)
Flags: needinfo?(ehsan)
Comment hidden (Intermittent Failures Robot)
Summary: Perma browser_privacypane_2.js - deleteOnClose should not be disabled - Got true, expected false → Perma beta browser_privacypane_2.js - deleteOnClose should not be disabled - Got true, expected false
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
Assignee

Updated

5 months ago
Assignee: nobody → jhofmann
Status: NEW → ASSIGNED
Flags: needinfo?(jhofmann)
Priority: -- → P1
Assignee

Updated

5 months ago
Flags: needinfo?(ewright)
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)

Comment 16

5 months ago
Pushed by jhofmann@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/993c073ab27a
Properly clean up prefs in browser_contentblocking.js. r=ewright

Comment 18

5 months ago
bugherder
Status: ASSIGNED → RESOLVED
Last Resolved: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 66
Comment hidden (Intermittent Failures Robot)
You need to log in before you can comment on or make changes to this bug.