Open Bug 1456807 Opened 2 years ago Updated 5 months ago

Intermittent browser/modules/test/browser/browser_taskbar_preview.js | Win7Features available when it should be - Got false, expected true

Categories

(Firefox :: Shell Integration, defect, P3)

defect

Tracking

()

REOPENED
Tracking Status
firefox61 --- disabled
firefox62 --- disabled

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure, leave-open, Whiteboard: [stockwell disabled])

Attachments

(3 files)

Filed by: ncsoregi [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=175511012&repo=autoland

https://queue.taskcluster.net/v1/task/V7h33H1fQoaz9xp9jSPSXQ/runs/0/artifacts/public/logs/live_backing.log

11:04:18     INFO -  947 INFO TEST-START | browser/modules/test/browser/browser_taskbar_preview.js
11:04:18     INFO -  TEST-INFO | started process screenshot
11:04:18     INFO -  TEST-INFO | screenshot: exit 0
11:04:18    ERROR -  948 INFO TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_taskbar_preview.js | Win7Features available when it should be - Got false, expected true
11:04:18     INFO -  Stack trace:
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:test_is:1285
11:04:18     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser/browser_taskbar_preview.js:test:10
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1109
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:970
11:04:18     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
11:04:18     INFO -  Not taking screenshot here: see the one that was previously logged
11:04:18    ERROR -  949 INFO TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_taskbar_preview.js | Got the expected number of windows - Got 1, expected 0
11:04:18     INFO -  Stack trace:
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:test_is:1285
11:04:18     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser/browser_taskbar_preview.js:test:24
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1109
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:970
11:04:18     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
11:04:18     INFO -  950 INFO TEST-PASS | browser/modules/test/browser/browser_taskbar_preview.js | Browser has expected number of tabs - Browser starts with one preview -
11:04:18     INFO -  Not taking screenshot here: see the one that was previously logged
11:04:18    ERROR -  951 INFO TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_taskbar_preview.js | Browser has one preview per tab - Browser starts with one preview - Got 0, expected 1
11:04:18     INFO -  Stack trace:
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:test_is:1285
11:04:18     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser/browser_taskbar_preview.js:checkPreviews:89
11:04:18     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser/browser_taskbar_preview.js:test:26
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1109
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:970
11:04:18     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
11:04:18     INFO -  Not taking screenshot here: see the one that was previously logged
11:04:18    ERROR -  952 INFO TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_taskbar_preview.js | Browser starts with one preview - Got 0, expected 1
11:04:18     INFO -  Stack trace:
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:test_is:1285
11:04:18     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser/browser_taskbar_preview.js:checkPreviews:90
11:04:18     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser/browser_taskbar_preview.js:test:26
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1109
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:970
11:04:18     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
11:04:18     INFO -  953 INFO TEST-PASS | browser/modules/test/browser/browser_taskbar_preview.js | Browser has expected number of tabs - Correct number of previews after adding -
11:04:18     INFO -  Not taking screenshot here: see the one that was previously logged
11:04:18    ERROR -  954 INFO TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_taskbar_preview.js | Browser has one preview per tab - Correct number of previews after adding - Got 0, expected 4
The first failure occurred in a job which started at 2018-04-25 10:51 UTC, but retrigggers from yesterday already failed intermittently for pushes on 2018-04-23: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=1cfb6834f6564175eadd9c935ac625b30316c8a9&filter-searchStr=windows10%20m-e10s%20browser-chrome

OpenCloudConfig only lists the Mercurial update from 4.3.3 to 4.5.3 as difference.

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=175511012&repo=autoland&lineNumber=4136

11:04:18    ERROR -  948 INFO TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_taskbar_preview.js | Win7Features available when it should be - Got false, expected true
11:04:18     INFO -  Stack trace:
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:test_is:1285
11:04:18     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser/browser_taskbar_preview.js:test:10
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1109
11:04:18     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:970
11:04:18     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795

https://dxr.mozilla.org/mozilla-central/rev/8b2c1fc3d6c348f053fe33a478fa3b1ddb5eb8a6/browser/modules/test/browser/browser_taskbar_preview.js#10
> is(!!Win7Features, isWin7OrHigher, "Win7Features available when it should be");

That means we are returning here: https://dxr.mozilla.org/mozilla-central/rev/8b2c1fc3d6c348f053fe33a478fa3b1ddb5eb8a6/browser/base/content/browser.js#222
and failing nsIWinTaskbar.available https://dxr.mozilla.org/mozilla-central/rev/8b2c1fc3d6c348f053fe33a478fa3b1ddb5eb8a6/browser/base/content/browser.js#211
https://dxr.mozilla.org/mozilla-central/rev/8b2c1fc3d6c348f053fe33a478fa3b1ddb5eb8a6/widget/windows/WinTaskbar.cpp#315
See Also: → 1457329
Hi! I continued with the investigation and this is what I found; Worked on this range http://tinyurl.com/y8w9md4b and the push most likely that caused this is the one from dao http://tinyurl.com/ybbwhmrb. Before that we have 4 pushes with around 20 jobs on which this fail does not appear. So I would say this is where it started https://hg.mozilla.org/integration/autoland/rev/f8cac1a891b5a7343b4ae359d4a987a58f025cda
It fails around 2-3 times from 20 jobs. 

I don't see a direct connection between the push and the failure but maybe on this one Dao can shed some light. Thank you.
Flags: needinfo?(csabou) → needinfo?(dao+bmo)
That's because the test changed chunks. The last push which run all browser-chrome chunks for Windows 10 x64 opt had it in bc6: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=fa8202661c163cf890873b2d6979db1a6917bebc&filter-searchStr=windows10%20x64%20opt%20browser-chrome

The push before dao's has it missing from bc1.
Thanks Aryx, you are right. Back to the rettriggers. Will come with new info.
Flags: needinfo?(dao+bmo)
After spending some time on this one I reached this resolution http://tinyurl.com/yderg6nw 
After this push from masayuki http://tinyurl.com/ycsmod5r there are three pushes (with 30 jobs) on which this test ran and did not fail with this error. This moves between chunks (bc1, bc6, bc5), I verified and the test ran on those pushes.
  
At the recommendation from Aryx I need info Jim to take a look at this. Thank you.
Flags: needinfo?(jmathies)
will take a look.
Flags: needinfo?(jmathies)
Priority: P5 → P3
Attached patch Disable testSplinter Review
Attachment #8975230 - Flags: review?(csabou)
Attachment #8975230 - Flags: review?(cbrindusan)
Comment on attachment 8975230 [details] [diff] [review]
Disable test

Review of attachment 8975230 [details] [diff] [review]:
-----------------------------------------------------------------

I'm ok with the patch. The only thing that concerns me is this would disable the test on all platforms and I don't know how important this test is even if it has a high failure rate. Joel, do you think it's ok to disable this everywhere? It has failure rate of about 11-15 fails a day in the last 6 days.
Attachment #8975230 - Flags: review?(csabou) → review+
Flags: needinfo?(jmaher)
at first I was concerned, but now I am less concerned.  This test only runs on windows (win7 and win10), and it is failing on win10.

I think the best bet is to:
skip-if = os != win || (os == win && bits == 64)


that would leave it running on win7.
Flags: needinfo?(jmaher)
Makes sense. Attached the modified patch.
Thanks !
Attachment #8975236 - Flags: review?(jmaher)
Comment on attachment 8975236 [details] [diff] [review]
Disable test on Win10-64

Review of attachment 8975236 [details] [diff] [review]:
-----------------------------------------------------------------

one nit, I don't see a hg username in the patch- it isn't critical, but nice to have your name associated with a patch.
Attachment #8975236 - Flags: review?(jmaher) → review+
Attachment #8975230 - Flags: review?(cbrindusan) → review+
Pushed by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2266807e2c54
Disable browser_taskbar_preview.js for frequent failures. r=jmaher
Pushed by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/72418bf8c10d
Disable browser_taskbar_preview.js for frequent failures. r=jmaher
https://hg.mozilla.org/mozilla-central/rev/2266807e2c54
https://hg.mozilla.org/mozilla-central/rev/72418bf8c10d
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
Target Milestone: --- → Firefox 62
https://hg.mozilla.org/releases/mozilla-beta/rev/d09efd7ece67
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: FIXED → ---
Target Milestone: Firefox 62 → ---

This appears to be skipped on all platforms now, including Windows 7:

$ ./mach test-info tests browser_taskbar_preview.js
===== browser_taskbar_preview.js =====
Found browser/modules/test/browser/browser_taskbar_preview.js in source control.
 /home/gbrown/objdirs/firefox/backend.TestManifestBackend is out of date with respect to /home/gbrown/objdirs/firefox/config.status
Test configuration changed. Regenerating backend.
browser/modules/test/browser/browser_taskbar_preview.js found in manifest browser/modules/test/browser/browser.ini
  flavor: browser-chrome
  skip-if: os != win || (os == win && bits == 64)

Bugzilla quick search for 'browser/modules/test/browser/browser_taskbar_preview.js,browser_taskbar_preview.js':
Bug 1456807: Intermittent browser/modules/test/browser/browser_taskbar_preview.js | Win7Features available when it should be - Got false, expected true
Querying ActiveData...
Found records matching 'browser/modules/test/browser/browser_taskbar_preview.js' in ActiveData.

Test results for browser/modules/test/browser/browser_taskbar_preview.js on mozilla-central,mozilla-inbound,autoland between 2019-11-05 and 2019-11-12
linux64/asan-opt-e10s:                        0 failures (   161 skipped) in    161 runs
linux64/ccov-debug-e10s:                      0 failures (    19 skipped) in     19 runs
linux64/debug-e10s-service-worker:            0 failures (    21 skipped) in     21 runs
linux64/debug-e10s:                           0 failures (   305 skipped) in    305 runs
linux64/debug-fis:                            0 failures (   368 skipped) in    368 runs
linux64/opt-e10s:                             0 failures (   175 skipped) in    175 runs
linux64/opt-fis:                              0 failures (    28 skipped) in     28 runs
macosx1014-64-shippable/opt-e10s:             0 failures (   303 skipped) in    303 runs
macosx1014-64/debug-e10s:                     0 failures (   497 skipped) in    497 runs
windows10-64-qr/debug-e10s-qr:                0 failures (   475 skipped) in    475 runs
windows10-64-qr/opt-e10s-qr:                  0 failures (    21 skipped) in     21 runs
windows10-64-qr/opt-fis-qr:                   0 failures (    21 skipped) in     21 runs
windows10-64-shippable-qr/opt-e10s-qr:        0 failures (   156 skipped) in    156 runs
windows10-64-shippable/opt-e10s:              0 failures (   165 skipped) in    165 runs
windows10-64/asan-opt-e10s:                   0 failures (   161 skipped) in    161 runs
windows10-64/ccov-debug-e10s:                 0 failures (    18 skipped) in     18 runs
windows10-64/debug-e10s:                      0 failures (   510 skipped) in    510 runs
windows10-64/opt-e10s:                        0 failures (    20 skipped) in     20 runs
windows10-64/opt-fis:                         0 failures (    21 skipped) in     21 runs
windows10-aarch64/opt-e10s:                   0 failures (    17 skipped) in     17 runs
windows7-32-shippable/opt-e10s:               0 failures (   158 skipped) in    158 runs
windows7-32/debug-e10s:                       0 failures (   158 skipped) in    158 runs
windows7-32/opt-e10s:                         0 failures (    20 skipped) in     20 runs

Total: 0 failures in 3798 runs or 0.000 failures/run

I think instead of:

skip-if: os != win || (os == win && bits == 64)

we probably want:

skip-if: os != "win" || (os == "win" && bits == 64)

(quotes around "win").

:NarcisB - Can you confirm and fix?

Flags: needinfo?(nbeleuzu)
Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b03fcaeaca87
Add quotes to the skip-if syntax. r=gbrown
Flags: needinfo?(nbeleuzu)
You need to log in before you can comment on or make changes to this bug.