Open Bug 1430530 Opened 7 years ago Updated 2 years ago

Intermittent gfx/tests/mochitest/test_acceleration.html | Advanced Layers enabled on Windows; Direct3D 11:false - didn't expect +0, but got it

Categories

(Core :: Graphics, defect, P3)

Product:
Core
Component:
Graphics
Platform:
Unspecified
Windows 10
Type:
defect

Tracking

()

People

(Reporter: intermittent-bug-filer, Unassigned, NeedInfo)

References

Details

(Keywords: intermittent-failure, leave-open, Whiteboard: [gfx-noted][stockwell disabled])

Attachments

(1 file, 4 obsolete files)

disable_on_Win.patch
614 bytes, patch
jmaher
: review+
Details | Diff | Splinter Review
Filed by: ncsoregi [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=156299928&repo=mozilla-inbound https://queue.taskcluster.net/v1/task/BrCUizC5R9miM0u-5YMHKg/runs/0/artifacts/public/logs/live_backing.log 06:19:13 INFO - 71 INFO TEST-PASS | gfx/tests/mochitest/test_acceleration.html | DirectWrite enabled on Windows 7 or newer 3448 06:19:13 INFO - Buffered messages finished 3449 06:19:13 ERROR - 72 INFO TEST-UNEXPECTED-FAIL | gfx/tests/mochitest/test_acceleration.html | Advanced Layers enabled on Windows; Direct3D 11:false - didn't expect +0, but got it 3450 06:19:13 INFO - SimpleTest.isnot@SimpleTest/SimpleTest.js:324:5 3451 06:19:13 INFO - runTest@gfx/tests/mochitest/test_acceleration.html:135:9 3452 06:19:13 INFO - EventListener.handleEvent*@gfx/tests/mochitest/test_acceleration.html:25:1
Looks like Advanced Layers isn't being enabled for some reason, perhaps a problem initializing D3D11. > 06:19:07 INFO - GECKO(10788) | [GPU 220, Compositor] WARNING: No device or incompatible device.: file z:/build/build/src/gfx/layers/d3d11/TextureD3D11.cpp, line 840
OS: Unspecified → Windows 10
Whiteboard: [gfx-noted]
There are 33 failures in the past week. Platforms: windows10-64 opt/pgo and one occurrence on windows10-64-nightly opt. Recent log failure: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=176866429&lineNumber=2106 Relevant part of the log: 23:48:20 ERROR - 63 INFO TEST-UNEXPECTED-FAIL | gfx/tests/mochitest/test_acceleration.html | Advanced Layers enabled on Windows; Direct3D 11:false - didn't expect +0, but got it 23:48:20 INFO - SimpleTest.isnot@SimpleTest/SimpleTest.js:324:5 23:48:20 INFO - runTest@gfx/tests/mochitest/test_acceleration.html:136:9 23:48:20 INFO - EventListener.handleEvent*@gfx/tests/mochitest/test_acceleration.html:25:1 23:48:20 INFO - 64 INFO TEST-PASS | gfx/tests/mochitest/test_acceleration.html | WebRender disabled on Windows 23:48:20 INFO - GECKO(5248) | MEMORY STAT | vsize 2098644MB | vsizeMaxContiguous 128450632MB | residentFast 76MB | heapAllocated 14MB 23:48:20 INFO - 65 INFO TEST-OK | gfx/tests/mochitest/test_acceleration.html | took 1157ms 23:48:20 INFO - 66 INFO TEST-START | Shutdown 23:48:20 INFO - 67 INFO Passed: 4 23:48:20 WARNING - 68 INFO Failed: 1 23:48:20 WARNING - One or more unittests failed. 23:48:20 INFO - 69 INFO Todo: 0 23:48:20 INFO - 70 INFO Mode: e10s 23:48:20 INFO - 71 INFO Slowest: 1158ms - /tests/gfx/tests/mochitest/test_acceleration.html 23:48:20 INFO - 72 INFO SimpleTest FINISHED 23:48:20 INFO - 73 INFO TEST-INFO | Ran 1 Loops 23:48:20 INFO - 74 INFO SimpleTest FINISHED :milan Can you please take a look at this?
Flags: needinfo?(milaninbugzilla)
I see this at the top:'gpu_required': 'False'; could that mean that we sometimes get an instance that doesn't have the GPU and then things go bad?
Flags: needinfo?(milaninbugzilla) → needinfo?(nerli)
Over the last 7 days there are 43 failures present on this bug. These happen only on windows10-64. Here is the most recent log example: https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=177610744&lineNumber=2114 06:53:53 ERROR - 63 INFO TEST-UNEXPECTED-FAIL | gfx/tests/mochitest/test_acceleration.html | Advanced Layers enabled on Windows; Direct3D 11:false - didn't expect +0, but got it 06:53:53 INFO - SimpleTest.isnot@SimpleTest/SimpleTest.js:324:5 06:53:53 INFO - runTest@gfx/tests/mochitest/test_acceleration.html:136:9 06:53:53 INFO - EventListener.handleEvent*@gfx/tests/mochitest/test_acceleration.html:25:1 06:53:53 INFO - 64 INFO TEST-PASS | gfx/tests/mochitest/test_acceleration.html | WebRender disabled on Windows 06:53:53 INFO - GECKO(5452) | MEMORY STAT | vsize 2098643MB | vsizeMaxContiguous 129138263MB | residentFast 76MB | heapAllocated 14MB 06:53:53 INFO - 65 INFO TEST-OK | gfx/tests/mochitest/test_acceleration.html | took 1180ms 06:53:53 INFO - 66 INFO TEST-START | Shutdown
Flags: needinfo?(dbolter)
Flags: needinfo?(dbolter)
Priority: P3 → P5
Over the last 7 days this bug has 45 failures. These happen on windows7-32, windows10-64-nightly, windows10-64. Here is the most recent log example: https://treeherder.mozilla.org/logviewer.html#?job_id=178891564&repo=mozilla-inbound&lineNumber=2109 00:39:37 INFO - 58 INFO SimpleTest START 00:39:37 INFO - 59 INFO TEST-START | gfx/tests/mochitest/test_acceleration.html 00:39:38 INFO - TEST-INFO | started process screenshot 00:39:38 INFO - TEST-INFO | screenshot: exit 0 00:39:38 INFO - Buffered messages logged at 00:39:38 00:39:38 INFO - 60 INFO TEST-PASS | gfx/tests/mochitest/test_acceleration.html | Acceleration enabled on Windows XP or newer 00:39:38 INFO - 61 INFO TEST-PASS | gfx/tests/mochitest/test_acceleration.html | Direct2D enabled on Windows 8 or newer 00:39:38 INFO - 62 INFO TEST-PASS | gfx/tests/mochitest/test_acceleration.html | DirectWrite enabled on Windows 7 or newer 00:39:38 INFO - Buffered messages finished 00:39:38 ERROR - 63 INFO TEST-UNEXPECTED-FAIL | gfx/tests/mochitest/test_acceleration.html | Advanced Layers enabled on Windows; Direct3D 11:false - didn't expect +0, but got it 00:39:38 INFO - SimpleTest.isnot@SimpleTest/SimpleTest.js:324:5 00:39:38 INFO - runTest@gfx/tests/mochitest/test_acceleration.html:136:9 00:39:38 INFO - EventListener.handleEvent*@gfx/tests/mochitest/test_acceleration.html:25:1 00:39:38 INFO - 64 INFO TEST-PASS | gfx/tests/mochitest/test_acceleration.html | WebRender disabled on Windows 00:39:38 INFO - GECKO(6708) | MEMORY STAT | vsize 2098643MB | vsizeMaxContiguous 129965714MB | residentFast 76MB | heapAllocated 14MB 00:39:38 INFO - 65 INFO TEST-OK | gfx/tests/mochitest/test_acceleration.html | took 1162ms 00:39:38 INFO - 66 INFO TEST-START | Shutdown
Flags: needinfo?(dbolter)
I'm curious about Milan's comment 14. Could this be related to our test infra being flaky about reporting a GPU?
Flags: needinfo?(dbolter) → needinfo?(bas)
Hi guys! There are 38 failures in the last 7 days. Mostly on windows10-64, only 1 on windows7-32. Do you have any updates on this, should we go ahead with disabling the test?
Flags: needinfo?(dbolter)
(In reply to Narcis Beleuzu [:NarcisB] from comment #22) > Hi guys! > There are 38 failures in the last 7 days. Mostly on windows10-64, only 1 on > windows7-32. > Do you have any updates on this, should we go ahead with disabling the test? This test is pretty important, we've had people accidentally 'switch off' acceleration on automation in the past, and that's why we introduced it. Disabling is probably risky. (In reply to David Bolter [:davidb] (NeedInfo me for attention) from comment #19) > I'm curious about Milan's comment 14. Could this be related to our test > infra being flaky about reporting a GPU? It's either that, or somehow some of our test hardware isn't getting acceleration, I wonder whether the drivers our testing hardware is running are inconsistent, and whether we've recently perhaps blacklisted a driver used on some 'subset' of the machines. This seems to be only reporting a failure to get Advanced Layers, now we recently did blocklist an NVidia driver range for advanced layers specifically. I can't seem to find a device ID or driver version for https://perf-html.io/public/b7050df4af0bb9e4e093d1eae1886b371df22b5c/calltree/?range=12.8214_13.2717&thread=6&threadOrder=0-2-3-4-6-1-5&v=3. Nor am I sure how to get there, but something tells me it might be in the blacklisted range!
Flags: needinfo?(bas)
DBurns do you know who can look into comment 23 regarding gfx driver versions on test hardware?
Flags: needinfo?(dbolter) → needinfo?(dburns)
:fubar, can you help figure out what graphics drivers we have on windows 10. These are GPU instance types, specifically g2.2xlarge. on another note, the failures show up on try- why are people annotating failures on try? we should remove that from our bugzilla magic reports.
Flags: needinfo?(dburns) → needinfo?(klibby)
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #25) > why are people annotating > failures on try? I annotate most of my try pushes, it's the best way I've found to determine if an orange-laden push is just full of intermittents or actually exposing bugs in the patches.
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #25) > :fubar, can you help figure out what graphics drivers we have on windows 10. > These are GPU instance types, specifically g2.2xlarge. Swapping NI to markco to dig that info up.
Flags: needinfo?(klibby) → needinfo?(mcornmesser)
Nvidia 369.95-quadro-grid-desktop-notebook-win10-64bit
Flags: needinfo?(mcornmesser)
Looks like we're not blacklisting Windows 10 anyway, but even if we were, that driver version is not in the blacklist. As far as I can tell the only option we'd have here is add more annotation to the Advanced Layers decision making process and seeing why on some runs it decides not to enable it.
Flags: needinfo?(jmaher)
Attachment #8981417 - Flags: review?(jmaher)
Comment on attachment 8981417 [details] [diff] [review] Skipped test_acceleration.html for frequent failures on win10 !debug Review of attachment 8981417 [details] [diff] [review]: ----------------------------------------------------------------- ::: gfx/tests/mochitest/mochitest.ini @@ +5,2 @@ > subsuite = gpu > fail-if = (os == "win" && os_version == "5.1" && e10s) # Bug 1253862 can you remove this fail-if line since you are in here? 1) I don't know if skip-if and fail-if can coexist in the same test. 2) we don't run winxp (os_version == "5.1") anymore
Attachment #8981417 - Flags: review?(jmaher) → review-
Flags: needinfo?(jmaher)
Attachment #8981417 - Attachment is obsolete: true
Attachment #8981435 - Flags: review?(jmaher)
Comment on attachment 8981435 [details] [diff] [review] Skipped test_acceleration.html for frequent failures on win10 !debug Review of attachment 8981435 [details] [diff] [review]: ----------------------------------------------------------------- sorry for the next r-, each line is important! ::: gfx/tests/mochitest/mochitest.ini @@ -2,5 @@ > > [test_acceleration.html] > skip-if = (os == 'win' && os_version == '10.0' && !debug) # Bug 1430530 > -subsuite = gpu > -fail-if = (os == "win" && os_version == "5.1" && e10s) # Bug 1253862 we still need |subsuite = gpu|
Attachment #8981435 - Flags: review?(jmaher) → review-
Attachment #8981435 - Attachment is obsolete: true
Attachment #8981439 - Flags: review?(jmaher)
Attachment #8981439 - Flags: review?(jmaher)
Attachment #8981439 - Attachment is obsolete: true
Attachment #8981475 - Flags: review?(jmaher)
Comment on attachment 8981475 [details] [diff] [review] Skipped test_acceleration.html for frequent failures on win10 !debug Review of attachment 8981475 [details] [diff] [review]: ----------------------------------------------------------------- excellent
Attachment #8981475 - Flags: review?(jmaher) → review+
Keywords: checkin-needed, leave-open
Whiteboard: [gfx-noted][stockwell disable-recommended] → [gfx-noted][stockwell disabled]
Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/4e1b975f4897 Disable test_acceleration.html for frequent failures on windows 10 !debug. r=jmaher
Keywords: checkin-needed
Flags: needinfo?(nerli)
There are 110 failures in the last 7 days. All of them are on Win7 & Win10 - debug Latest log: https://treeherder.mozilla.org/logviewer.html#?job_id=194658379&repo=mozilla-inbound&lineNumber=2954 03:53:47 INFO - TEST-START | gfx/tests/mochitest/test_acceleration.html 03:53:47 INFO - GECKO(1172) | ++DOMWINDOW == 7 (0000025831C9B400) [pid = 10396] [serial = 7] [outer = 0000025834020000] 03:53:48 INFO - GECKO(1172) | ++DOMWINDOW == 8 (00000258340A6000) [pid = 10396] [serial = 8] [outer = 0000025834020000] 03:53:48 INFO - GECKO(1172) | [Child 10396, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file z:/build/build/src/xpcom/base/nsSystemInfo.cpp, line 119 03:53:48 INFO - TEST-INFO | started process screenshot 03:53:48 INFO - TEST-INFO | screenshot: exit 0 03:53:48 INFO - Buffered messages logged at 03:53:48 03:53:48 INFO - TEST-PASS | gfx/tests/mochitest/test_acceleration.html | Acceleration enabled on Windows XP or newer 03:53:48 INFO - TEST-PASS | gfx/tests/mochitest/test_acceleration.html | Direct2D enabled on Windows 8 or newer 03:53:48 INFO - TEST-PASS | gfx/tests/mochitest/test_acceleration.html | DirectWrite enabled on Windows 7 or newer 03:53:48 INFO - Buffered messages finished 03:53:48 INFO - TEST-UNEXPECTED-FAIL | gfx/tests/mochitest/test_acceleration.html | Advanced Layers enabled on Windows; Direct3D 11:false - didn't expect +0, but got it 03:53:48 INFO - SimpleTest.isnot@SimpleTest/SimpleTest.js:324:5 03:53:48 INFO - runTest@gfx/tests/mochitest/test_acceleration.html:135:9 03:53:48 INFO - EventListener.handleEvent*@gfx/tests/mochitest/test_acceleration.html:25:1 03:53:48 INFO - TEST-PASS | gfx/tests/mochitest/test_acceleration.html | WebRender disabled on Windows 03:53:48 INFO - GECKO(1172) | MEMORY STAT | vsize 1533MB | vsizeMaxContiguous 131720051MB | residentFast 85MB | heapAllocated 12MB 03:53:48 INFO - TEST-OK | gfx/tests/mochitest/test_acceleration.html | took 1407ms
Flags: needinfo?(dbolter)
Whiteboard: [gfx-noted][stockwell disabled] → [gfx-noted][stockwell disabled][stockwell needswork:owner]
Due to the high frequency of failures let's disable this on all windows.
Attachment #8981475 - Attachment is obsolete: true
Attachment #9003146 - Flags: review?(jmaher)
Comment on attachment 9003146 [details] [diff] [review] disable_on_Win.patch Review of attachment 9003146 [details] [diff] [review]: ----------------------------------------------------------------- Thanks Cosmin
Attachment #9003146 - Flags: review?(jmaher) → review+
Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/349025ae0ac8 Disable gfx/tests/mochitest/test_acceleration.html on windows for frequent failures. r=jmaher
Jeff should we worry about the tests being disabled?
Flags: needinfo?(dbolter) → needinfo?(jmuizelaar)
Yes. We should worry about this. We want this to in place to ensure that we're testing the right things. Someone should probably investigate.
Flags: needinfo?(jmuizelaar) → needinfo?(dbolter)
NI Matt for awareness. It feels like we have higher priorities but tests are important.
Flags: needinfo?(dbolter) → needinfo?(matt.woodrow)
Whiteboard: [gfx-noted][stockwell disabled][stockwell needswork:owner] → [gfx-noted][stockwell disabled]
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: