Intermittent accessible/tests/browser/windows/ia2/browser_osPicker.js | Uncaught exception in test bound - at chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:962 - Error: Traceback (most recent call last):
Categories
(Core :: Disability Access APIs, defect, P5)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox137 | --- | fixed |
People
(Reporter: intermittent-bug-filer, Assigned: Gijs)
References
(Blocks 1 open bug)
Details
(Keywords: intermittent-failure, test-verify-fail)
Attachments
(1 file)
Filed by: abutkovits [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=450669060&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bJqY1lr6SEuRUTh8R5v-ew/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bJqY1lr6SEuRUTh8R5v-ew/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1
[task 2024-03-13T22:53:29.996Z] 22:53:29 INFO - TEST-PASS | accessible/tests/browser/windows/ia2/browser_osPicker.js | Actually remote browser -
[task 2024-03-13T22:53:29.997Z] 22:53:29 INFO - Focusing file input
[task 2024-03-13T22:53:29.997Z] 22:53:29 INFO - Buffered messages finished
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - TEST-UNEXPECTED-FAIL | accessible/tests/browser/windows/ia2/browser_osPicker.js | Uncaught exception in test bound - at chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:962 - Error: Traceback (most recent call last):
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - File "Z:\task_171036748209443\build\tests\mochitest\browser\accessible\tests\browser\windows\a11y_setup.py", line 229, in wait
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - ctypes.oledll.ole32.CoWaitForMultipleHandles(
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - File "_ctypes/callproc.c", line 1000, in GetResult
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - OSError: [WinError -2147417835] OLE has sent a request and is waiting for a reply
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO -
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - During handling of the above exception, another exception occurred:
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO -
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - Traceback (most recent call last):
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - File "<string>", line 85, in web_socket_transfer_data
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - File "<string>", line 3, in run
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - File "Z:\task_171036748209443\build\tests\mochitest\browser\accessible\tests\browser\windows\a11y_setup.py", line 234, in wait
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - raise TimeoutError("Timeout before desired event received")
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - TimeoutError: Timeout before desired event received
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO -
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - Stack trace:
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - runPython/</gPythonSocket.onmessage@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:962:16
[task 2024-03-13T22:53:30.000Z] 22:53:30 INFO - EventHandlerNonNull*runPython/<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:955:5
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - runPython@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:954:10
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - @chrome://mochitests/content/browser/accessible/tests/browser/windows/ia2/browser_osPicker.js:17:11
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - async*accessibleTask/</<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:554:15
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - async*withNewTab@resource://testing-common/BrowserTestUtils.sys.mjs:146:22
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - async*accessibleTask/<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:473:28
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - handleTask@chrome://mochikit/content/browser-test.js:1139:26
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - _runTaskBasedTest@chrome://mochikit/content/browser-test.js:1211:18
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:1353:14
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - nextTest/<@chrome://mochikit/content/browser-test.js:1122:14
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - SimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13
[task 2024-03-13T22:53:30.001Z] 22:53:30 INFO - Leaving test bound
[task 2024-03-13T22:53:30.295Z] 22:53:30 INFO - GECKO(876) | MEMORY STAT | vsize 2104318MB | vsizeMaxContiguous 67330064MB | residentFast 301MB | heapAllocated 105MB
[task 2024-03-13T22:53:30.296Z] 22:53:30 INFO - TEST-OK | accessible/tests/browser/windows/ia2/browser_osPicker.js | took 10918ms
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
||
Comment 4•8 months ago
|
||
Tier 1 failure here
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
Assignee | |
Comment 9•1 month ago
|
||
Happy new year! AFAICT all ia2 tests are almost-permafailing on Windows 11 artifact builds with:
[task 2025-01-16T17:47:04.409Z] 17:47:04 INFO - TEST-UNEXPECTED-FAIL | accessible/tests/browser/windows/ia2/browser_groupPosition.js | Uncaught exception in test bound testGroupPosition - at chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:1034 - Error: Traceback (most recent call last):
[task 2025-01-16T17:47:04.409Z] 17:47:04 INFO - File "<string>", line 42, in web_socket_transfer_data
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - File "D:\task_173704572874916\build\tests\mochitest\browser\accessible\tests\browser\windows\a11y_setup.py", line 50, in <module>
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - ia2Mod = comtypes.client.GetModule(ia2Tlb)
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - File "D:\task_173704572874916\build\venv\Lib\site-packages\comtypes\client\_generate.py", line 118, in GetModule
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - tlib = _load_tlib(pathname) # don't register
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - ^^^^^^^^^^^^^^^^^^^^
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - File "D:\task_173704572874916\build\venv\Lib\site-packages\comtypes\client\_generate.py", line 150, in _load_tlib
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - return typeinfo.LoadTypeLibEx(obj)
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - File "D:\task_173704572874916\build\venv\Lib\site-packages\comtypes\typeinfo.py", line 523, in LoadTypeLibEx
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - _oleaut32.LoadTypeLibEx(c_wchar_p(szFile), regkind, byref(ptl))
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - File "_ctypes/callproc.c", line 1000, in GetResult
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - OSError: [WinError -2147312566] Error loading type library/DLL
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO -
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - Stack trace:
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - runPython/</gPythonSocket.onmessage@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:1034:16
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - EventHandlerNonNull*runPython/<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:1027:5
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - runPython@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:1026:10
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - testGroupPosition@chrome://mochitests/content/browser/accessible/tests/browser/windows/ia2/browser_groupPosition.js:22:21
[task 2025-01-16T17:47:04.410Z] 17:47:04 INFO - accessibleTask/wrapped/<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:602:15
[task 2025-01-16T17:47:04.411Z] 17:47:04 INFO - async*withNewTab@resource://testing-common/BrowserTestUtils.sys.mjs:121:22
[task 2025-01-16T17:47:04.411Z] 17:47:04 INFO - async*wrapped@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:521:28
[task 2025-01-16T17:47:04.411Z] 17:47:04 INFO - handleTask@chrome://mochikit/content/browser-test.js:1147:26
[task 2025-01-16T17:47:04.411Z] 17:47:04 INFO - _runTaskBasedTest@chrome://mochikit/content/browser-test.js:1219:18
[task 2025-01-16T17:47:04.411Z] 17:47:04 INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:1360:14
[task 2025-01-16T17:47:04.411Z] 17:47:04 INFO - nextTest/<@chrome://mochikit/content/browser-test.js:1136:14
[task 2025-01-16T17:47:04.411Z] 17:47:04 INFO - SimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13
e.g. this log.
and here's 4 recent trypushes, and for 3 of them the jobs are orange - https://treeherder.mozilla.org/jobs?repo=try&author=gijskruitbosch%40gmail.com&test_paths=accessible%2Ftests%2Fbrowser%2Fwindows%2Fia2&fromchange=8bcef4dd633466bbf2331eb8e53f5b8c4a076d7d&tochange=fe276d2c539c4f3e665157f40449195e5829c994 - and I think only 1 of those broke the browser badly enough for it to be the patch's fault 😉 . Maybe some machines are broken and don't have the requisite libraries? Or something?
Is this on the a11y team's radar and/or expected? Is there something we can do to avoid the noise on artifact trypushes?
|
||
Comment 10•1 month ago
•
|
||
thanks for the tag, I don't think this was on our radar 😳 I don't have the expertise to know what kind of failures we're dealing with here, but I'll bring this back to the team. Jamie is on PTO until the 27th, and I think this python code is his specialty
:nlapre do you have any idea what might be going on here?
|
|
||
Updated•1 month ago
|
|
||
Comment 11•1 month ago
|
||
Indeed, Jamie added this drawing on his extensive experience using the comtypes library in NVDA. I don't understand it anywhere near as well, but it sure looks to me like the path we're building does not point to a valid ia2Typelib.tlb file, which we appear to expect in the current working directory if on CI. That it fails on artifact builds specifically, and on most runs but not all(?) makes me think that some machines do have the ia2Typelib.tld in place and others don't.
I don't think it's a case of not having the right Python library since the stack does show the code calling into the comtypes lib. I think browser_groupPosition.js is implicated here only because it's the first test file listed in the browser.toml.
I don't know how ia2Typelib.tlb is meant to get onto the CI machines, and I don't understand what aspect of artifact builds might affect its presence. Is there a way for mere mortals to inspect the state of the failing / succeeding machines?
|
||
Comment 12•28 days ago
|
||
On CI, we don't have access to the obj dir when tests are run. That's why we need a different path for the tlb for local vs CI.
For CI, we copy the ia2 typelib into the archive used for tests so that it ends up in the test (current) directory. That leads to the question: why isn't this file being included in the test archive for artifact builds? I don't really know enough about the test archive stuff to answer that. I would NI :jmaher for any thoughts here, but it seems he has NIs blocked right now. Maybe on PTO?
|
|
Assignee | |
Comment 13•24 days ago
|
||
(In reply to James Teh [:Jamie] from comment #12)
On CI, we don't have access to the obj dir when tests are run. That's why we need a different path for the tlb for local vs CI.
For CI, we copy the ia2 typelib into the archive used for tests so that it ends up in the test (current) directory. That leads to the question: why isn't this file being included in the test archive for artifact builds? I don't really know enough about the test archive stuff to answer that. I would NI :jmaher for any thoughts here, but it seems he has NIs blocked right now. Maybe on PTO?
--> jmaher?
|
||
Comment 14•24 days ago
|
||
hey, I am back, thanks for the mention :gijs. I did check a full build and IA2Typelib.tlb is in the target.mochitests.tests.tar.gz file. Looking at an artifact build, the file isn't there. I am not 100% sure why, but here are some thoughts:
- on an artifact build we don't have
accessible\interfaces\ia2\IA2Typelib.tlbin the objdir ?- I think this is the case, on my local win11 machine, I do artifacts locally and I do not have
{objdir}/accessible/interfaces/ia2/
- I think this is the case, on my local win11 machine, I do artifacts locally and I do not have
- possibly we need to include this file elsewhere. I suspect not, but a random thought.
is there some magic in the artifact builds that download <something>.dll and assume all work in accessible\interfaces is done?
|
|
||
Comment 15•24 days ago
|
||
Ah, thinking about it a bit more, I guess this makes some sense. The tlb isn't necessary for Firefox to actually run, so it wouldn't be considered a necessary artifact. I don't know much about artifact builds, though, so I'm not sure how to teach the build system that this artifact is necessary and should be included as an artifact. I'll look into it more.
|
|
||
Comment 16•24 days ago
|
||
:glandium, do you know much about artifact builds and how to ensure certain files/directories are processed.
Some parts of Firefox that aren't produced in artifact builds need to be (or can be) copied wholesale from the build providing the artifacts. We have a few different examples kicking around. It seems like this .tlb is part of the Firefox package, so it sounds like we need to understand why https://searchfox.org/mozilla-central/rev/c5432a86ece2ce8671e7aefbe43fed9a10151227/python/mozbuild/mozbuild/artifacts.py#799 isn't enough. I have a guess: https://searchfox.org/mozilla-central/rev/c5432a86ece2ce8671e7aefbe43fed9a10151227/python/mozbuild/mozbuild/action/test_archive.py#291-297 is getting this from the object directory, but the artifact build doesn't put it there; it puts it somewhere under dist/. If you ensured that this TLB was installed to and consumed from dist/..., I think this might Just Work. (You probalby don't want it to be in the object directory for artifact builds because the build system won't necessarily know how to build it.)
If it's not enough, "extra archives" accumulate build outputs that are not part of the Firefox package and are only produced for artifact builds to consume. See https://searchfox.org/mozilla-central/rev/c5432a86ece2ce8671e7aefbe43fed9a10151227/python/mozbuild/mozbuild/artifacts.py#139-143 and it's consumers.
|
|
||
Comment 18•21 days ago
|
||
Thanks :nalexander. That's really helpful.
If I understand correctly after reading this documentation, the archive used for artifact builds is the same archive created by mach package, which in turn (I think?) is the same zip file consumed by users when they download a Firefox zip archive instead of an installer (from try or archive.mozilla.org). While it's not a big deal, I don't think we want these archives (or the installer) to include ia2Typelib.tlb because users don't need that at runtime. This is only needed for tests.
I guess that leaves us with extra archives, but it seems intense to have an extra archive just for this file. It also seems there might be an archive for artifacts just for tests, but I can't quite fathom that yet. That gets stuff from obj/dist/bin which doesn't appear in the normal package zip, which would be better for this purpose if I can figure it out.
|
|
||
Comment 19•21 days ago
|
||
The other option would be to just disable these tests for artifact builds. Is there a way to do that? The tests in accessible/tests/browser/{windows,mac,atk} are specifically platform tests; i.e. you can't change their result without changing platform code. Therefore, there's not much point in even running them for artifact builds. Is there some way to disable tests for artifact builds in the test manifests?
|
|
Assignee | |
Comment 20•20 days ago
|
||
(In reply to James Teh [:Jamie] from comment #19)
The other option would be to just disable these tests for artifact builds. Is there a way to do that? The tests in accessible/tests/browser/{windows,mac,atk} are specifically platform tests; i.e. you can't change their result without changing platform code. Therefore, there's not much point in even running them for artifact builds. Is there some way to disable tests for artifact builds in the test manifests?
Yes, skip-if = ["artifact"] should work (obvs. combined with other conditions as necessary), cf here and here
|
|
Assignee | |
Comment 21•20 days ago
|
||
|
||
Updated•20 days ago
|
|
|
||
Updated•20 days ago
|
|
||
Comment 22•20 days ago
|
||
|
||
Comment 23•20 days ago
|
||
| bugherder | ||
|
||
Updated•20 days ago
|
Description
•