1582972 - Perma tier2 WindowsError: [Error 216] This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: nerli [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=267643991&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/S4ocjo0wSaSuxdZAeP6gTA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-09-20T15:27:37.740Z] 15:27:37 INFO - Calling ['C:\tasks\task_1568988503\build\venv\Scripts\python', '-u', 'C:\tasks\task_1568988503\build\tests\xpcshell\runxpcshelltests.py', '--total-chunks', '3', '--this-chunk', '1', '--setpref=media.peerconnection.mtransport_process=false', '--setpref=network.process.enabled=false', u'--symbols-path=https://queue.taskcluster.net/v1/task/Sx3Rntt7QjGeuG6wWA0ltQ/artifacts/public/build/target.crashreporter-symbols.zip', '--test-plugin-path=C:\tasks\task_1568988503\build\application\firefox\plugins', '--log-raw=C:\tasks\task_1568988503\build\blobber_upload_dir\xpcshell_raw.log', '--log-errorsummary=C:\tasks\task_1568988503\build\blobber_upload_dir\xpcshell_errorsummary.log', '--utility-path=tests/bin', '--xpcshell=C:\tasks\task_1568988503\build\application\firefox/xpcshell.exe', '--manifest=tests/xpcshell/tests/xpcshell.ini'] with output_timeout 1000
[task 2019-09-20T15:27:38.652Z] 15:27:38 INFO - Found node at C:/tasks/task_1568988503/fetches\node/node.exe
[task 2019-09-20T15:27:38.652Z] 15:27:38 INFO - Found moz-http2 at C:\tasks\task_1568988503\build\tests\xpcshell\moz-http2\moz-http2.js
[task 2019-09-20T15:27:38.672Z] 15:27:38 INFO - Could not run moz-http2 server: [Error 216] This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher
[task 2019-09-20T15:27:38.672Z] 15:27:38 INFO - Traceback (most recent call last):
[task 2019-09-20T15:27:38.672Z] 15:27:38 INFO - File "C:\tasks\task_1568988503\build\tests\xpcshell\runxpcshelltests.py", line 1689, in <module>
[task 2019-09-20T15:27:38.674Z] 15:27:38 INFO - main()
[task 2019-09-20T15:27:38.674Z] 15:27:38 INFO - File "C:\tasks\task_1568988503\build\tests\xpcshell\runxpcshelltests.py", line 1684, in main
[task 2019-09-20T15:27:38.684Z] 15:27:38 INFO - if not xpcsh.runTests(options):
[task 2019-09-20T15:27:38.684Z] 15:27:38 INFO - File "C:\tasks\task_1568988503\build\tests\xpcshell\runxpcshelltests.py", line 1343, in runTests
[task 2019-09-20T15:27:38.689Z] 15:27:38 INFO - self.trySetupNode()
[task 2019-09-20T15:27:38.689Z] 15:27:38 INFO - File "C:\tasks\task_1568988503\build\tests\xpcshell\runxpcshelltests.py", line 1157, in trySetupNode
[task 2019-09-20T15:27:38.694Z] 15:27:38 INFO - startServer('moz-http2', os.path.join(myDir, 'moz-http2', 'moz-http2.js'))
[task 2019-09-20T15:27:38.694Z] 15:27:38 INFO - File "C:\tasks\task_1568988503\build\tests\xpcshell\runxpcshelltests.py", line 1139, in startServer
[task 2019-09-20T15:27:38.699Z] 15:27:38 INFO - stderr=PIPE, env=self.env, cwd=os.getcwd())
[task 2019-09-20T15:27:38.699Z] 15:27:38 INFO - File "c:\mozilla-build\python\Lib\subprocess.py", line 710, in init
[task 2019-09-20T15:27:38.704Z] 15:27:38 INFO - errread, errwrite)
[task 2019-09-20T15:27:38.704Z] 15:27:38 INFO - File "c:\mozilla-build\python\Lib\subprocess.py", line 958, in _execute_child
[task 2019-09-20T15:27:38.706Z] 15:27:38 INFO - startupinfo)
[task 2019-09-20T15:27:38.706Z] 15:27:38 INFO - WindowsError: [Error 216] This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher
[task 2019-09-20T15:27:38.754Z] 15:27:38 ERROR - Return code: 1
[task 2019-09-20T15:27:38.754Z] 15:27:38 ERROR - No tests run or test summary not found
[task 2019-09-20T15:27:38.754Z] 15:27:38 INFO - TinderboxPrint: xpcshell-xpcshell<br/><em class="testfail">T-FAIL</em>
[task 2019-09-20T15:27:38.755Z] 15:27:38 WARNING - # TBPL FAILURE #
[task 2019-09-20T15:27:38.755Z] 15:27:38 WARNING - setting return code to 2
[task 2019-09-20T15:27:38.755Z] 15:27:38 WARNING - The xpcshell suite: xpcshell ran with return status: FAILURE
[task 2019-09-20T15:27:38.755Z] 15:27:38 INFO - Running post-action listener: _package_coverage_data
[task 2019-09-20T15:27:38.755Z] 15:27:38 INFO - Running post-action listener: _resource_record_post_action
[task 2019-09-20T15:27:38.756Z] 15:27:38 INFO - Running post-action listener: process_java_coverage_data
[task 2019-09-20T15:27:38.756Z] 15:27:38 INFO - [mozharness: 2019-09-20 15:27:38.756000Z] Finished run-tests step (success)
[task 2019-09-20T15:27:38.756Z] 15:27:38 INFO - Running post-run listener: _resource_record_post_run

Comment 1

[Noemi Erli[:noemi_erli]]

After Bug 1578963 landed on central, this started to permafail

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=cc11baca8c21d4154fedf25f4ed89fc8d608fc49&tochange=52cb15e3f79439a2e7207eb1f4a785a58c675d08&fromchange=9f25a73981b1e6c79c738ab3971113ae498d119d

Comment 2

[Intermittent Failures Robot]

15 failures in 192 pushes (0.078 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 15

Platform breakdown:

• windows10-aarch64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1582972&startday=2019-09-21&endday=2019-09-21&tree=all

Comment 3

[Intermittent Failures Robot]

22 failures in 4350 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 22

Platform breakdown:

• windows10-aarch64: 22

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1582972&startday=2019-09-16&endday=2019-09-22&tree=all

Comment 4

[Geoff Brown [:gbrown]]

:junior - I don't see any node configuration difference between x86 and aarch64 windows. Do we need a windows aarch64 version of node?

Comment 5

[Junior [inactive]]

(In reply to Geoff Brown [:gbrown] from comment #4)

:junior - I don't see any node configuration difference between x86 and aarch64 windows. Do we need a windows aarch64 version of node?

We should, but I have no idea why it works previously.
AFAICT we didn't have a specific version of node when we used tooltool before bug 1571573.
Looks like there's another path to grab nodejs I didn't figure out

Comment 6

[Junior [inactive]]

I don't know why win32 works for aarch64 and win64 before for testing...

https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=8b50000b89ddcc8ebb0a259f481e1d2459026839&selectedJob=268027574

21:38:50 INFO - INFO - File node-win32.exe retrieved from local cache C:\tooltool-cache
21:38:50 INFO - Return code: 0
21:38:50 INFO - ENV: MOZ_NODE_PATH is now C:\tasks\task_1569272394\build\node-win32.exe

..

21:38:51 INFO - Found node at C:\tasks\task_1569272394\build\node-win32.exe
21:38:51 INFO - Found moz-http2 at C:\tasks\task_1569272394\build\tests\xpcshell\moz-http2\moz-http2.js

Comment 7

[Intermittent Failures Robot]

18 failures in 948 pushes (0.019 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 18

Platform breakdown:

• windows10-aarch64: 18

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1582972&startday=2019-09-25&endday=2019-09-25&tree=all

Comment 8

[Intermittent Failures Robot]

15 failures in 850 pushes (0.018 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 15

Platform breakdown:

• windows10-aarch64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1582972&startday=2019-09-27&endday=2019-09-27&tree=all

Comment 9

[Intermittent Failures Robot]

72 failures in 4530 pushes (0.016 failures/push) were associated with this bug in the last 7 days.

This is the #16 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 72

Platform breakdown:

• windows10-aarch64: 72

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1582972&startday=2019-09-23&endday=2019-09-29&tree=all

Comment 10

[Andreea Pavel [:apavel]]

Mike can you take a look?

Comment 11

[Mike Hommey [:glandium]]

Attachment: Bug 1582972 - Use win32-node for aarch64 tests.

Bug 1571573 removed the manual download of node by mozharness. That made
the tasks use whatever comes from fetches, and for aarch64 tests, it
turned out to be a win64 node binary, which doesn't work on aarch64.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/345d89faf40e
Use win32-node for aarch64 tests. r=nalexander

Comment 13

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/345d89faf40e

Comment 14

[Intermittent Failures Robot]

15 failures in 820 pushes (0.018 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 12
• try: 3

Platform breakdown:

• windows10-aarch64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1582972&startday=2019-09-30&endday=2019-09-30&tree=all

Comment 15

[Oana Pop-Rus]

There have been 45 failures in the last 7 days.

Affected platform is: windows10-aarch64 opt.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=269172272&repo=mozilla-central&lineNumber=1024

Comment 16

[Geoff Brown [:gbrown]]

There have been no failures since the fix landed, on Sept 30; no follow-up is needed.

Comment 17

[Intermittent Failures Robot]

15 failures in 4292 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 12
• try: 3

Platform breakdown:

• windows10-aarch64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1582972&startday=2019-09-30&endday=2019-10-06&tree=all