1525416 - windows/aarch64 - toolkit/components/ctypes/tests/unit/test_jsctypes.js | xpcshell return code: -1073741819

Status: RESOLVED FIXED

(Core :: js-ctypes, defect, P5)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: egao [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=226206279&repo=try

https://queue.taskcluster.net/v1/task/e17BNGL8RJG1pPoO1IDmJA/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Is this intermittent or permanent? Later in the test suite execution there is also this failure

test_ext_runtime_getPlatformInfo.js | - Architecture is valid - false == true
Similar platform issues could have also caused the ctype issue.

Comment 2

[Edwin Takahashi (:egao | infrequent contributor)]

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #1)

Is this intermittent or permanent? Later in the test suite execution there is also this failure

test_ext_runtime_getPlatformInfo.js | - Architecture is valid - false == true
Similar platform issues could have also caused the ctype issue.

It is uncertain whether this is an intermittent or a permafail for win10/aarch64 hardware. I suspect it is a permafail until a fix is applied, though there isn't additional runs to verify this.

I will update the bug summary accordingly after a verification run.

As for the subsequent failure, I believe that should be logged as a separate bug.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I am not sure why test_jsctypes.js is failing here for windows on arm64, the detailed log doesn't help much.

Possibly we should make sure this test doesn't run in parallel, maybe running it sequentially similar to this:
https://searchfox.org/mozilla-central/source/devtools/shared/adb/test/xpcshell.ini#10

:m_kato, do you know more about this test (test_jsctypes.js) and could help figure this out?

Comment 4

[Makoto Kato [:m_kato]]

(In reply to Edwin Gao (:egao) from comment #2)

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #1)

Is this intermittent or permanent? Later in the test suite execution there is also this failure

test_ext_runtime_getPlatformInfo.js | - Architecture is valid - false == true
Similar platform issues could have also caused the ctype issue.

It is uncertain whether this is an intermittent or a permafail for win10/aarch64 hardware. I suspect it is a permafail until a fix is applied, though there isn't additional runs to verify this.

validArchs should have aarch64 in https://searchfox.org/mozilla-central/source/toolkit/components/extensions/test/xpcshell/test_ext_runtime_getPlatformInfo.js#8

toolkit/components/ctypes/tests/unit/test_jsctypes.js | xpcshell return code: -1073741819

log doesn't have crash stack by 0xc0000005. So I need attach debugger when running test.

Comment 5

[Intermittent Failures Robot]

2 failures in 2658 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform breakdown:

• windows10-aarch64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1525416&startday=2019-02-04&endday=2019-02-10&tree=all

Comment 6

[Edwin Takahashi (:egao | infrequent contributor)]

Try push with modification let validArchs = ["arm", "x86-32", "x86-64", "aarch64"]; has completed running.

Treeherder: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=227771812&revision=7a397ef4e73ead67f0ab8f1c7ae098d1d079d0ce

Taskcluster: https://taskcluster-artifacts.net/G05FQ86WR3qxc0zR-JrGVw/0/public/logs/live_backing.log

The following errors are no longer reproducible after the change:

TEST-UNEXPECTED-FAIL | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_runtime_getPlatformInfo.js | - Extension left running at test shutdown - "running" == "unloaded"
TEST-UNEXPECTED-FAIL | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_runtime_getPlatformInfo.js | - Architecture is valid - false == true
TEST-UNEXPECTED-FAIL | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_runtime_getPlatformInfo.js | xpcshell return code: 0

Still reproducible is the following:

TEST-UNEXPECTED-FAIL | toolkit/components/ctypes/tests/unit/test_jsctypes.js | xpcshell return code: -1073741819

Comment 7

[Makoto Kato [:m_kato]]

TEST-UNEXPECTED-FAIL | toolkit/components/ctypes/tests/unit/test_jsctypes.js | xpcshell return code: -1073741819

dmajor, do you know root cause that aarch64 windows doesn't pass jsctypes test?

Comment 8

[(Away)]

This trampoline... https://searchfox.org/mozilla-central/rev/cb7faaf6b4ad2528390186f1ce64618dea71031e/js/src/ctypes/libffi/src/aarch64/ffi.c#910

...does not have execute permission, only PAGE_READWRITE. I bet some piece of libffi's ifdef horror isn't considering arm64.

Comment 9

[(Away)]

Attachment: Teach libffi/src/closures.c about ARM64_WIN64

I added ARM64_WIN64 to every line that mentioned X86_WIN32 and X86_WIN64.

This makes sure the allocation routines do the proper VirtualProtect on the trampolines.

Comment 10

[Edwin Takahashi (:egao | infrequent contributor)]

I can re-run the suite after the patch is merged to m-c.

:needinfo myself

Comment 11

[(Away)]

mach test toolkit/components/ctypes/tests/unit/test_jsctypes.js passes with this patch. I haven't run the other suites.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/430f461b9636
Teach libffi/src/closures.c about ARM64_WIN64 r=froydnj

Comment 13

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/430f461b9636

Comment 14

[Edwin Takahashi (:egao | infrequent contributor)]

Re-ran the test locally with a build run after this patch has been integrated to m-c.

Issue cannot be reproduced. Confirm this has been fixed for opt-xpcshell-6.

From the test run locally:

17:19:27     INFO -  TEST-PASS | toolkit/components/ctypes/tests/unit/test_jsctypes.js | took 26725ms