1513231 - wasm spec jit-tests failing on (some) ARM hardware

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, defect, P2)

Description

[Lars T Hansen [:lth]]

Forked from bug 1454918.

Wasm jit-tests failing on Moto G5:

TEST-UNEXPECTED-FAIL | tests/jit-test/jit-test/tests/wasm/spec/float_memory.wast.js | #1 module successfully instantiated: PASS. (code 3, args "") [0.3 s]

TEST-UNEXPECTED-FAIL | tests/jit-test/jit-test/tests/wasm/spec/memory_redundancy.wast.js | #1 module successfully instantiated: PASS. (code 3, args "") [0.2 s]

Comment 1

[Lars T Hansen [:lth]]

Specifically see bug 1454918 comment 11 for link to failing run.

Comment 2

[Chris Peterson [:cpeterson]]

Should we fix this ARM64 wasm issue before pushing ARM64 Fennec Nightly builds to users?

Tentatively tagging as [arm64:m1].

Comment 3

[Lars T Hansen [:lth]]

Note this IIUC this is an ARM-32 issue, not an ARM-64 issue.  It should be fixed, but likely it's been in the wild for many months already.  I would not make it a blocker, but I do want to investigate further.

Comment 4

[Chris Peterson [:cpeterson]]

Thanks for the clarification.

Comment 5

[Bob Clary [:bc] (inactive)]

Note that I had to disable several wasm tests to green up the jittests. These included both arm7 and arm64.
See <https://hg.mozilla.org/mozilla-central/rev/4688c503e2ee>

jit-test/tests/wasm/atomic.js arm64
jit-test/tests/wasm/baseline-abs-addr-opt.js arm64
jit-test/tests/wasm/bce.js arm64
jit-test/tests/wasm/memory.js arm64
jit-test/tests/wasm/spec/float_memory.wast.js arm7
jit-test/tests/wasm/spec/memory_redundancy.wast.js arm7

Comment 6

[Lars T Hansen [:lth]]

Bob, do you have a link to a failed test run?  I'm curious why the four non-/spec/ tests failed on arm64, did they time out or were there other kinds of failures?  (Note we should not track those here because this bug is exclusively about ARM-32 but we might as well have the discussion here.)

Comment 7

[Bob Clary [:bc] (inactive)]

I backed out those skips and did a full try run with rebuild 20 which was definitely overkill but guaranteed to get everything:
<https://treeherder.mozilla.org/#/jobs?repo=try&revision=879e771626c04bace4d2c0a57e957ae6d1efa38b>.

Looks like all "out of memory":
<https://taskcluster-artifacts.net/AmQku0U-QymQOVSq7ETc7A/0/public/logs/live_backing.log>

baseline-abs-addr-opt.js, memory.js expect a RuntimeError but the others have uncaught out of memory exceptions.

Comment 8

[Lars T Hansen [:lth]]

Thanks!  We need to track down why those tests OOM.  They shouldn't.  Tracking this as bug 1517412.

Comment 9

[Lars T Hansen [:lth]]

Bob, can I still get access to the Moto G5 on try somehow? (I can't find it with mach try fuzzy, only pixel2 devices.) I pretty much know why the ARM-32 tests fail on the Pixel 2 but I dearly would like to know whether the Moto G5 has the same problem or if there is a separate problem there.

Comment 10

[Bob Clary [:bc] (inactive)]

I'll add one device to the appropriate device pool and give you a patch to enable it.

Comment 11

[Bob Clary [:bc] (inactive)]

I did two patches: One to add g5 opt jittests and one to enable debug jittests for both the g5 and p2. The first g5 debug jit1 chunk took almost 45 minutes, so I added another device to the g5 unittest pool for a total of 2 while reducing the g5 perftest pool by a corresponding number.

You can see the results of both patches at https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=fec4f602855dd0de6a27590db6d54e9bba4554a5. Let me know when you are no longer interested in the g5 jit so I can put those devices back into the perf pool.

If you don't need debug, you can just apply the g5 opt patch.

Remember you need to use mach try fuzzy to submit try jobs for android-hw. Please use try to not run too many tests and DOS the android-hw devices. You can use --no-push to see which tests will actually run for a give mach try fuzzy invocation.

Comment 12

[Lars T Hansen [:lth]]

Deep magic :)

Comment 13

[Lars T Hansen [:lth]]

OK, things seem to be going ok but the test job is queued, presumably behind the ones you've got running. Not a problem, just means we'll see results tomorrow instead of today.

Comment 14

[Bob Clary [:bc] (inactive)]

It has completed now with a failure.

Comment 15

[Lars T Hansen [:lth]]

Indeed. These are the same failures as on the Pixel 2. You may reclaim the Moto G5s until further notice. Thanks!

Comment 16

[Bob Clary [:bc] (inactive)]

done. Let me know if you need them reinstated.