1480725 - Stress tests for wasm atomics

Status: RESOLVED FIXED

(Core :: JavaScript: WebAssembly, enhancement, P3)

Description

[Lars T Hansen [:lth]]

We have single-threaded test cases for the wasm atomics but not really any tests that test atomicity by, say, hammering on a location to produce a result that will be wrong if atomicity is incorrectly implemented.  We need these.

Comment 1

[Lars T Hansen [:lth]]

Attachment: worker-threads.js

Comment 2

[Lars T Hansen [:lth]]

Attachment: bug1480725-stress-test-wasm-atomics.js

Much more can be done here but this should be a good start.  See commit msg and comments in the code for more about the mechanics.

Comment 3

[Luke Wagner [:luke]]

Comment on attachment 9002399 [details] [diff] [review]
bug1480725-stress-test-wasm-atomics.js

Review of attachment 9002399 [details] [diff] [review]:
-----------------------------------------------------------------

Cool, thanks for writing this.

Comment 4

[Luke Wagner [:luke]]

It'd be nice to have other browsers running this; I wonder if webplatform-tests is the best way?

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e2b35cf75610
stress test wasm atomics.  r=luke

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/50af5b3ee1e2
mark tests as slow temporarily r=me CLOSED TREE

Comment 7

[Lars T Hansen [:lth]]

These time out on the buildbot even with a reduced iteration count, https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=d9d4bf8a92b36bb8ebadbaecf951e4bfa268d877&filter-searchStr=jit.  Not sure what's going on.  They run fine locally.

Comment 8

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/e2b35cf75610
https://hg.mozilla.org/mozilla-central/rev/50af5b3ee1e2

Comment 9

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

I imagine wpt is the easiest way to get this running in other browsers, yeah. Do other JS shells have similar worker APIs? Otherwise I guess it'd have to be web workers only.

Comment 10

[Lars T Hansen [:lth]]

(In reply to :Ms2ger (he/him; ⌚ UTC+1/+2) from comment #9)
> I imagine wpt is the easiest way to get this running in other browsers,
> yeah. Do other JS shells have similar worker APIs? Otherwise I guess it'd
> have to be web workers only.

Other JS shells have very different worker APIs, unfortunately.  That's why there is a fairly elaborate and contorted "agent" infrastructure in test262 to abstract away from the shell APIs.  (I get to say "contorted" and other bad things about it because I did most of the design work on it.)  I've been promising for some time to design a worker API for JS shells that the implementations could all agree on, but time has not allowed me to complete that :-(  The root for that work, with links into github repos for the TC39 discussions etc, is bug 1414823.

As you can see from comment 6 and comment 7 above, we're already having problems with these tests on the buildbots because the buildbots are very, very slow and apparently incapable of running tests of this kind - they time out at even very moderate iteration counts.  A test that takes 17 seconds on my machine appears to time out after an hour on the build bots.  This *could* be evidence of a mutual exclusion bug - the kind of bug the tests are designed to find - but given that the timeout is entirely repeatable on the buildbot but not locally suggests an infrastructure problem really.

Until that's been diagnosed we should not worry overmuch about moving these tests to WPT.

Comment 11

[Lars T Hansen [:lth]]

(In reply to Lars T Hansen [:lth] from comment #7)
> These time out on the buildbot even with a reduced iteration count,
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-
> inbound&revision=d9d4bf8a92b36bb8ebadbaecf951e4bfa268d877&filter-
> searchStr=jit.  Not sure what's going on.  They run fine locally.

Hypothesis:

If our SpiderMonkey worker threads are virtualized onto some hypervisor cpu pool on the buildbot so that we only actually run one thread at a time then (a) the tests are pretty useless and (b) they can take a long time to run, because each thread will finish its work and then sit and spin until preempted, at which point another thread gets to go.  If preemption is delayed because timeslices are somewhat long then completion could take a while.  Furthermore, if threads are not actually preempted except at syscalls (say) then we can't finish, because there are no syscalls in the wait loop - by design.

It is possible that this is why some of the tests in js/src/tests/shell have been flagged as slow, too.  (Meaning, never run them.)

Adding some kind of explicit wait (that involves a syscall or manual preemption) in the test case runs counter to the purpose of the test case, as that will reduce the chances of overlapping operations from multiple threads.