Closed Bug 1480725 Opened Last year Closed Last year

Stress tests for wasm atomics

Categories

(Core :: Javascript: WebAssembly, enhancement, P3)

enhancement

Tracking

()

RESOLVED FIXED
mozilla63
Tracking Status
firefox63 --- fixed

People

(Reporter: lth, Assigned: lth)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

We have single-threaded test cases for the wasm atomics but not really any tests that test atomicity by, say, hammering on a location to produce a result that will be wrong if atomicity is incorrectly implemented.  We need these.
Attached file worker-threads.js (obsolete) —
Blocks: resab
No longer blocks: 1444925
Much more can be done here but this should be a good start.  See commit msg and comments in the code for more about the mechanics.
Attachment #8997387 - Attachment is obsolete: true
Attachment #9002399 - Flags: review?(luke)
Attachment #9002399 - Attachment is patch: true
Attachment #9002399 - Attachment mime type: application/x-javascript → text/plain
Comment on attachment 9002399 [details] [diff] [review]
bug1480725-stress-test-wasm-atomics.js

Review of attachment 9002399 [details] [diff] [review]:
-----------------------------------------------------------------

Cool, thanks for writing this.
Attachment #9002399 - Flags: review?(luke) → review+
It'd be nice to have other browsers running this; I wonder if webplatform-tests is the best way?
Pushed by lhansen@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/50af5b3ee1e2
mark tests as slow temporarily r=me CLOSED TREE
These time out on the buildbot even with a reduced iteration count, https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=d9d4bf8a92b36bb8ebadbaecf951e4bfa268d877&filter-searchStr=jit.  Not sure what's going on.  They run fine locally.
https://hg.mozilla.org/mozilla-central/rev/e2b35cf75610
https://hg.mozilla.org/mozilla-central/rev/50af5b3ee1e2
Status: ASSIGNED → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
I imagine wpt is the easiest way to get this running in other browsers, yeah. Do other JS shells have similar worker APIs? Otherwise I guess it'd have to be web workers only.
(In reply to :Ms2ger (he/him; ⌚ UTC+1/+2) from comment #9)
> I imagine wpt is the easiest way to get this running in other browsers,
> yeah. Do other JS shells have similar worker APIs? Otherwise I guess it'd
> have to be web workers only.

Other JS shells have very different worker APIs, unfortunately.  That's why there is a fairly elaborate and contorted "agent" infrastructure in test262 to abstract away from the shell APIs.  (I get to say "contorted" and other bad things about it because I did most of the design work on it.)  I've been promising for some time to design a worker API for JS shells that the implementations could all agree on, but time has not allowed me to complete that :-(  The root for that work, with links into github repos for the TC39 discussions etc, is bug 1414823.

As you can see from comment 6 and comment 7 above, we're already having problems with these tests on the buildbots because the buildbots are very, very slow and apparently incapable of running tests of this kind - they time out at even very moderate iteration counts.  A test that takes 17 seconds on my machine appears to time out after an hour on the build bots.  This *could* be evidence of a mutual exclusion bug - the kind of bug the tests are designed to find - but given that the timeout is entirely repeatable on the buildbot but not locally suggests an infrastructure problem really.

Until that's been diagnosed we should not worry overmuch about moving these tests to WPT.
(In reply to Lars T Hansen [:lth] from comment #7)
> These time out on the buildbot even with a reduced iteration count,
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-
> inbound&revision=d9d4bf8a92b36bb8ebadbaecf951e4bfa268d877&filter-
> searchStr=jit.  Not sure what's going on.  They run fine locally.

Hypothesis:

If our SpiderMonkey worker threads are virtualized onto some hypervisor cpu pool on the buildbot so that we only actually run one thread at a time then (a) the tests are pretty useless and (b) they can take a long time to run, because each thread will finish its work and then sit and spin until preempted, at which point another thread gets to go.  If preemption is delayed because timeslices are somewhat long then completion could take a while.  Furthermore, if threads are not actually preempted except at syscalls (say) then we can't finish, because there are no syscalls in the wait loop - by design.

It is possible that this is why some of the tests in js/src/tests/shell have been flagged as slow, too.  (Meaning, never run them.)

Adding some kind of explicit wait (that involves a syscall or manual preemption) in the test case runs counter to the purpose of the test case, as that will reduce the chances of overlapping operations from multiple threads.
You need to log in before you can comment on or make changes to this bug.