2046921 - Assertion failure: cx->isThrowingOutOfMemory() || cx->isThrowingOverRecursed() in js::atomics_notify_impl when resolving a foreign-compartment Atomics.waitAsync promise with nuked reaction wrappers

Status: NEW

(Core :: JavaScript: Standard Library, defect, P3)

Description

[Christian Holler (:decoder)]

Summary

js::atomics_notify_impl (js/src/builtin/AtomicsObject.cpp) resolves same-thread Atomics.waitAsync promises directly after leaving the futex critical section. The loop asserts that PromiseObject::resolve can only fail because of OOM or over-recursion:

// js/src/builtin/AtomicsObject.cpp:1522-1529
RootedValue resultMsg(cx, StringValue(cx->names().ok));
for (uint32_t i = 0; i < promisesToResolve.length(); i++) {
  AutoRealm ar(cx, promisesToResolve[i]);
  if (!PromiseObject::resolve(cx, promisesToResolve[i], resultMsg)) {
    MOZ_ASSERT(cx->isThrowingOutOfMemory() || cx->isThrowingOverRecursed());
    return false;
  }
}

This assumption is wrong when the promise being resolved is a foreign-compartment promise whose reaction list contains a cross-compartment wrapper that has been nuked (the browser does this on window/sandbox teardown via js::NukeCrossCompartmentWrappers). In that case promise resolution fails with a JSMSG_DEAD_OBJECT TypeError rather than an OOM:

// js/src/builtin/Promise.cpp:1828-1834 (EnqueuePromiseReactionJobCrossRealm)
} else {
  JSObject* unwrappedReactionObj = UncheckedUnwrap(reactionObj);
  if (JS_IsDeadWrapper(unwrappedReactionObj)) {
    JS_ReportErrorNumberASCII(cx, GetErrorMessage, nullptr,
                              JSMSG_DEAD_OBJECT);
    return false;
  }

• Debug builds: assertion failure (crash) at js/src/builtin/AtomicsObject.cpp:1526.
• Release builds: Atomics.notify throws an unexpected TypeError: can't access dead object, and — worse — all remaining same-thread waiters that were already dequeued from the SAB waiter list (their WaitAsyncNotifyTask already destroyed via ExtractAndForget, timeouts already cleared via maybeClearTimeout) are silently dropped: their promises can never be resolved again (lost wakeups for unrelated, perfectly healthy realms), even though they were counted in the reported woken count.

Root cause

The same-thread resolve path was introduced by Bug 1467846 part 20 ("Don't dispatch task for same-thread notification"), which added the MOZ_ASSERT on the assumption that resolving a promise with a plain string value cannot run user code and so can only fail on OOM/over-recursion. Bug 2013562 later added the AutoRealm on line 1524 so foreign-compartment promises are resolved in their own realm — but the error-handling assumption was left unchanged.

Resolving a promise triggers its stored reactions (ResolvePromiseInternal -> FulfillMaybeWrappedPromise -> TriggerPromiseReactions -> EnqueuePromiseReactionJob). A reaction created in a different compartment than the promise (e.g. Promise.prototype.then.call(wrappedPromise, fn) performed from another realm) is stored in the promise's reaction list as a cross-compartment wrapper. If that wrapper is subsequently nuked, enqueueing the reaction job fails with a dead-object TypeError, which propagates out of PromiseObject::resolve and violates the assert.

The sibling cross-thread path (OffThreadPromiseTask::run, js/src/vm/OffThreadPromiseRuntimeState.cpp:133-141) handles arbitrary resolve failure gracefully (cx->clearPendingException()), confirming the same-thread path's assumption is too strong.

Reproduction

Shell version: SpiderMonkey JS shell, product_version = 20260603-5cffec5992d3 (m-c-asan-debug, debug ASan build).

Run the deterministic testcase poc.js (attached):

m-c-asan-debug/dist/bin/js --fuzzing-safe poc.js

poc.js:

const g = newGlobal({ newCompartment: true });
const sab = new SharedArrayBuffer(4);
g.sab = sab;
const p = g.eval("Atomics.waitAsync(new Int32Array(sab), 0, 0).value");
Promise.prototype.then.call(p, () => {});
nukeAllCCWs();
Atomics.notify(new Int32Array(sab), 0);

nukeAllCCWs() models the browser-side js::NukeCrossCompartmentWrappers mechanism (window/sandbox teardown / Cu.nukeSandbox); the buggy code itself is generic Atomics/Promise engine code shipped in Firefox. No special prefs are required.

demo-release.js (attached) demonstrates the release-mode damage without assertions: a second healthy waiter in an unrelated realm waiting on the same location is dequeued by the failing Atomics.notify but its promise is never resolved (lost wakeup), and Atomics.notify throws a spec-disallowed TypeError.

Crash signature (debug ASan build)

Assertion failure: cx->isThrowingOutOfMemory() || cx->isThrowingOverRecursed(), at js/src/builtin/AtomicsObject.cpp:1526

  #1 js::atomics_notify_impl(JSContext*, js::SharedArrayRawBuffer*, unsigned long, long, long*)
     js/src/builtin/AtomicsObject.cpp:1526:7
  #2 atomics_notify(JSContext*, unsigned int, JS::Value*)
     js/src/builtin/AtomicsObject.cpp:1599:8
  #3 CallJSNative(JSContext*, ...) js/src/vm/Interpreter.cpp:498:13

Suggested fix

Treat per-promise resolution failure in the same-thread loop the way the dispatch path (OffThreadPromiseTask::run) does: drop the too-strong MOZ_ASSERT, and don't abort the loop on a per-promise failure — clear (or report) the pending exception and continue resolving the remaining promises, so one realm with nuked wrappers cannot eat wakeups belonging to other realms.

Files

• poc.js — deterministic assertion-failure testcase (debug build).
• demo-release.js — release-mode demonstration (unexpected TypeError + lost wakeup for an unrelated realm).
• t1.js-t7.js — exploration testcases (t7 is the original finding).

Comment 1

[Christian Holler (:decoder)]

Attachment: poc.js — deterministic assertion-failure testcase (debug build)

Comment 2

[Christian Holler (:decoder)]

Attachment: demo-release.js — release-mode demonstration (unexpected TypeError + lost wakeup)

Comment 3

[Christian Holler (:decoder)]

Attachment: t1.js — exploration testcase

Comment 4

[Christian Holler (:decoder)]

Attachment: t2.js — exploration testcase

Comment 5

[Christian Holler (:decoder)]

Attachment: t3.js — exploration testcase

Comment 6

[Christian Holler (:decoder)]

Attachment: t4.js — exploration testcase

Comment 7

[Christian Holler (:decoder)]

Attachment: t5.js — exploration testcase

Comment 8

[Christian Holler (:decoder)]

Attachment: t6.js — exploration testcase

Comment 9

[Christian Holler (:decoder)]

Attachment: t7.js — exploration testcase (original finding)