Closed Bug 2024229 Opened 8 months ago Closed 8 months ago

Data race on lifoBlocksToFree during compacting GC causes null deref [@ GCRuntime::freeFromBackgroundThread]

Categories

(Core :: JavaScript: GC, defect)

defect

Tracking

()

RESOLVED DUPLICATE of bug 2022337

People

(Reporter: bugmon, Unassigned)

Details

(5 keywords)

Attachments

(2 files)

Summary

Data race in SpiderMonkey's GC background-free infrastructure causing a SingleLinkedList invariant violation (head_ == null, last_ != null) and subsequent null-pointer dereference. The crash occurs at js/src/ds/LifoAlloc.h:387 in popFirst(), triggered when GCRuntime::freeFromBackgroundThread() (Sweeping.cpp:585) operates on a LifoAlloc that received corrupted state from the shared GCRuntime::lifoBlocksToFree via transferFrom(). Root cause: the compacting-GC path directly consumes lifoBlocksToFree without joining the already-scheduled BackgroundFreeTask, and/or GCParallelTask double-scheduling when helper threads are saturated by off-thread baseline batching.


Steps to Reproduce

Shell flags required: --fuzzing-safe --baseline-batching=on --baseline-queue-capacity=64

Run test.js under the SpiderMonkey JS shell. The testcase:

  • Lowers GC heap budget (gcparam("maxBytes", 20MB))
  • Generates 80 unique eval'd closures per round → batches them for off-thread Baseline compilation (saturating helper threads)
  • Allocates heavy garbage including "x".repeat(10) strings to populate lifoBlocksToFree with oversize chunks
  • Calls gc(gc, "shrinking") repeatedly → enters the compacting-GC code path

The crash reproduced on 3/3 independent runs within a few seconds. Removing "shrinking" from the gc() call eliminates the crash, confirming the compacting-GC path is a required participant.


Affected Code

File: js/src/ds/LifoAlloc.h, line 387 — crash site

UniquePtr<T, D> popFirst() {
    MOZ_ASSERT(head_);                         // no-op in release build
    UniquePtr<T, D> result = std::move(head_); // result.get() == nullptr
    head_ = std::move(result->next_);          // ★ line 387: nullptr->next_ → SEGV@0x0
    ...
}

The loop guard in LifoAlloc::freeAll() (LifoAlloc.cpp:125) is while (!oversize_.empty()), but empty() checks last_ not head_:

bool empty() const { return !last_; }   // LifoAlloc.h — tests wrong field for this purpose

When the invariant bool(head_) == bool(last_) is broken by concurrent mutation, empty() returns false, the loop is entered, and popFirst() dereferences a null head_.

Register evidence (from the crash dump):

Register Value Meaning
rax 0x0 result.release() → the moved-from head_ was null
r13 0x0 head_ field value
rbx 0x7a0e66bf1220 &oversize_ (base of local lifoBlocks list)
r12 0x7a0e66bf1230 &oversize_.head_ (+0x10)
r15 0x7a0e66bf1238 &oversize_.last_ (+0x18) — still non-null

File: js/src/gc/Sweeping.cpp, line 585 — victim consumer

void GCRuntime::freeFromBackgroundThread(AutoLockHelperThreadState& lock) {
  do {
    LifoAlloc lifoBlocks(...);                          // stack-local on helper thread
    lifoBlocks.transferFrom(&lifoBlocksToFree.ref());   // ← copies corrupt {head_,last_} pair
    ...
    AutoUnlockHelperThreadState unlock(lock);           // lock DROPPED
    lifoBlocks.freeAll();                               // ★ line 585 → crash
  } while (hasBuffersForBackgroundFree());              // lock reacquired for check
}

transferFrom() → prependAll() → steal() copies head_ and last_ as two independent, non-atomic stores and performs no invariant validation, so a torn {null, non-null} state propagates silently.

File: js/src/gc/Compacting.cpp, line 864 — racing consumer

// inside updateRuntimePointersToRelocatedCells — only reached during shrinking GC
{
    AutoLockHelperThreadState lock;
    lifoBlocksToFree.ref().freeAll();   // ★ directly mutates shared list WITHOUT freeTask.join()
}

Diagnostic evidence of concurrency: A second AddressSanitizer:DEADLYSIGNAL is interleaved into the ASan report (immediately after Thread T4 created by T0 here:). This is the signature of another thread faulting while ASan is mid-report on T4 — proving two threads crashed on related state near-simultaneously.


Exploit Chain

  1. JS entry: gc(gc, "shrinking") invoked from content script under low maxBytes.
  2. Helper saturation: --baseline-batching=on --baseline-queue-capacity=64 + 80 eval'd closures fill all helper threads with long-running BaselineCompileTasks. The BackgroundFreeTask sits in Queued state or is delayed.
  3. Free-queue population: Memory pressure forces minor GCs; ICStubSpace::freeAllAfterMinorGC / ICScript::prepareForDestruction queue oversize BumpChunks into lifoBlocksToFree. freeTask is started.
  4. Helper thread T4 picks up freeTask → GCParallelTask::runHelperThreadTask → freeFromBackgroundThread: takes lock, transferFrom(), drops lock, enters freeAll().
  5. Main thread T0 enters compacting phase → updateRuntimePointersToRelocatedCells → Compacting.cpp:864: takes lock (T4 released it), calls freeAll() directly on lifoBlocksToFree without joining freeTask.
  6. Invariant torn: popFirst() performs two sequential non-atomic stores (head_ = …, then conditional last_ = nullptr). Interleaving across the T4/T0 lock-drop-reacquire boundary — or double-scheduling of freeTask itself when GCParallelTask::wasStarted() fails to recognize Queued state — leaves oversize_ with head_ = null, last_ = stale.
  7. Corruption propagation: On T4's next loop iteration, transferFrom() copies the broken pair into the local lifoBlocks.
  8. Crash: while (!oversize_.empty()) sees last_ != null → enters → popFirst() reads head_ == null → nullptr->next_ → SEGV at 0x000000000000.

Security Impact

Property Assessment
Severity Moderate–High
Immediate effect Helper-thread null-deref → process crash (DoS)
Escalation potential The surviving last_ is a dangling raw pointer to a BumpChunk disconnected from head_. Any path reaching oversize_.last()->... (e.g. LifoAlloc::isEmpty()) before the crash would perform a use-after-free. Depending on interleaving, the race could also manifest as double-free of BumpChunks rather than null-deref.
Attacker control Content JavaScript can reliably trigger via memory-pressure loops + closures + gc(). No shell-only primitives needed.
Preconditions Off-thread baseline batching enabled; shrinking/compacting GC must occur; ≥2 helper threads. Batching is on the shipping track, and shrinking GCs fire organically under real memory pressure.

Suggested Fix

Primary fix — join freeTask before the compacting phase directly drains lifoBlocksToFree:

// js/src/gc/Compacting.cpp  (inside updateRuntimePointersToRelocatedCells)
-  {
-    AutoLockHelperThreadState lock;
-    lifoBlocksToFree.ref().freeAll();
-  }
+  // BackgroundFreeTask::freeFromBackgroundThread() drops the helper-thread lock
+  // between transferFrom() and its loop condition. Holding the lock here is
+  // insufficient: the task's next iteration can observe a torn oversize_ list.
+  // Join the task before directly consuming lifoBlocksToFree.
+  freeTask.join();
+  {
+    AutoLockHelperThreadState lock;
+    lifoBlocksToFree.ref().freeAll();
+  }

Defense-in-depth — close the Queued-state gap in GCParallelTask:

// js/src/gc/GCParallelTask.h
 bool wasStarted(const AutoLockHelperThreadState& lock) const {
-  return isDispatched(lock) || isRunning(lock);
+  return isQueued(lock) || isDispatched(lock) || isRunning(lock);
 }

This prevents startOrRunIfIdle() from double-scheduling a task that is queued but not yet dispatched when all helper threads are busy.

Attached file Crash stack trace —
Group: core-security → javascript-core-security
Component: JavaScript Engine → JavaScript: GC
Flags: needinfo?(jcoppeard)
Attached file test.js —

This looks very similar to bug 2022997, which also purported to find bugs involving lifoBlocksToFree. In that case I concluded that the real problem was bug 2022337.

I failed to reproduce this locally, even using rr chaos mode. I tried running with the problem from bug 2022337 reinserted; that didn't fail normally, but did fail in chaos mode. In that case, head_ is 0x0 but last_ is 0x3, which is the bogus value written by jitcode. This is consistent with the description above:

When the invariant bool(head_) == bool(last_) is broken by concurrent mutation, empty() returns false, the loop is entered, and popFirst() dereferences a null head_.

Much like bug 2022997, I find it difficult to follow Claude's proposed chain of events. I once again suspect that Claude is having a hard time reasoning about (or remembering?) the bug in jitcode clobbering the LifoAlloc's memory, and tries to invent a story that doesn't include the jitcode bug. That story doesn't make any sense, because it's not a true story.

It looks to me like all the locking here is working just fine. Jon, can you take a quick look at this to make sure I'm not missing anything obvious? Assuming I'm not, I think we should be fine to close this as a duplicate of 2022997.

(In reply to Iain Ireland [:iain] from comment #4)
Thanks for looking into this Iain.

The locking is fine here. lifoBlocksToFree is wrapped in HelperThreadLockData and we take the helper thread lock whenever we access it.

I'll dupe to bug 2022997.

Status: UNCONFIRMED → RESOLVED
Closed: 8 months ago
Duplicate of bug: 2022997
Flags: needinfo?(jcoppeard)
Resolution: --- → DUPLICATE

Can we add the commit the crash was reproduced at to these bugs in the future?

Flags: needinfo?(jkratzer)
Duplicate of bug: 2022337
No longer duplicate of bug: 2022997

(In reply to Jon Coppeard (:jonco) from comment #6)

Can we add the commit the crash was reproduced at to these bugs in the future?

https://github.com/MozillaSecurity/clauditor/issues/61

Flags: needinfo?(jkratzer)
Attachment #9554237 - Attachment mime type: application/x-javascript → text/plain
Group: javascript-core-security
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: