Closed Bug 2045459 Opened 5 months ago Closed 4 months ago

Use-after-free: Zone::forEachNonEmptyChunk's clearCurrentChunk side effect defeats concurrent BackgroundUnmarkTask, leaving stale mark bits

Categories

(Core :: JavaScript: GC, defect, P1)

defect

Tracking

()

RESOLVED FIXED
154 Branch
Tracking Status
firefox-esr153 --- fixed
firefox154 --- fixed

People

(Reporter: jandem, Assigned: sfink)

References

Details

(Keywords: ai-involved)

Attachments

(2 external files)

Summary

JS::Zone::forEachNonEmptyChunk (js/src/gc/GC-inl.h:364) calls gc->clearCurrentChunk(this, lock) as a side effect before iterating, moving the zone's current chunk back onto the head of its available (or full) chunk list. This violates the one-direction chunk-movement invariant that BackgroundUnmarkTask::unmark (js/src/gc/GC.cpp:3111-3178, added by Bug 2004011 Part 2, commit a3d0ba23922c) relies on to clear mark bits chunk-by-chunk while the mutator runs:

Chunks may only move in one direction while we are running: from the available pool to the the current chunk to the full list. (GC.cpp:3121-3122)

If clearCurrentChunk runs while the unmark walker is past the head of the zone's available list, the former current chunk is pushed onto the list behind the walker and never visited; the subsequent "unmark the current chunk" step reads zone->currentChunk_ == nullptr (cleared by clearCurrentChunk) and skips it. The chunk's mark bits from the previous GC are never cleared.

Stale-black live cells in the missed chunk are then treated as already-marked during the new GC's mark phase, so their children are never traced. Children reachable only through such a cell remain unmarked and are swept while still reachable — use-after-free / heap corruption.

Mechanism (verified in source)

  1. Zone::forEachNonEmptyChunk (GC-inl.h:364) unconditionally calls clearCurrentChunk before walking the available + full lists.
  2. clearCurrentChunk (GC.cpp:535-563) sets zone->currentChunk_ = nullptr (line 548) and, for a partially-free chunk, pushes it onto the available-list head (line 562) — a reverse-direction current → available move.
  3. BackgroundUnmarkTask::unmark's available-list walk (GC.cpp:3142-3154) restarts only when its own current chunk leaves the list; it never revisits a chunk freshly inserted at the head behind it. The "unmark the current chunk" step (3157-3160) then reads the now-nullptr currentChunk_ and skips it. The former current chunk (holding stale-black cells) is never unmarked.

Trigger path (debug builds)

A minor GC can run between slices of an incremental major GC, including during Prepare while the unmark task walks chunk lists. In debug builds, StoreBuffer::CellPtrEdge<JSString>::trace (Tenuring.cpp:826-833) asserts via GCRuntime::isPointerWithinTenuredCell (Verifier.cpp:1285), which calls forEachNonEmptyChunk on every zone — including zones the minor GC never allocates into. A CellPtrEdge<JSString> is trivially planted from fuzzing-safe JS via RegExpStatics (/abc/.exec(nurseryString)). Each buffered string edge during a minorgc() in Prepare lands a clearCurrentChunk inside the concurrent unmark walk.

Verification

  • Threaded (default): SIGSEGV in 100% of runs, within 3-4 iterations, always after state0=Prepare.
  • --no-threads control: exits 0 cleanly in 100% of runs (unmark runs synchronously; no race window).
  • Crash signature (deterministic across runs):
    rax = 0xfffe4b4b00000000        # NaN-boxed Value over JS_SWEPT_TENURED_PATTERN (0x4b)
    => movzbl 0x10(%rax),%ecx        # READ of swept memory
    #3 JSObject::zoneFromAnyThread
    #4 js::CheckTracedThing<JSObject>     (Marking.cpp:159)
    #5 js::gc::TraceEdgeInternal "objectFixedSlots"
    
    A reachable object's fixed slot points at swept-tenured poison, i.e. a cell swept while reachable. gczeal(15) CheckHeapAfterGC surfaces it deterministically; the corruption is real, not a gczeal artifact.

Reachability

As reachable today this is debug-build-only: the trigger is the MOZ_ASSERT in CellPtrEdge<JSString>::trace (Tenuring.cpp:830) → isPointerWithinTenuredCell → forEachNonEmptyChunk, plus js::debug::GetMarkInfo (Marking.cpp:3348). The release-build memory-reporting path (JS::IterateChunks) is not reachable during an in-progress incremental GC: CollectRuntimeStatsHelper (MemoryMetrics.cpp:671) enters AutoPrepareForTracing → AutoFinishGC → FinishGC(GCReason::PREPARE_FOR_TRACING), finishing any incremental GC before iterating; the same applies to the other AutoPrepareForTracing users. The invariant violation itself is build-independent — any future lock-only (non-session) caller of forEachNonEmptyChunk during Prepare would make it release-reachable.

Suggested fix

Make Zone::forEachNonEmptyChunk non-mutating: iterate currentChunk_ in place instead of folding it back via clearCurrentChunk, exactly as countTotalChunks / countEmptyChunks (GC.cpp:426-464) already do. (Alternative: make the unmark walk robust to current → available/full transitions, e.g. via a generation counter / head re-check.)

Test case (repro.js)

// Race BackgroundUnmarkTask::unmark (GC.cpp:3111) against the clearCurrentChunk
// side effect of forEachNonEmptyChunk (GC-inl.h:364), reached via the debug
// assert in StoreBuffer::CellPtrEdge<JSString>::trace (Tenuring.cpp:830 ->
// GCRuntime::isPointerWithinTenuredCell). clearCurrentChunk pushes the victim
// zone's current chunk onto the head of its available list while the unmark
// walker is past the head, so the chunk's stale mark bits are never cleared.
// A stale-black anchor's children are then never traced and get swept while
// reachable -> heap corruption, caught by gczeal(15) CheckHeapAfterGC.
var g = newGlobal({newCompartment: true});
g.eval(`
var anchors = [];
function build(n) {
  anchors = []; var kids = []; var junk = [];
  for (var i = 0; i < n; i++) { kids.push({m: i}); junk.push([i,i,i,i,i,i,i,i]); }
  for (var i = 0; i < n; i++) { anchors.push({child: kids[i]}); junk.push([i,i,i,i,i,i,i,i]); }
  kids = null; junk = null;
}
function touch() {
  for (var i = 0; i < anchors.length; i++)
    if (anchors[i].child.m !== i) throw new Error("CORRUPT at " + i);
  return anchors.length;
}
`);

function makeEdges() {
  for (var i = 0; i < 20; i++) /abc/.exec("abcdefghijklmnopqrstuvwxyz0123456789" + i);
}

const N = 120000;
g.eval("build(" + N + ")");

for (var iter = 0; iter < 8; iter++) {
  gc();                       // blacken all live cells; bits persist
  g.eval("" + iter);          // re-pick victim zone's current chunk
  makeEdges();                // string cell edges in store buffer
  startgc(1);                 // Prepare; background unmark task walking
  var st0 = gcstate();
  for (var r = 0; r < 10; r++) {
    minorgc();                // each string edge -> forEachNonEmptyChunk -> clearCurrentChunk
    if (gcstate() != "Prepare") break;
    g.eval("0;" + iter + ";" + r);
    makeEdges();
  }
  var rounds = r;
  gczeal(15);                 // CheckHeapAfterGC at the end of this GC
  while (gcstate() != "NotActive") gcslice(50000);
  gczeal(0);
  print("iter " + iter + " state0=" + st0 + " rounds=" + rounds +
        " touch=" + g.eval("touch()"));
}
print("done - no corruption observed");

Run: js --fuzzing-safe repro.js crashes within ~1-7 iterations; js --fuzzing-safe --no-threads repro.js exits 0.

Jon, could you take a look at this newer sec-high bug when you have a chance.

Blocks: sm-security
Severity: -- → S2
Flags: needinfo?(jcoppeard)

This is debug only so not security sensitive.

Group: javascript-core-security
Flags: needinfo?(jcoppeard)

Steve, I found this comment/assertion about string deduplication. Is this still true?

    // Nursery string deduplication requires all tenured string -> nursery
    // string edges to be registered with the whole cell buffer in order to
    // correctly set the non-deduplicatable bit.

https://searchfox.org/firefox-main/source/js/src/gc/Tenuring.cpp#827-831

The assertion is a bit annoying because it wants to walk all the runtime's chunks to check the source of the edge. Also it won't catch edges that in memory owned by tenured cells that is not in the tenured heap directly.

Flags: needinfo?(sphink)

The fix for this problem is to assert background unmarking isn't running when we iterate chunks (we already finish GC when going via the public functions to iterate chunks) and do something else for this assertion.

No longer blocks: sm-security

(In reply to Jon Coppeard (:jonco) from comment #3)

Steve, I found this comment/assertion about string deduplication. Is this still true?

    // Nursery string deduplication requires all tenured string -> nursery
    // string edges to be registered with the whole cell buffer in order to
    // correctly set the non-deduplicatable bit.

https://searchfox.org/firefox-main/source/js/src/gc/Tenuring.cpp#827-831

The assertion is a bit annoying because it wants to walk all the runtime's chunks to check the source of the edge. Also it won't catch edges that in memory owned by tenured cells that is not in the tenured heap directly.

The assertion is valid, the reason for it is not. tenured string -> nursery string edges must be in the whole cell buffer rather an edge-only buffer because if the base nursery string's char data pointer is modified, then the dependent tenured string's chars pointer must also be modified. The non-deduplicatable bit isn't even set for this anymore.

Also, this assertion isn't just for deduplication anymore. It's needed for whenever a base string's chars move, which can happen when replacing strings in the atoms cache with their atoms during tenuring.

My try at an updated comment:

    // If a tenured dependent string's base moves its chars, the dependent
    // string must update its own chars pointer. This requires processing the
    // tenured dependent string via the whole cell buffer, since just an edge
    // pointer is not enough to find the pointing-from JSString to update.

I am looking at how to do the assertion differently, but I think I'll have to discuss during the meeting today. Er, in 2 minutes.

Severity: S2 → S3
Priority: -- → P1
Assignee: nobody → sphink
Status: NEW → ASSIGNED
Pushed by sfink@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/6137a655be6b https://hg.mozilla.org/integration/autoland/rev/4b62759cceb7 Avoid clearing the current chunk for isPointerWithinTenuredCell() check r=jonco
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → 154 Branch
Flags: needinfo?(sphink)

Please add an ESR153 uplift request on this when you get a chance.

Flags: needinfo?(sphink)

firefox-esr153 Uplift Approval Request

  • User impact if declined/Reason for urgency: Requesting uplift to fix intermittent test failures.
  • Code covered by automated testing?: yes
  • Fix verified in Nightly?: yes
  • Needs manual QE testing?: no
  • Steps to reproduce for manual QE testing:
  • Risk associated with taking this patch: low
  • Explanation of risk level: The affected code is only used by assertions and testing functions. This has been on central for four months with no regressions.
  • String changes made/needed?: None.
  • Is Android affected?: yes
Attachment #9668964 - Flags: approval-mozilla-esr153?
Flags: needinfo?(sphink)
Attachment #9668964 - Flags: approval-mozilla-esr153? → approval-mozilla-esr153+
QA Whiteboard: [qa-triage-done-c162/b161]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: