Closed Bug 2068387 Opened 2 months ago Closed 2 months ago

Assertion failure: zone->cellsToAssertNotGray().empty() in GCRuntime::checkGCStateNotInUse when gray bits are invalidated during gray marking

Categories

(Core :: JavaScript: GC, defect, P3)

defect

Tracking

()

RESOLVED FIXED
157 Branch
Tracking Status
firefox-esr140 --- unaffected
firefox-esr153 --- wontfix
firefox155 --- wontfix
firefox156 --- wontfix
firefox157 --- fixed

People

(Reporter: bugmon, Assigned: jonco)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: assertion, regression, testcase)

Attachments

(5 files, 1 external file, 1 obsolete file)

Found by a scan of commit 71d5988e4d1fa34dcd39fde9b5b8a2d52caeb165^..124287f870cdb59246ab42cbf2aa0c1e6cdf86ad. The scan that produced this finding was directed at the changes landed by that commit, which may or may not be its actual cause. Please judge independently whether they are really connected.

In DEBUG builds, GCRuntime::beginSweepingSweepGroup (js/src/gc/Sweeping.cpp:1816-1836) both checks and clears each zone's deferred gray-assert list inside if (areGrayBitsValid()), so when grayBitsValid is cleared after JS::AssertCellIsNotGray/DeferGrayMarkingCheck has already appended cells (GC.cpp:6083-6092) but before the sweep-group transition, zone->cellsToAssertNotGray() is never emptied and the stale Cell* pointers leak past the end of the collection. GCRuntime::checkGCStateNotInUse, reached from finishCollection, then aborts on MOZ_ASSERT(zone->cellsToAssertNotGray().empty()) at js/src/gc/GC.cpp:3902 (the same invariant is also asserted in beginMarkPhase at GC.cpp:3349); release builds are unaffected since the vector and both assertions are #ifdef DEBUG.

Exploit Chain

  1. gczeal(25); startgc(1): an incremental GC yields while gray marking; the function stored in grayRoot()[0] is marked gray and its zone is in Zone::MarkBlackAndGray.
  2. o.x = grayRoot()[0]: the write barrier calls JS::AssertCellIsNotGray on the gray function; because the zone is MarkBlackAndGray and the cell is not black, DeferGrayMarkingCheck appends the Cell* to zone->cellsToAssertNotGray() (GC.cpp:6083-6092).
  3. setGrayBitsInvalid(): GCRuntime::grayBitsValid becomes false (GC.cpp:4233-4237).
  4. finishgc(): the GC proceeds to beginSweepingSweepGroup; the if (areGrayBitsValid()) block at Sweeping.cpp:1820 is skipped, so the vector is neither checked nor cleared.
  5. finishCollection -> checkGCStateNotInUse asserts zone->cellsToAssertNotGray().empty() at GC.cpp:3902 and the shell aborts.

Affected Code

Steps to Reproduce

  1. Build the SpiderMonkey debug shell with /app/mozconfigs/mozconfig.js.linux.debug from an unmodified tree (build_dir /firefox/obj-js-debug).
  2. Run js_shell_evaluator with js_binary=/firefox/obj-js-debug/dist/bin/js (shell runs with --fuzzing-safe) and the testcase: gczeal(0); grayRoot()[0] = function f() {}; gczeal(25); startgc(1); var o = {}; o.x = grayRoot()[0]; setGrayBitsInvalid(); finishgc(); gczeal(0); gc();
  3. Observe: Assertion failure: zone->cellsToAssertNotGray().empty(), at js/src/gc/GC.cpp:3902 in GCRuntime::checkGCStateNotInUse <- finishCollection <- finishGC.
  4. Control: remove the setGrayBitsInvalid() line and the same script runs to completion without any assertion.
  5. No prefs, flags, or source patches are required.

Security Impact

  • Severity: Low
  • Attacker capability: None demonstrated beyond a DEBUG-build assertion abort. The vector, the deferral, and both assertions exist only under #ifdef DEBUG (js/src/gc/Zone.h:605-613, GC.cpp:6046, Sweeping.cpp:1816), so release builds contain no affected code. In DEBUG builds the retained Cell* pointers would be dereferenced (isMarkedGray/getTraceKind) in a later beginSweepingSweepGroup only if the earlier asserts at GC.cpp:3902 and GC.cpp:3349 were not hit, which they are.
  • Preconditions: DEBUG build of SpiderMonkey. A zone must be in MarkBlackAndGray with at least one cell deferred via AssertCellIsNotGray, and grayBitsValid must become false before the sweep-group transition. In the shell this is done with the fuzzing-safe testing functions gczeal(25), startgc, setGrayBitsInvalid and finishgc; outside testing functions the flag is cleared only on OOM in UnmarkGrayTracer (Marking.cpp:3486) or in resetIncrementalGC (GC.cpp:4123, 4155).

The full analysis report is attached (analysis.md).

Attached patch fix.patch — — Splinter Review
Attached file crash_stack.txt —
Attached file testcase.js —
Attached file progress.json —
Attached file analysis.md —

This bug was filed automatically by an AI-powered security testing tool. The analysis, testcase, and any suggested patches were generated by an AI model and have NOT been verified by a human.

Please review all details carefully before triaging. The issue was confirmed by an independent verifier, but for non-ASAN issues, the verifier has been made less strict. Additionally, the root cause analysis and suggested fix may contain errors.

Group: core-security → javascript-core-security
Blocks: sm-security
Severity: -- → S3
Priority: -- → P3
Attachment #9634615 - Attachment is obsolete: true
Assignee: nobody → jcoppeard

If the gray marking state is not valid we can't check these cells. We still
need to clear the list though.

:jonco, please update all release status flags to reflect the affected/unaffected/disabled state correctly. Which branches (beta, release, and/or ESR) are affected by this flaw?

Flags: needinfo?(jcoppeard)

This is debug-only code so this doesn't affect release builds and is not security sensitive.

Group: javascript-core-security
Keywords: sec-other
Flags: needinfo?(jcoppeard)
Keywords: regression
Regressed by: 2061758
Attachment #9635939 - Attachment description: (secure) → Bug 2068387 - Clear list of cells to check are not marked gray if we can't check them r?#spidermonkey-reviewers
Pushed by jcoppeard@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/559ee6c1707a https://hg.mozilla.org/integration/autoland/rev/35cee3b8c58c Clear list of cells to check are not marked gray if we can't check them r=spidermonkey-reviewers,dminor
Status: UNCONFIRMED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → 157 Branch

Set release status flags based on info from the regressing bug 2061758

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: