Open Bug 2078776 Opened 1 month ago Updated 1 month ago

Assertion failure: isThrowingOutOfMemory() in JSContext::recoverFromOutOfMemory via tryAttachSpecializedFunctionBind with near-maximum-length function name

Categories

(Core :: JavaScript Engine: JIT, defect, P3)

defect

Tracking

()

UNCONFIRMED

People

(Reporter: bugmon, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: assertion, testcase)

Attachments

(3 files)

InlinableNativeIRGenerator::tryAttachSpecializedFunctionBind treats every failure of BoundFunctionObject::initTemplateSlotsForSpecializedBind as out-of-memory and calls cx_->recoverFromOutOfMemory(). That callee builds "bound " + targetName in a StringBuilder, and when the target name is at least JSString::MAX_LENGTH - 6 characters long the builder's growth check reports JSMSG_ALLOC_OVERFLOW (an InternalError, not OOM). recoverFromOutOfMemory then fails MOZ_ASSERT(isThrowingOutOfMemory()) with that exception pending.

A script author can abort a debug build of SpiderMonkey with an assertion failure by calling Function.prototype.bind, from a call site that attaches its first inline cache stub, on a function whose inferred name is within six characters of the maximum string length. The testcase needs roughly 3 GB of memory. On the non-debug ASAN shell the same testcase exits normally with no sanitizer report: the pending exception is cleared and the bind call throws "InternalError: allocation size overflow" to script, so no memory corruption was demonstrated.

Build Info

Affected Code

File: js/src/jit/CacheIR.cpp, line 12804-12808

  if (!templateObj->initTemplateSlotsForSpecializedBind(
          cx_, numBoundArgs, targetIsConstructor, targetLength, targetName)) {
    cx_->recoverFromOutOfMemory();
    return AttachDecision::NoAction;
  }

File: js/src/vm/JSContext.cpp, line 734-739

void JSContext::recoverFromOutOfMemory() {
  if (isExceptionPending()) {
    MOZ_ASSERT(isThrowingOutOfMemory());
    clearPendingException();
  }
}

File: js/src/vm/BoundFunctionObject.cpp, line 487-504

bool BoundFunctionObject::initTemplateSlotsForSpecializedBind(
    JSContext* cx, uint32_t numBoundArgs, bool targetIsConstructor,
    uint32_t targetLength, JSAtom* targetName) {
  size_t len = 0;
  if (targetLength > numBoundArgs) {
    len = targetLength - numBoundArgs;
  }

  JSAtom* name = AppendBoundFunctionPrefix(cx, targetName);
  if (!name) {
    return false;
  }

  initFlags(numBoundArgs, targetIsConstructor);
  initLength(len);
  initName(name);
  return true;
}

File: js/src/vm/BoundFunctionObject.cpp, line 215-222

  StringBuilder sb(cx);
  if (!sb.append("bound ") || !sb.append(str)) {
    return nullptr;
  }
  JSAtom* atom = sb.finishAtom();
  if (!atom) {
    return nullptr;
  }

File: js/src/util/StringBuilder.h, line 98-108

  template <size_t EltSize>
  static size_t computeGrowth(size_t oldElts, size_t incr) {
    // Make a best effort to catch runaway string growth. Since this method
    // almost always returns more than the requested size (and the string can
    // then grow into it) it's unlikely we will catch this immediately.
    if (oldElts + incr >= size_t(JSString::MAX_LENGTH)) {
      return 0;
    }
    return detail::GrowEltsAggressively<EltSize>(oldElts, incr);
  }

File: mfbt/Vector.h, line 1090-1095

    newCap = detail::ComputeGrowth<AP, sizeof(T)>(mLength, aIncr, true);
    if (MOZ_UNLIKELY(newCap == 0)) {
      this->reportAllocOverflow();
      return false;
    }

The caller assumes OOM is the only failure mode of initTemplateSlotsForSpecializedBind. The callee can also fail with an allocation-overflow error: AppendBoundFunctionPrefix appends the target name to a StringBuilder whose alloc policy returns 0 from computeGrowth once the length would reach JSString::MAX_LENGTH, and Vector::growStorageBy turns that into reportAllocOverflow(), which reports JSMSG_ALLOC_OVERFLOW via JSContext::reportAllocOverflow (js/src/vm/JSContext.cpp:741-745). JS::MaxStringLength is (1 << 30) - 2 (js/public/String.h:385). Measured threshold on the ASAN shell with JITs disabled: a 1073741815-character name binds successfully (bound name length 1073741821); 1073741816 and above throw "InternalError: allocation size overflow", matching the >= comparison in computeGrowth rather than the > comparison in JSString::validateLength.

Exploit Chain

  1. Script creates a string of 1073741820 characters with "a".repeat(), which is below JS::MaxStringLength (1073741822).
  2. Script uses the string as a computed property key for an anonymous function expression, so the function gets that string as its inferred name and hasResolvedName() remains false.
  3. Script warms a function f 100 times with the bind branch not taken, so f runs with inline caches while its t.bind() call site has no stub attached.
  4. Script calls f with the long-named function and the bind branch taken; DoCallFallback runs CallIRGenerator::tryAttachStub -> tryAttachFunctionBind -> tryAttachSpecializedFunctionBind, which passes isFirstStub() and the other eligibility checks.
  5. initTemplateSlotsForSpecializedBind calls AppendBoundFunctionPrefix; appending the name to the StringBuilder reaches the length limit and reports JSMSG_ALLOC_OVERFLOW, returning false.
  6. tryAttachSpecializedFunctionBind calls cx_->recoverFromOutOfMemory(); with a non-OOM exception pending, MOZ_ASSERT(isThrowingOutOfMemory()) fails and the debug shell aborts. In the non-debug ASAN shell the exception is cleared instead and the bind call throws InternalError to script.

Steps to Reproduce

  1. Use an unmodified source tree and build the SpiderMonkey debug shell with /app/mozconfigs/mozconfig.js.linux.debug.
  2. Run testcase.js with js_shell_evaluator using js_binary=/firefox/obj-js-debug/dist/bin/js and timeout 180 (equivalent to js --fuzzing-safe testcase.js); no extra flags, prefs, environment variables or patches. About 3 GB of free memory is needed.
  3. Observe Assertion failure: isThrowingOutOfMemory(), at js/src/vm/JSContext.cpp:736 with JSContext::recoverFromOutOfMemory called from InlinableNativeIRGenerator::tryAttachSpecializedFunctionBind (js/src/jit/CacheIR.cpp:12806).

Security Impact

  • Severity: Low
  • Attacker capability: Abort a debug build of the JS engine with an assertion failure from plain script. On the non-debug ASAN shell the same testcase exits 0 with no sanitizer report and the bind call throws InternalError to script; no memory corruption or information disclosure was demonstrated.
  • Preconditions: Ability to run script that allocates a string of about 2^30 characters (roughly 3 GB of memory for the testcase). The assertion is only compiled into debug builds.

ASAN Report

[11617] Assertion failure: isThrowingOutOfMemory(), at js/src/vm/JSContext.cpp:736
#01: JSContext::recoverFromOutOfMemory()[/firefox/obj-js-debug/dist/bin/js +0x4078fc0]
#02: ???[/firefox/obj-js-debug/dist/bin/js +0x38367d5]
#03: ???[/firefox/obj-js-debug/dist/bin/js +0x3837690]
#04: ???[/firefox/obj-js-debug/dist/bin/js +0x383abae]
#05: ???[/firefox/obj-js-debug/dist/bin/js +0x383ca1f]
#06: ???[/firefox/obj-js-debug/dist/bin/js +0x3841c92]
#07: ???[/firefox/obj-js-debug/dist/bin/js +0x324c4d2]
#08: ??? (???:???)
UndefinedBehaviorSanitizer:DEADLYSIGNAL
==11617==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x000000000000 (pc 0x556da0980fd8 bp 0x7eb56b031d10 sp 0x7eb56b031d00 T11617)
==11617==The signal is caused by a READ memory access.
==11617==Hint: address points to the zero page.
    #0 0x556da0980fd8 in MOZ_CrashSequence(void*, long) /firefox/obj-js-debug/dist/include/mozilla/Assertions.h:261:3
    #1 0x556da0980fd8 in JSContext::recoverFromOutOfMemory() /firefox/obj-js-debug/js/src/vm/./../../../../js/src/vm/JSContext.cpp:736:5
    #2 0x556da013e7d4 in js::jit::InlinableNativeIRGenerator::tryAttachSpecializedFunctionBind(JS::Handle<JSObject*>, JS::Handle<js::BoundFunctionObject*>) /firefox/obj-js-debug/js/src/jit/./../../../../js/src/jit/CacheIR.cpp:12806:10
    #3 0x556da013f68f in js::jit::InlinableNativeIRGenerator::tryAttachFunctionBind() /firefox/obj-js-debug/js/src/jit/./../../../../js/src/jit/CacheIR.cpp:12896:14
    #4 0x556da0142bad in js::jit::CallIRGenerator::tryAttachInlinableNative(JS::Handle<JSFunction*>, js::jit::CallFlags) /firefox/obj-js-debug/js/src/jit/./../../../../js/src/jit/CacheIR.cpp:13198:20
    #5 0x556da0144a1e in js::jit::CallIRGenerator::tryAttachCallNative(JS::Handle<JSFunction*>) /firefox/obj-js-debug/js/src/jit/./../../../../js/src/jit/CacheIR.cpp:14069:16
    #6 0x556da0149c91 in js::jit::CallIRGenerator::tryAttachStub() /firefox/obj-js-debug/js/src/jit/./../../../../js/src/jit/CacheIR.cpp:14814:10
    #7 0x556d9fb544d1 in js::jit::DoCallFallback(JSContext*, js::jit::BaselineFrame*, js::jit::ICFallbackStub*, unsigned int, JS::Value*, JS::MutableHandle<JS::Value>) /firefox/obj-js-debug/js/src/jit/./../../../../js/src/jit/BaselineIC.cpp:1671:17
    #8 0x10a0cca5700e  ([anon:js-executable-memory]+0xc00e)

UndefinedBehaviorSanitizer can not provide additional info.
SUMMARY: UndefinedBehaviorSanitizer: SEGV /firefox/obj-js-debug/dist/include/mozilla/Assertions.h:261:3 in MOZ_CrashSequence(void*, long)
==11617==ABORTING
Attached patch fix.patch — — Splinter Review
Attached file crash_stack.txt —
Attached file testcase.js —

This bug was filed automatically by an AI-powered security testing tool. The analysis, testcase, and any suggested patches were generated by an AI model and have NOT been verified by a human.

Please review all details carefully before triaging. The issue was confirmed by an independent verifier, but for non-ASAN issues, the verifier has been made less strict. Additionally, the root cause analysis and suggested fix may contain errors.

Group: core-security → javascript-core-security
Blocks: sm-security
Severity: -- → S3
Priority: -- → P3

This isn't security sensitive. The proposed patch will compile to the exact same code in a release build.

Group: javascript-core-security
Keywords: sec-other
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: