1666856 - Hit MOZ_CRASH(Association for 0x14405c91dc70 NativeIterator has different size: expected 0x88 but got 0x58) at gc/Scheduling.cpp:601

Reporter

Description

•

4 years ago

The following testcase crashes on mozilla-central revision 20200923-7927a1705247 (debug build, run with --fuzzing-safe --cpu-count=2 --ion-offthread-compile=off --more-compartments):

evalInWorker(`
  Object.defineProperty(this, "fuzzutils", {  });
  newGlobalx = newGlobal;
  newGlobal = function(o) {
    let g = newGlobalx(o);
    g['evalInWorker'] = evalInWorker;
    return g;
  }
  var lfOffThreadGlobal = newGlobal();
  for (lfLocal in this)
    if (!(lfLocal in lfOffThreadGlobal))
      lfOffThreadGlobal[lfLocal] = this[lfLocal];
  var g86 = newGlobal({return: true});
  var g53 = newGlobal({newCompartment: true});
  var g86 = newGlobal({return: true});
  g53.eval("function f(obj, cls) { debugger; }");
  g53.eval("f(Date, 'Function');");
  function assertEq() {
    var arr = [];
    arr[10_000] = 1;
    arr[10_001] = 1;
    for (var static  in arr)
      assertEq(arr.length, 10_002);
  }
  g66 = newGlobal({newCompartment: true});
  g66.eval("function* gen() {}");
  try { evaluate(\`
    let g66 = newGlobal({newCompartment: true});
    g66.eval("function* gen() {}");
    actual = new Float32Array(0x1000);
    expected = new Float32Array(0x1000);
    assertEq(actual, expected)
  \`); } catch(exc) {}
  try { evaluate(\`
    assertEq(actual, expected)
  \`); } catch(exc) {}
  const g26 = newGlobal();
  const dbg = new Debugger();
  const sources = dbg.findSources();
  assertEq(sources.filter(s59 => s59.url == "a.js").length, 1);
`);

Backtrace:

received signal SIGSEGV, Segmentation fault.
#0  MOZ_Crash (aFilename=<optimized out>, aLine=601, aReason=0x555558510d10 <sPrintfCrashReason> "Association for 0x14405c91dc70 NativeIterator has different size: expected 0x88 but got 0x58") at dist/include/mozilla/Assertions.h:254
#1  js::gc::MemoryTracker::untrackGCMemory (this=<optimized out>, cell=0x14405c91dc70, nbytes=88, use=<optimized out>) at js/src/gc/Scheduling.cpp:598
#2  0x0000555556cbc606 in js::RemoveCellMemory (cell=0x7ffff7105770 <_IO_stdfile_2_lock>, nbytes=<optimized out>, use=<optimized out>, wasSwept=<optimized out>) at js/src/gc/ZoneAllocator.h:322
#3  js::RemoveCellMemory (cell=0x7ffff7105770 <_IO_stdfile_2_lock>, nbytes=<optimized out>, use=<optimized out>, wasSwept=<optimized out>) at js/src/gc/ZoneAllocator.h:328
#4  JSFreeOp::removeCellMemory (this=<optimized out>, cell=0x7ffff7105770 <_IO_stdfile_2_lock>, nbytes=<optimized out>, use=<optimized out>) at js/src/gc/FreeOp-inl.h:53
#5  JSFreeOp::free_ (this=<optimized out>, cell=0x7ffff7105770 <_IO_stdfile_2_lock>, p=0x7ffff4efee50, nbytes=<optimized out>, use=<optimized out>) at js/src/gc/FreeOp-inl.h:17
#6  0x00005555575f666e in JSClass::doFinalize (this=<optimized out>, fop=0x7ffff6965ee0, obj=0x14405c91dc70) at dist/include/js/Class.h:785
#7  JSObject::finalize (this=0x14405c91dc70, fop=0x7ffff6965ee0) at js/src/vm/JSObject-inl.h:100
#8  0x00005555575f5cff in js::gc::Arena::finalize<JSObject> (this=<optimized out>, this@entry=0x14405c91d000, fop=fop@entry=0x7ffff6965ee0, thingKind=thingKind@entry=js::gc::AllocKind::OBJECT2_BACKGROUND, thingSize=<optimized out>, thingSize@entry=48) at js/src/gc/GC.cpp:480
#9  0x00005555575f5a0a in FinalizeTypedArenas<JSObject> (fop=0x7ffff6965ee0, src=0x7ffff6964e50, dest=..., thingKind=<optimized out>, budget=...) at js/src/gc/GC.cpp:535
#10 0x00005555575b8964 in FinalizeArenas (fop=fop@entry=0x7ffff6965ee0, src=src@entry=0x7ffff6964e50, dest=..., thingKind=thingKind@entry=js::gc::AllocKind::OBJECT2_BACKGROUND, budget=...) at js/src/gc/GC.cpp:563
#11 0x00005555575b7c84 in js::gc::ArenaLists::backgroundFinalize (fop=fop@entry=0x7ffff6965ee0, listHead=0x0, empty=empty@entry=0x7ffff6965eb8) at js/src/gc/GC.cpp:2848
#12 0x00005555575bda74 in js::gc::GCRuntime::sweepBackgroundThings (this=this@entry=0x7ffff5995740, zones=...) at js/src/gc/GC.cpp:3415
#13 0x00005555575be826 in js::gc::GCRuntime::sweepFromBackgroundThread (this=0x7ffff5995740, lock=...) at js/src/gc/GC.cpp:3484
#14 0x00005555575be75a in js::gc::BackgroundSweepTask::run (this=0x7ffff5996f10, lock=...) at js/src/gc/GC.cpp:3475
#15 0x00005555575ec297 in js::GCParallelTask::runTask (this=this@entry=0x7ffff5996f10, lock=...) at js/src/gc/GCParallelTask.cpp:145
#16 0x00005555575bcbe2 in js::GCParallelTask::runFromMainThread (this=this@entry=0x7ffff5996f10) at js/src/gc/GCParallelTask.cpp:120
#17 0x00005555575be5ad in js::gc::GCRuntime::queueZonesAndStartBackgroundSweep (this=<optimized out>, this@entry=0x7ffff5995740, zones=...) at js/src/gc/GC.cpp:3467
#18 0x00005555575d3542 in js::gc::GCRuntime::endSweepingSweepGroup (this=0x7ffff5995740, fop=<optimized out>, budget=...) at js/src/gc/GC.cpp:5526
#19 0x0000555557615201 in sweepaction::SweepActionSequence::run (this=0x7ffff5a9a7e0, args=...) at js/src/gc/GC.cpp:6083
#20 0x00005555576045d7 in sweepaction::SweepActionForEach<js::gc::SweepGroupsIter, JSRuntime*>::run (this=0x7ffff58df370, args=...) at js/src/gc/GC.cpp:6118
#21 0x00005555575d688f in js::gc::GCRuntime::performSweepActions (this=this@entry=0x7ffff5995740, budget=...) at js/src/gc/GC.cpp:6250
#22 0x00005555575db983 in js::gc::GCRuntime::incrementalSlice (this=this@entry=0x7ffff5995740, budget=..., gckind=..., reason=<optimized out>, reason@entry=JS::GCReason::DESTROY_RUNTIME, session=...) at js/src/gc/GC.cpp:6831
#23 0x00005555575de40a in js::gc::GCRuntime::gcCycle (this=this@entry=0x7ffff5995740, nonincrementalByAPI=true, budget=..., gckind=..., reason=reason@entry=JS::GCReason::DESTROY_RUNTIME) at js/src/gc/GC.cpp:7238
#24 0x00005555575dff79 in js::gc::GCRuntime::collect (this=0x7ffff5995740, nonincrementalByAPI=false, budget=..., gckindArg=..., reason=JS::GCReason::DESTROY_RUNTIME) at js/src/gc/GC.cpp:7473
#25 0x0000555557598e2d in js::gc::GCRuntime::gc (this=0x7ffff7104540 <_IO_2_1_stderr_>, gckind=<optimized out>, reason=-152870179) at js/src/gc/GC.cpp:7549
#26 0x00005555570ceccb in JSRuntime::destroyRuntime (this=0x7ffff5995000) at js/src/vm/Runtime.cpp:288
#27 0x0000555556fde4b2 in js::DestroyContext (cx=0x7ffff60ba000) at js/src/vm/JSContext.cpp:223
#28 0x0000555556ba8a95 in WorkerMain(WorkerInput*)::$_8::operator()() const (this=<optimized out>) at js/src/shell/js.cpp:4153
#29 mozilla::ScopeExit<WorkerMain(WorkerInput*)::$_8>::~ScopeExit() (this=<optimized out>) at dist/include/mozilla/ScopeExit.h:106
#30 WorkerMain (input=<optimized out>) at js/src/shell/js.cpp:4227
#31 0x0000555556ba8f24 in js::detail::ThreadTrampoline<void (&)(WorkerInput*), WorkerInput*&>::callMain<0ul> (this=0x7ffff5979200) at js/src/threading/Thread.h:217
#32 js::detail::ThreadTrampoline<void (&)(WorkerInput*), WorkerInput*&>::Start (aPack=0x7ffff5979200) at js/src/threading/Thread.h:206
#33 0x00007ffff7bc16ba in start_thread (arg=0x7ffff6967700) at pthread_create.c:333
#34 0x00007ffff6e4641d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
rax	0x555558510d08	93825042287880
rbx	0x555558510d10	93825042287888
rcx	0x7ffff6e362dd	140737335485149
rdx	0x0	0
rsi	0x7ffff7105770	140737338431344
rdi	0x7ffff7104540	140737338426688
rbp	0x7ffff6964bf0	140737330433008
rsp	0x7ffff6964b60	140737330432864
r8	0x7ffff7105770	140737338431344
r9	0x7ffff6967700	140737330444032
r10	0x0	0
r11	0x0	0
r12	0x4	4
r13	0xffffffffffffff	72057594037927935
r14	0x58	88
r15	0x14405c91dc70	22266663525488
rip	0x55555765843b <js::gc::MemoryTracker::untrackGCMemory(js::gc::Cell*, unsigned long, js::MemoryUse)+571>
=> 0x55555765843b <js::gc::MemoryTracker::untrackGCMemory(js::gc::Cell*, unsigned long, js::MemoryUse)+571>:	movl   $0x259,0x0
   0x555557658446 <js::gc::MemoryTracker::untrackGCMemory(js::gc::Cell*, unsigned long, js::MemoryUse)+582>:	callq  0x555556befa82 <abort()>

Christian Holler (:decoder)

Reporter

Comment 1

•

4 years ago

Attached file Testcase — Details

Jason Kratzer [:jkratzer]

Comment 2

•

4 years ago

Bugmon Analysis:
Verified bug as reproducible on mozilla-central 20200923212316-efc5aeff23bd.
The bug appears to have been introduced in the following build range:

Start: f156ef6b0f03d4c5762a740c3db927d814bfd6cc (20200612235618)
End: a3226c9856a8fd2338bc878fd60b092d2e1a7b9a (20200612235731)
Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=f156ef6b0f03d4c5762a740c3db927d814bfd6cc&tochange=a3226c9856a8fd2338bc878fd60b092d2e1a7b9a

Whiteboard: [bugmon:update,bisect] → [bugmon:update,bisected,confirmed]

Jason Orendorff [:jorendorff]

Assignee

Updated

•

4 years ago

Flags: needinfo?(jorendorff)

Priority: -- → P1

Jason Orendorff [:jorendorff]

Assignee

Comment 3

•

4 years ago

NativeIterator::NativeIterator initializes guardsEnd_ to a pointer that indicates the guards vector is empty.
It then tracks enough memory to hold numGuards guards (in this case 3).
But it can then OOM before populating guards. The resulting half-initialized NativeIterator believes that its numGuards() is 0.

There is also another comment here about "a location not known to the GC".

The test is picky, but it should be possible to write a better one using oomAtAllocation. In theory we don't need six globals and extra compartments and Ion and runWorkerThread.

Jason Orendorff [:jorendorff]

Assignee

Comment 4

•

4 years ago

•

Edited

This asserts after 2 seconds for me in a debug build.

let arr = [];
arr[10000] = 1;

let success = false;
for (let n = 0; !success; n++) {
  try {
    oomAtAllocation(n);
    for (var key in arr) {}
    success = !resetOOMFailure();
  } catch (oom) {}
  gc();
}

Jason Orendorff [:jorendorff]

Assignee

Comment 5

•

4 years ago

Attached file Bug 1666856 - Leave NativeIterator in a valid-enough state on OOM during initialization. r?jonco. — Details

Phabricator Automation

Updated

•

4 years ago

Assignee: nobody → jorendorff

Status: NEW → ASSIGNED

Jason Orendorff [:jorendorff]

Assignee

Comment 6

•

4 years ago

Now, with the test below, I get an occasional assertion:

let i = 10000;
oomTest(() => {
  print(i - 10000);
  let arr = [];
  arr[i++] = 1;
  for (var key in arr) {}
});

It happens in the debugger, also not reliably but about 25% of the time. Stack trace is:

Assertion failure: IsCellPointerValid(*edge), at /Users/jorendorff/work/mozilla-central/js/src/gc/Marking.cpp:3179
Process 13311 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000100e7b71c js`js::gc::StoreBuffer::CellPtrEdge<JSObject>::trace(this=0x00000001094722a8, mover=0x00007ffeefbfe608) const at Marking.cpp:3179:3
   3176	    return;
   3177	  }
   3178	
-> 3179	  MOZ_ASSERT(IsCellPointerValid(*edge));
   3180	  MOZ_ASSERT((*edge)->getTraceKind() == JS::MapTypeToTraceKind<T>::kind);
   3181	
   3182	  if (std::is_same_v<JSString, T>) {
Target 0: (js) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000100e7b71c js`js::gc::StoreBuffer::CellPtrEdge<JSObject>::trace(this=0x00000001094722a8, mover=0x00007ffeefbfe608) const at Marking.cpp:3179:3
    frame #1: 0x0000000100e7b65c js`js::gc::StoreBuffer::MonoTypeBuffer<js::gc::StoreBuffer::CellPtrEdge<JSObject> >::trace(this=0x0000000107e29660, mover=0x00007ffeefbfe608) at Marking.cpp:2962:15
    frame #2: 0x0000000100e91a7a js`js::gc::StoreBuffer::traceCells(this=0x0000000107e29528, mover=0x00007ffeefbfe608) at StoreBuffer.h:528:16
    frame #3: 0x0000000100e902d3 js`js::Nursery::doCollection(this=0x0000000107e29080, reason=DESTROY_RUNTIME, tenureCounts=0x00007ffeefbfe7a8) at Nursery.cpp:1140:6
    frame #4: 0x0000000100e8fad1 js`js::Nursery::collect(this=0x0000000107e29080, kind=GC_NORMAL, reason=DESTROY_RUNTIME) at Nursery.cpp:1003:31
    frame #5: 0x0000000100defd29 js`js::gc::GCRuntime::collectNursery(this=0x0000000107e26760, kind=GC_NORMAL, reason=DESTROY_RUNTIME, phase=EVICT_NURSERY_FOR_MAJOR_GC) at GC.cpp:7690:13
    frame #6: 0x0000000100def070 js`js::gc::GCRuntime::gcCycle(this=0x0000000107e26760, nonincrementalByAPI=true, budget=SliceBudget @ 0x00007ffeefbfea90, gckind=0x00007ffeefbfeba0, reason=DESTROY_RUNTIME) at GC.cpp:7202:5
    frame #7: 0x0000000100df0ad0 js`js::gc::GCRuntime::collect(this=0x0000000107e26760, nonincrementalByAPI=true, budget=SliceBudget @ 0x00007ffeefbfebc0, gckindArg=0x00007ffeefbfec20, reason=DESTROY_RUNTIME) at GC.cpp:7473:9
    frame #8: 0x0000000100db213c js`js::gc::GCRuntime::gc(this=0x0000000107e26760, gckind=GC_NORMAL, reason=DESTROY_RUNTIME) at GC.cpp:7549:3
    frame #9: 0x00000001006f978e js`JSRuntime::destroyRuntime(this=0x0000000107e26000) at Runtime.cpp:288:8
    frame #10: 0x00000001005683b8 js`js::DestroyContext(cx=0x0000000107e34000) at JSContext.cpp:223:7
    frame #11: 0x00000001002b3665 js`JS_DestroyContext(cx=0x0000000107e34000) at jsapi.cpp:401:55
    frame #12: 0x00000001000d3b28 js`main::$_1::operator(this=0x00007ffeefbfef08)() const at js.cpp:11527:41
    frame #13: 0x00000001000d3b07 js`mozilla::ScopeExit<main::$_1>::~ScopeExit(this=0x00007ffeefbfef08) at ScopeExit.h:106:7
    frame #14: 0x0000000100017a85 js`mozilla::ScopeExit<main::$_1>::~ScopeExit(this=0x00007ffeefbfef08) at ScopeExit.h:104:16
    frame #15: 0x0000000100011c57 js`main(argc=2, argv=0x00007ffeefbff090, envp=0x00007ffeefbff0a8) at js.cpp:11653:1
    frame #16: 0x00007fff6cf1fcc9 libdyld.dylib`start + 1
(lldb) p *edge
(JSObject *) $0 = 0xe5e5e5e5e5e5e5e5

Jason Orendorff [:jorendorff]

Assignee

Comment 7

•

4 years ago

edge points to the field ni->objectBeingIterated_.value of a NativeIterator object that failed to be initialized and was freed.

Phabricator Automation

Updated

•

4 years ago

Attachment #9177712 - Attachment description: Bug 1666856 - Fix incorrect malloc bookkeeping on OOM initializing a NativeIterator. r?jonco. → Bug 1666856 - Leave NativeIterator in a valid-enough state on OOM during initialization. r?jonco.

Pulsebot

Comment 8

•

4 years ago

Pushed by jorendorff@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e8537a8ecd09
Leave NativeIterator in a valid-enough state on OOM during initialization. r=jonco.

Jason Orendorff [:jorendorff]

Assignee

Updated

•

4 years ago

Flags: needinfo?(jorendorff)

Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)

Comment 9

•

4 years ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/e8537a8ecd09

Status: ASSIGNED → RESOLVED

Closed: 4 years ago

status-firefox83: affected → fixed

Resolution: --- → FIXED

Target Milestone: --- → 83 Branch

Jason Kratzer [:jkratzer]

Comment 10

•

4 years ago

Bugmon Analysis:
Verified bug as fixed on rev mozilla-central 20200930092918-a58462ad649e.
Removing bugmon keyword as no further action possible.
Please review the bug and re-add the keyword for further analysis.

Status: RESOLVED → VERIFIED

status-firefox83: fixed → verified

Keywords: bugmon

Ryan VanderMeulen [:RyanVM]

Updated

•

4 years ago

status-firefox82: --- → wontfix

status-firefox-esr78: --- → unaffected

Flags: in-testsuite+

Regressed by: 1640284

BMO Automation

Updated

•

4 years ago

Has Regression Range: --- → yes

Testcase 4 years ago Christian Holler (:decoder) 1.24 KB, text/plain		Details
Bug 1666856 - Leave NativeIterator in a valid-enough state on OOM during initialization. r?jonco. 4 years ago Jason Orendorff [:jorendorff] 47 bytes, text/x-phabricator-request		Details \| Review