Closed Bug 734196 Opened 8 years ago Closed 8 years ago

Crash due to call stack overflow [@ js::ArrayBuffer::obj_trace]

Categories

(Core :: JavaScript Engine, defect, critical)

x86_64
Linux
defect
Not set
critical

Tracking

()

RESOLVED FIXED
mozilla13

People

(Reporter: decoder, Assigned: terrence)

References

(Blocks 1 open bug)

Details

(Keywords: crash, testcase, Whiteboard: js-triage-needed)

Attachments

(1 file)

The following test crashes on mozilla-central revision 453d5c733caa (options -m -n -a):


var x = new ArrayBuffer(2);
gczeal(4);
actual = [].concat(x).toString();
var count2 = countHeap();


In a debug build, the recursion looks like this:

#7  0x00000000005b7809 in js::ArrayBuffer::obj_trace (trc=0x7ffff7fae220, obj=0x7ffff61031a0) at /srv/repos/mozilla-central/js/src/jstypedarray.cpp:334
#8  0x0000000000445044 in js::ObjectImpl::privateWriteBarrierPre (this=0x7ffff61031a0, old=0x7ffff6103238) at ../vm/ObjectImpl-inl.h:105
#9  0x0000000000445653 in JSObject::setPrivate (this=0x7ffff61031a0, data=0x7ffff6105080) at ../jsobjinlines.h:119
#10 0x00000000005b781c in js::ArrayBuffer::obj_trace (trc=0x7ffff7fae220, obj=0x7ffff61031a0) at /srv/repos/mozilla-central/js/src/jstypedarray.cpp:335
#11 0x0000000000445044 in js::ObjectImpl::privateWriteBarrierPre (this=0x7ffff61031a0, old=0x7ffff6103238) at ../vm/ObjectImpl-inl.h:105
#12 0x0000000000445653 in JSObject::setPrivate (this=0x7ffff61031a0, data=0x7ffff6105080) at ../jsobjinlines.h:119
#13 0x00000000005b781c in js::ArrayBuffer::obj_trace (trc=0x7ffff7fae220, obj=0x7ffff61031a0) at /srv/repos/mozilla-central/js/src/jstypedarray.cpp:335

while in an opt build, it's just setPrivate and js::ArrayBuffer::obj_trace and the rest seems inlined.
This looks similar to bug 733372, Terrence. I think the best fix is to call setPrivate only if IS_GC_MARKING_TRACER(trc). Hopefully that fix would have worked for bug 733372 as well.
Assignee: general → terrence
I audited all call sites of setPrivate and modified all of the ones that could possibly exhibit this problem.  This should fix all possible instances of this particular problem now and in the future, without hindering our ability to move objects as we will need to later.
Attachment #604209 - Flags: review?(wmccloskey)
Comment on attachment 604209 [details] [diff] [review]
v0: A complete fix.

Thanks!
Attachment #604209 - Flags: review?(wmccloskey) → review+
https://hg.mozilla.org/mozilla-central/rev/1587745454a6
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla13
A testcase for this bug was automatically identified at js/src/jit-test/tests/basic/bug734196.js.
Flags: in-testsuite+
You need to log in before you can comment on or make changes to this bug.