784187 - crash on null deref in js::Vector<js::types::CompilerOutput, 0ul, js::TempAllocPolicy>::operator[]

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, defect)

Description

[Daniel Holbert [:dholbert]]

Attachment: backtrace in GDB

My mozilla-inbound debug build is crashing when I run mochitests in layout/style like so:
  TEST_PATH=layout/style/ make mochitest-plain
with null-pointer crash in JS engine code.

Backtrace attached.  Here are the top few lines of the backtrace (from GDB):
{
Program received signal SIGSEGV, Segmentation fault.
0x00002ae108f46b38 in js::Vector<js::types::CompilerOutput, 0ul, js::TempAllocPolicy>::operator[] (this=0x0, i=3) at ./../../dist/include/js/Vector.h:342
(gdb) bt
#0  0x00002ae108f46b38 in js::Vector<js::types::CompilerOutput, 0ul, js::TempAllocPolicy>::operator[] (this=0x0, i=3) at ./../../dist/include/js/Vector.h:342
#1  0x00002ae108f331fc in js::types::RecompileInfo::compilerOutput (this=0x2ae124b63de8, cx=0x2ae1261cdc10) at ../../../mozilla/js/src/jsinferinlines.h:84
#2  0x00002ae108f39ca9 in js::types::TypeCompartment::addPendingRecompile (this=0x2ae123fc1448, cx=0x2ae1261cdc10, info=...) at ../../../mozilla/js/src/jsinfer.cpp:2109
#3  0x00002ae108f37e10 in TypeConstraintFreeze::newType (this=0x2ae124b63dd0, cx=0x2ae1261cdc10, source=0x2ae124b5e868, type=...) at ../../../mozilla/js/src/jsinfer.cpp:1376
}

Comment 1

[Daniel Holbert [:dholbert]]

Attachment: backtrace in GDB

(sorry, that last attachment had an extra copy of the backtrace -- pasted in too much GDB output. This one's more concise)

Comment 2

[Daniel Holbert [:dholbert]]

The crash here is from this chunk, at level #1 in the backtrace (jsinferinlines.h:84):

> inline CompilerOutput*
> RecompileInfo::compilerOutput(JSContext *cx) const
> {
>     return &(*cx->compartment->types.constrainedOutputs)[outputIndex];
> }

As shown in the attached gdb log (attachment 653552 [details]), cx->compartment->types.constrainedOutputs is null here, and we're dereferencing it, which makes us crash.

Comment 3

[Daniel Holbert [:dholbert]]

Ah, my m-i build was at revision https://hg.mozilla.org/integration/mozilla-inbound/rev/1906fe4159fe, which has since been backed out for causing failures. I'm pretty sure this was just one of the issues that it caused.  After updating my tree, I haven't crashed (yet) in a run through the layout/style mochitests, so I think the backout fixed this.

Marking dependency on that bug and resolving as WORKSFORME.