463789 - TM: Crash [@ nanojit::LirBuffer::validate()] - js1_5/Regress/regress-280769-[15].js - FAIL

Status: VERIFIED FIXED

(Core :: JavaScript Engine, defect)

Description

[Bob Clary [:bc] (inactive)]

Regression in js shell on tracemonkey first noticed 2008-11-06 10PM PT.
I'm not sure what is up with this. I've tried automatic bisection but due to build and mercurial failures can't get a good regression revision.

Running the tests js1_5/Regress/regress-280769-[15].js as I normally do results in the tests being loaded and begun to be run then an exiting with exit code 4 and messages about the test files not being found. This does not occur with other tests in the same directory.

bclary@bc-winxp02 /work/mozilla/mozilla.com/test.mozilla.com/www/tests/mozilla.org/js/js1_5/Regress
$ /work/mozilla/builds/1.9.1-tracemonkey-test/mozilla/js/src/WINNT5.1_DBG.OBJ/js -f ../../shell.js -f ../shell.js -f ./shell.js -f regress-280769-1.js -f ../../js-test-driver-end.js
BUGNUMBER: 280769
STATUS: Do not crash on overflow of 64K boundary of [] offset in regexp search string 
recorder: started(0), aborted(0), completed(0), different header(0), trees trashed(0), slot promoted(0), unstable loop variable(0), breaks(0), returns(0), unstableInnerCalls(0)
monitor: triggered(0), exits(0), type mismatch(0), global mismatch(0)
can't open ../../shell.js: No such file or directory
can't open ../shell.js: No such file or directory
can't open ./shell.js: No such file or directory
can't open regress-280769-1.js: No such file or directory
can't open ../../js-test-driver-end.js: No such file or directory

bclary@bc-winxp02 /work/mozilla/mozilla.com/test.mozilla.com/www/tests/mozilla.org/js/js1_5/Regress
$ /work/mozilla/builds/1.9.1-tracemonkey-test/mozilla/js/src/WINNT5.1_DBG.OBJ/js -f ../../shell.js -f ../shell.js -f ./shell.js -f regress-280769-5.js -f ../../js-test-driver-end.js
BUGNUMBER: 280769
STATUS: Do not overflow 64K string offset
recorder: started(0), aborted(0), completed(0), different header(0), trees trashed(0), slot promoted(0), unstable loop variable(0), breaks(0), returns(0), unstableInnerCalls(0)
monitor: triggered(0), exits(0), type mismatch(0), global mismatch(0)
can't open ../../shell.js: No such file or directory
can't open ../shell.js: No such file or directory
can't open ./shell.js: No such file or directory
can't open regress-280769-5.js: No such file or directory
can't open ../../js-test-driver-end.js: No such file or directory

loading the shell, attaching msvc debugger and loading the files manually gives an crash|exception at:

>	js.exe!nanojit::LirBuffer::validate()  Line 149 + 0x6 bytes	C++
 	js.exe!nanojit::LirBuffer::next()  Line 194	C++
 	js.exe!nanojit::LirBufWriter::ensureRoom(unsigned int count=2)  Line 221 + 0xb bytes	C++
 	js.exe!nanojit::LirBufWriter::insLink(nanojit::LOpcode op=LIR_tramp, nanojit::LIns * target=0x00df601c)  Line 428	C++
 	js.exe!nanojit::LirBufWriter::ensureReferenceable(nanojit::LIns * i=0x00df601c, int addedDistance=1)  Line 267 + 0xe bytes	C++
 	js.exe!nanojit::LirBufWriter::ins2(nanojit::LOpcode op=LIR_lt, nanojit::LIns * o1=0x01088f6c, nanojit::LIns * o2=0x00df601c)  Line 345 + 0x1e bytes	C++
 	js.exe!RegExpNativeCompiler::compileFlatSingleChar(RENode * node=0x00d64b18, nanojit::LIns * pos=0x01088f6c, avmplus::List<nanojit::LIns *,0> & fails={...})  Line 2018 + 0x22 bytes	C++
 	js.exe!RegExpNativeCompiler::compileNode(RENode * node=0x00d64b18, nanojit::LIns * pos=0x01088f6c, avmplus::List<nanojit::LIns *,0> & fails={...})  Line 2098 + 0x14 bytes	C++
 	js.exe!RegExpNativeCompiler::compileFlatSingleChar(RENode * node=0x00d64af8, nanojit::LIns * pos=0x01088f6c, avmplus::List<nanojit::LIns *,0> & fails={...})  Line 2034	C++
 	js.exe!RegExpNativeCompiler::compileNode(RENode * node=0x00d64af8, nanojit::LIns * pos=0x01088f34, avmplus::List<nanojit::LIns *,0> & fails={...})  Line 2098 + 0x14 bytes	C++
 	js.exe!RegExpNativeCompiler::compileFlatSingleChar(RENode * node=0x00d64ad8, nanojit::LIns * pos=0x01088f34, avmplus::List<nanojit::LIns *,0> & fails={...})  Line 2034	C++
 	js.exe!RegExpNativeCompiler::compileNode(RENode * node=0x00d64ad8, nanojit::LIns * pos=0x01088efc, avmplus::List<nanojit::LIns *,0> & fails={...})  Line 2098 + 0x14 bytes	C++
 	js.exe!RegExpNativeCompiler::compileFlatSingleChar(RENode * node=0x00d64ab8, nanojit::LIns * pos=0x01088efc, avmplus::List<nanojit::LIns *,0> & fails={...})  Line 2034	C++
... ad infinitum.

I'll leave this in the debugger for a while, If you have questions.

Comment 1

[Bob Clary [:bc] (inactive)]

this is a stack overflow.

Comment 2

[Andreas Gal :gal]

This looks familiar.

Comment 3

[Andreas Gal :gal]

David, is this the bug Rick just posted a patch for? (https://bugzilla.mozilla.org/show_bug.cgi?id=461073)

Comment 4

[Andreas Gal :gal]

Oh ... the _native_ stack overflowed. Ok. Then its the regexp compiler of course. If you could leave this in the debugger for a bit that would be great. CC'ing dmandelin.

Comment 5

[David Mandelin [:dmandelin]]

OK. I think I understand the source of the stack overflow, although I can't duplicate this on OSX. I walk the regular expression recursively, so if there are 100k flat char nodes, I'll recur to that depth. And I think that Windows does default to much smaller stacks than Unix, so that's probably why the problem showed up there first.

This is pretty painful. I think I can fix this particular problem by converting the compilation of the 'next' field of the regexp node from mutual recursion to tail recursion, but the code becomes harder to read. Also, the problem could still occur with deeply nested alternations, because I use recursion for that too. Hopefully this problem has been solved before for Mozilla and I can just use the same technique.

Comment 6

[David Mandelin [:dmandelin]]

Attachment: Patch

This makes flat strings not use recursion, so hopefully it will fix this particular error. I can try to test on Windows later or else we can just push it and see how bc's testing goes.

Comment 7

[David Mandelin [:dmandelin]]

Landed as http://hg.mozilla.org/mozilla-central/rev/cb84ae597941.

Comment 8

[Mike Beltzner [:beltzner, not reading bugmail]]

So, fixed?

Comment 9

[Bob Clary [:bc] (inactive)]

yes.

Comment 10

[Bob Clary [:bc] (inactive)]

v 1.9.1, 1.9.2