Closed Bug 620327 Opened 14 years ago Closed 14 years ago

Intermittent crash in jsreftest.html?test=ecma/LexicalConventions/7.1-2.js or js1_5/decompilation/regress-457824.js or js1_5/extensions/regress-390597.js [@ js_regexp_toString]

Categories

(Core :: JavaScript Engine, defect)

Product:
Core
Component:
JavaScript Engine
Platform:
x86_64
macOS
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
blocking2.0 --- betaN+

People

(Reporter: philor, Assigned: cdleary)

References

Details

(Keywords: intermittent-failure, Whiteboard: [softblocker][fixed-in-tracemonkey])

Attachments

(1 file, 2 obsolete files)

Use a StringBuffer builder, add OOM check.
4.15 KB, patch
Waldo
: review+
Details | Diff | Splinter Review 
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1292818001.1292818304.26489.gz
Rev3 MacOSX Snow Leopard 10.6.2 mozilla-central opt test jsreftest on 2010/12/19 20:06:41
s: talos-r3-snow-041

REFTEST TEST-START | file:///Users/cltbld/talos-slave/test/build/jsreftest/tests/jsreftest.html?test=ecma/LexicalConventions/7.1-2.js
TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/jsreftest/tests/jsreftest.html?test=ecma/LexicalConventions/7.1-2.js | Exited with code 1 during test run
INFO | automation.py | Application ran for: 0:01:53.039568
INFO | automation.py | Reading PID log: /var/folders/H5/H5TD8hgwEqKq9hgKlayjWU+++TM/-Tmp-/tmpFjYCcnpidlog
PROCESS-CRASH | file:///Users/cltbld/talos-slave/test/build/jsreftest/tests/jsreftest.html?test=ecma/LexicalConventions/7.1-2.js | application crashed (minidump found)
Operating system: Mac OS X
                  10.6.2 10C540
CPU: amd64
     family 6 model 23 stepping 10
     2 CPUs

Crash reason:  EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash address: 0x696ffffe

Thread 0 (crashed)
 0  0x7fffffe00f2e
    rbx = 0x2cebbbdc   r12 = 0x0a9cf600   r13 = 0x00000000   r14 = 0x2e7b9290
    r15 = 0x1675ddee   rip = 0xffe00f2e   rsp = 0x5fbfc4f0   rbp = 0x5fbfc4f0
    Found by: given as instruction pointer in context
 1  XUL!js_regexp_toString [jsregexp.cpp:302d1d3e2817 : 596 + 0x15]
    rip = 0x0102c442   rsp = 0x5fbfc500
    Found by: stack scanning
 2  XUL!Decompile [jsopcode.cpp:302d1d3e2817 : 4217 + 0xc]
    rbx = 0x0abaeb75   r12 = 0x000000a2   r13 = 0x00000000   r14 = 0x000000a2
    r15 = 0x000000a2   rip = 0x00ff1692   rsp = 0x5fbfc560   rbp = 0x00000000
    Found by: call frame info
 3  XUL!DecompileCode [jsopcode.cpp:302d1d3e2817 : 4861 + 0x1a]
    rbx = 0x0abaea8c   r12 = 0x281217f0   r13 = 0x25d9cf10   r14 = 0x00000001
    r15 = 0x0abaeb75   rip = 0x00ffae55   rsp = 0x5fbfc7c0   rbp = 0x0abae800
    Found by: call frame info
 4  XUL!DecompileExpression [jsopcode.cpp:302d1d3e2817 : 5302 + 0x16]
    rbx = 0x00000001   r12 = 0x0abaea8c   r13 = 0x25d9cf10   r14 = 0x0abaea8c
    r15 = 0x0abaeb75   rip = 0x00ffb045   rsp = 0x5fbfc880   rbp = 0x0abae800
    Found by: call frame info
 5  XUL!js_DecompileValueGenerator [jsopcode.cpp:302d1d3e2817 : 5165 + 0xd]
    rbx = 0x0bb760b0   r12 = 0x00000000   r13 = 0x00000000   r14 = 0x0abae800
    r15 = 0x0abaeb80   rip = 0x00ffb439   rsp = 0x5fbfc910   rbp = 0x25d9cf10
    Found by: call frame info
 6  XUL!js_ReportIsNullOrUndefined [jsopcode.h:302d1d3e2817 : 493 + 0x7]
    rbx = 0x00000000   r12 = 0x25d9cf10   r13 = 0x0bb76150   r14 = 0x25d9cf10
    r15 = 0x67516300   rip = 0x00f4ea3b   rsp = 0x5fbfc990   rbp = 0x00000000
    Found by: call frame info
 7  XUL!js_ValueToNonNullObject [jsobj.cpp:302d1d3e2817 : 6117 + 0x11]
    rbx = 0x00000000   r12 = 0x0bb76150   r13 = 0x25d9cf10   r14 = 0x25d9cf10
    r15 = 0x67516300   rip = 0x00fd55a3   rsp = 0x5fbfc9d0   rbp = 0x00000000
    Found by: call frame info
 8  XUL!js::mjit::stubs::GetElem [StubCalls-inl.h:302d1d3e2817 : 62 + 0x7]
    rbx = 0x0afa2030   r12 = 0x5fbfcaa0   r13 = 0x0bb76158   r14 = 0x25d9cf10
    r15 = 0x67516300   rip = 0x010f4bd8   rsp = 0x5fbfca00   rbp = 0x5fbfcaa0
    Found by: call frame info
 9  XUL!js::mjit::ic::GetElement [PolyIC.cpp:302d1d3e2817 : 2291 + 0x4]
    rbx = 0x0afa2030   r12 = 0x25d9cf10   r13 = 0x00000000   r14 = 0xffffffff
    r15 = 0x67516300   rip = 0x011458af   rsp = 0x5fbfca50   rbp = 0x5fbfcaa0
    Found by: call frame info
10  0x10b1886b2
    rbx = 0x0bb760b0   r12 = 0x67516300   r13 = 0x00000000   r14 = 0xffffffff
    r15 = 0x67516300   rip = 0x0b1886b3   rsp = 0x5fbfcaa0   rbp = 0x5fbfcb20
    Found by: call frame info
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1293298012.1293298402.4431.gz
Rev3 MacOSX Snow Leopard 10.6.2 mozilla-central opt test jsreftest on 2010/12/25 09:26:52
s: talos-r3-snow-051

REFTEST TEST-START | file:///Users/cltbld/talos-slave/test/build/jsreftest/tests/jsreftest.html?test=js1_5/decompilation/regress-457824.js
TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/jsreftest/tests/jsreftest.html?test=js1_5/decompilation/regress-457824.js | Exited with code 1 during test run
INFO | automation.py | Application ran for: 0:03:28.510758
INFO | automation.py | Reading PID log: /var/folders/H5/H5TD8hgwEqKq9hgKlayjWU+++TM/-Tmp-/tmp2NilPJpidlog
PROCESS-CRASH | file:///Users/cltbld/talos-slave/test/build/jsreftest/tests/jsreftest.html?test=js1_5/decompilation/regress-457824.js | application crashed (minidump found)
Operating system: Mac OS X
                  10.6.2 10C540
CPU: amd64
     family 6 model 23 stepping 10
     2 CPUs

Crash reason:  EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash address: 0x2d3ff10e

Thread 0 (crashed)
 0  0x7fffffe01249
    rbx = 0x25967be4   r12 = 0x0384c800   r13 = 0x00000000   r14 = 0x29e5b8a0
    r15 = 0x12cb3df2   rip = 0xffe01249   rsp = 0x5fbfc4d8   rbp = 0x5fbfc4d8
    Found by: given as instruction pointer in context
 1  XUL!js_regexp_toString [jsregexp.cpp:c84a2abbc663 : 596 + 0x15]
    rip = 0x01025e02   rsp = 0x5fbfc500
    Found by: stack scanning
 2  XUL!Decompile [jsopcode.cpp:c84a2abbc663 : 4217 + 0xc]
    rbx = 0x3f8c6175   r12 = 0x000000a2   r13 = 0x00000000   r14 = 0x000000a2
    r15 = 0x000000a2   rip = 0x00feb042   rsp = 0x5fbfc560   rbp = 0x00000000
    Found by: call frame info
 3  XUL!DecompileCode [jsopcode.cpp:c84a2abbc663 : 4861 + 0x1a]
    rbx = 0x3f8c608c   r12 = 0x05b6ee90   r13 = 0x1ee73780   r14 = 0x00000001
    r15 = 0x3f8c6175   rip = 0x00ff4805   rsp = 0x5fbfc7c0   rbp = 0x3f8c5e00
    Found by: call frame info
 4  XUL!DecompileExpression [jsopcode.cpp:c84a2abbc663 : 5302 + 0x16]
    rbx = 0x00000001   r12 = 0x3f8c608c   r13 = 0x1ee73780   r14 = 0x3f8c608c
    r15 = 0x3f8c6175   rip = 0x00ff49f5   rsp = 0x5fbfc880   rbp = 0x3f8c5e00
    Found by: call frame info
 5  XUL!js_DecompileValueGenerator [jsopcode.cpp:c84a2abbc663 : 5165 + 0xd]
    rbx = 0x0502e0b0   r12 = 0x00000000   r13 = 0x00000000   r14 = 0x3f8c5e00
    r15 = 0x3f8c6180   rip = 0x00ff4de9   rsp = 0x5fbfc910   rbp = 0x1ee73780
    Found by: call frame info
 6  XUL!js_ReportIsNullOrUndefined [jsopcode.h:c84a2abbc663 : 493 + 0x7]
    rbx = 0x00000000   r12 = 0x1ee73780   r13 = 0x0502e150   r14 = 0x1ee73780
    r15 = 0x2cbf4400   rip = 0x00f4841b   rsp = 0x5fbfc990   rbp = 0x00000000
    Found by: call frame info
 7  XUL!js_ValueToNonNullObject [jsobj.cpp:c84a2abbc663 : 6117 + 0x11]
    rbx = 0x00000000   r12 = 0x0502e150   r13 = 0x1ee73780   r14 = 0x1ee73780
    r15 = 0x2cbf4400   rip = 0x00fcef63   rsp = 0x5fbfc9d0   rbp = 0x00000000
    Found by: call frame info
 8  XUL!js::mjit::stubs::GetElem [StubCalls-inl.h:c84a2abbc663 : 62 + 0x7]
    rbx = 0x3f8d7830   r12 = 0x5fbfcaa0   r13 = 0x0502e158   r14 = 0x1ee73780
    r15 = 0x2cbf4400   rip = 0x010ee748   rsp = 0x5fbfca00   rbp = 0x5fbfcaa0
    Found by: call frame info
 9  XUL!js::mjit::ic::GetElement [PolyIC.cpp:c84a2abbc663 : 2291 + 0x4]
    rbx = 0x3f8d7830   r12 = 0x1ee73780   r13 = 0x00000000   r14 = 0xffffffff
    r15 = 0x2cbf4400   rip = 0x0113f42f   rsp = 0x5fbfca50   rbp = 0x5fbfcaa0
    Found by: call frame info
10  0x1035986b2
    rbx = 0x0502e0b0   r12 = 0x2cbf4400   r13 = 0x00000000   r14 = 0xffffffff
    r15 = 0x2cbf4400   rip = 0x035986b3   rsp = 0x5fbfcaa0   rbp = 0x5fbfcb20
    Found by: call frame info
Severity: normal → critical
blocking2.0: --- → ?
Summary: Intermittent crash in jsreftest.html?test=ecma/LexicalConventions/7.1-2.js [@ js_regexp_toString] → Intermittent crash in jsreftest.html?test=ecma/LexicalConventions/7.1-2.js or js1_5/decompilation/regress-457824.js [@ js_regexp_toString]
This is making me nervous in part because of the unreasonable time when it started. The first instance was on http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=150af817b65d&tochange=302d1d3e2817, roc's reftest-harness rewrite, the only vaguely interesting thing before that in http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=d1da1005b6d6&tochange=150af817b65d is peterv's bug 605672. The last TM merge was on December 15th, four days before it started failing multiple times a day, and we're not seeing it on TM anyway.
blocking2.0: ? → betaN+
Phil, I see a spate of these reports from Dec 19 through Dec 25, but not after that. Does that mean this has stopped happening on m-c?
Whiteboard: [orange] → [orange][softblocker]
Sure, make me expose my superstitions in public.

Nobody has ever come up with a credible theory for why, but empirically, randomorange happens more often per-push when there are more pushes. So, no, I didn't hide any instances of it over the hols, but I'd put way more faith in it not happening over the next two twenty-push days than in it not happening over the last five eight-push days.
OK, if I parsed that right, we should wait at least another week or so before concluding it went away.
A week wouldn't hurt, but I think what I meant by all that was that the large number of pushes yesterday and today would be enough to persuade me, and would persuade me more than the week before did.
(In reply to comment #21)
> A week wouldn't hurt, but I think what I meant by all that was that the large
> number of pushes yesterday and today would be enough to persuade me, and would
> persuade me more than the week before did.

Excellent. I'm all for clearing out things from my list.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → WORKSFORME
I'm not a very successful gambler.

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1294801322.1294801749.11522.gz&fulltext=1#err0
Rev3 MacOSX Snow Leopard 10.6.2 mozilla-central opt test jsreftest on 2011/01/11 19:02:02
s: talos-r3-snow-040

TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/jsreftest/tests/jsreftest.html?test=js1_5/extensions/regress-390597.js | Exited with code 1 during test run

Crash reason:  EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash address: 0x403000ce

Thread 0 (crashed)
 0  0x7fffffe01259
    rbx = 0x2804b99c   r12 = 0x0a064400   r13 = 0x00000000   r14 = 0x26468f70
    r15 = 0x468c9220   rip = 0xffe01259   rsp = 0x5fbfc498   rbp = 0x5fbfc498
    Found by: given as instruction pointer in context
 1  XUL!js_regexp_toString [jsregexp.cpp:b6f7632f63b7 : 603 + 0x19]
    rip = 0x010463d5   rsp = 0x5fbfc4c0
    Found by: stack scanning
 2  XUL!Decompile [jsopcode.cpp:b6f7632f63b7 : 4259 + 0x35]
    rbx = 0x0a05d000   r12 = 0x000000a2   r13 = 0x00000000   r14 = 0x000000a2
    r15 = 0x000000a2   rip = 0x01006009   rsp = 0x5fbfc520   rbp = 0x0a05d000
    Found by: call frame info
 3  XUL!DecompileCode [jsopcode.cpp:b6f7632f63b7 : 4895 + 0x1c]
    rbx = 0x00000000   r12 = 0x0a05d000   r13 = 0x2e61a5f0   r14 = 0x26468f70
    r15 = 0x00000001   rip = 0x01011208   rsp = 0x5fbfc790   rbp = 0x00000008
    Found by: call frame info
 4  XUL!DecompileExpression [jsopcode.cpp:b6f7632f63b7 : 5326 + 0x14]
    rbx = 0x00000001   r12 = 0x00000001   r13 = 0x26468f70   r14 = 0x0a05d385
    r15 = 0x0a05d393   rip = 0x0101158f   rsp = 0x5fbfc860   rbp = 0x0a05d000
    Found by: call frame info
 5  XUL!js_DecompileValueGenerator [jsopcode.cpp:b6f7632f63b7 : 5196 + 0xf]
    rbx = 0x1d3c40b0   r12 = 0x00000000   r13 = 0x00000000   r14 = 0x0a05d394
    r15 = 0x0a05d000   rip = 0x0101194b   rsp = 0x5fbfc8e0   rbp = 0x26468f70
    Found by: call frame info
 6  XUL!js_ReportIsNullOrUndefined [jsopcode.h:b6f7632f63b7 : 493 + 0x7]
    rbx = 0x00000000   r12 = 0x26468f70   r13 = 0x1d3c4150   r14 = 0x26468f70
    r15 = 0x40241080   rip = 0x00f5ef6b   rsp = 0x5fbfc960   rbp = 0x00000000
    Found by: call frame info
 7  XUL!js_ValueToNonNullObject [jsobj.cpp:b6f7632f63b7 : 6226 + 0x11]
    rbx = 0x00000000   r12 = 0x1d3c4150   r13 = 0x26468f70   r14 = 0x26468f70
    r15 = 0x40241080   rip = 0x00fe8cc3   rsp = 0x5fbfc9a0   rbp = 0x00000000
    Found by: call frame info
 8  XUL!js::mjit::stubs::GetElem [StubCalls-inl.h:b6f7632f63b7 : 62 + 0x7]
    rbx = 0x0a353380   r12 = 0x5fbfca80   r13 = 0x1d3c4158   r14 = 0x26468f70
    r15 = 0x40241080   rip = 0x0110fcc8   rsp = 0x5fbfc9d0   rbp = 0x5fbfca80
    Found by: call frame info
 9  XUL!js::mjit::ic::GetElement [PolyIC.cpp:b6f7632f63b7 : 2447 + 0x4]
    rbx = 0x0a353380   r12 = 0x26468f70   r13 = 0x00000000   r14 = 0xffffffff
    r15 = 0x40241080   rip = 0x01169349   rsp = 0x5fbfca20   rbp = 0x5fbfca80
    Found by: call frame info
10  0x10a98a6f0
    rbx = 0x1d3c40b0   r12 = 0x40241080   r13 = 0x00000000   r14 = 0xffffffff
    r15 = 0x40241080   rip = 0x0a98a6f1   rsp = 0x5fbfca80   rbp = 0x5fbfcb00
    Found by: call frame info
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Summary: Intermittent crash in jsreftest.html?test=ecma/LexicalConventions/7.1-2.js or js1_5/decompilation/regress-457824.js [@ js_regexp_toString] → Intermittent crash in jsreftest.html?test=ecma/LexicalConventions/7.1-2.js or js1_5/decompilation/regress-457824.js or js1_5/extensions/regress-390597.js [@ js_regexp_toString]
Attached patch Use a StringBuffer builder. (obsolete) — — Splinter Review 
It's time to party like it's 2011 and we're using C++. (Might as well see what happens -- looks like the original traces were indicating flagCount, maybe?)

Also interesting -- I don't know when we would ever have an instance of js_RegExpClass that had no internal |RegExp|, and asserting on it turned up no failing tests, so I removed that bit as well.

Depends on patch for bug 617935.
Assignee: general → cdleary
Status: REOPENED → ASSIGNED
Attachment #503083 - Flags: review?(dmandelin)
Doesn't that happen in case of OOM while creating a regular expression, like, say, if RegExp::create in js_XDRRegExpObject returned NULL?  Although in that case, I guess proper error-checking means the regular expression never escapes for a toString call to fail.  I wouldn't be so sure we do that correctly all the time -- just looking at js_CloneRegExpObject it looks like clones could have a NULL private if RegExp::create failed at just the right time.

I'm all for creating RegExps, then creating the objects that back them with the RegExp provided, to eliminate the js_RegExpClass-without-RegExp concern.  I'm just not sure this one-off is advisable without a more careful audit of how such objects are created, and probably some internal API changes too.
Waldo found a recent (fx4 era, yarr timeframe) regression in js_CloneRegExpObject and noted it in comment 26. Separate bug? Null-checking and return, easy fix.

/be
(In reply to comment #26)

I'm pretty sure the biggest weirdness comes from swapping js::RegExp guts a la bug 623435 and the rest of the paths look fine to me -- no direct instantiations of js_RegExpClass outside of the regexp files I've been cleaning up and all possible regexp_compile_sub paths guaranteeing either a non-null js::RegExp or failure. The only way the XDR can screw up is if you use it completely wrong by ignoring the return code.

In any case, I'm fine with erring on the side of caution. We can probably factor code in some of these fastcalls to use js::RegExp::createObject* anyway, which are easier to reason about.
(In reply to comment #28)
> Waldo found a recent (fx4 era, yarr timeframe) regression in
> js_CloneRegExpObject and noted it in comment 26. Separate bug? Null-checking
> and return, easy fix.

Oof, nice catch. Erring on the side of caution sounds even better now.
Adds missing OOM check and puts the null js::RegExp private check back in.
Attachment #503083 - Attachment is obsolete: true
Attachment #503098 - Flags: review?(jwalden+bmo)
Attachment #503083 - Flags: review?(dmandelin)
Attachment #503098 - Attachment is obsolete: true
Attachment #503098 - Flags: review?(jwalden+bmo)
Attachment #503284 - Flags: review?(jwalden+bmo)
Comment on attachment 503284 [details] [diff] [review]
Use a StringBuffer builder, add OOM check.

GOOD GRIEF THIS IS SO MUCH BETTER IT'S NOT EVEN FUNNY.

How much of a bribe would it take to get the code in jsexn.cpp converted this way?
Attachment #503284 - Flags: review?(jwalden+bmo) → review+
http://hg.mozilla.org/tracemonkey/rev/99c9ed53df99

Not marking as fixed-in-tracemonkey to prevent this bug from being marked as fixed when it merges to m-c. /me crosses fingers.
http://hg.mozilla.org/mozilla-central/rev/99c9ed53df99

I'm going to speculatively mark as fixed in an attempt to test the hypothesis that optimism yields results.
Status: ASSIGNED → RESOLVED
Closed: 14 years ago → 14 years ago
Resolution: --- → FIXED
Whiteboard: [orange][softblocker] → [orange][softblocker][fixed-in-tracemonkey]
Whiteboard: [orange][softblocker][fixed-in-tracemonkey] → [softblocker][fixed-in-tracemonkey]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: