Last Comment Bug 652438 - TM: Crash [@ JSString::isLinear] on out-of-memory
: TM: Crash [@ JSString::isLinear] on out-of-memory
Status: RESOLVED FIXED
[sg:dos null-deref] fixed-in-tracemonkey
: crash, testcase
Product: Core
Classification: Components
Component: JavaScript Engine (show other bugs)
: Trunk
: x86_64 Linux
: -- critical (vote)
: mozilla6
Assigned To: Luke Wagner [:luke]
:
Mentors:
Depends on:
Blocks: langfuzz
  Show dependency treegraph
 
Reported: 2011-04-24 09:26 PDT by Christian Holler (:decoder)
Modified: 2013-12-27 14:19 PST (History)
9 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
-


Attachments
fix (924 bytes, patch)
2011-04-24 20:58 PDT, Luke Wagner [:luke]
jwalden+bmo: review+
asa: approval‑mozilla‑beta-
Details | Diff | Review

Description Christian Holler (:decoder) 2011-04-24 09:26:37 PDT
The following testcase (IMPORTANT: run with "-A 500 -m -j" to limit the memory usage) crashes on TM revision a03a4fea1679 (tested on 32 bit):

function f()
{
}
function dosubst()
{
  var s = f;
  for (var i = 0; i < 18; i++)
  {
    s += s;
  }
  var index = s.indexOf(f);
  while(true) {
    dosubst();
  }
}
dosubst();


Backtrace (looks like a safe null-pointer deref):

out of memory
==2747== Invalid read of size 4
==2747==    at 0x807BBEE: JSString::isLinear() const (jsstr.h:289)
==2747==    by 0x807BEBC: JSLinearString::chars() const (jsstr.h:421)
==2747==    by 0x807BFAB: JSString::getChars(JSContext*) (jsstr.h:697)
==2747==    by 0x81B228E: str_indexOf(JSContext*, unsigned int, js::Value*) (jsstr.cpp:1448)
==2747==    by 0x57B1788: ???
==2747==    by 0x82857D0: js::mjit::EnterMethodJIT(JSContext*, JSStackFrame*, void*, js::Value*) (MethodJIT.cpp:684)
==2747==    by 0x82858F5: CheckStackAndEnterMethodJIT(JSContext*, JSStackFrame*, void*) (MethodJIT.cpp:713)
==2747==    by 0x82859CA: js::mjit::JaegerShot(JSContext*) (MethodJIT.cpp:730)
==2747==    by 0x832E3A9: js::Interpret(JSContext*, JSStackFrame*, unsigned int, JSInterpMode) (jsinterp.cpp:4699)
==2747==    by 0x8234FFC: js::RecordTracePoint(JSContext*, js::TraceMonitor*, unsigned int&, bool*, bool) (jstracer.cpp:16741)
==2747==    by 0x82356CD: js::MonitorTracePoint(JSContext*, unsigned int&, bool*, void**, unsigned int*, unsigned int*, unsigned int) (jstracer.cpp:16898)
==2747==    by 0x82F2984: RunTracer(js::VMFrame&, js::mjit::ic::TraceICInfo&) (InvokeHelpers.cpp:981)
==2747==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
Comment 1 Luke Wagner [:luke] 2011-04-24 20:58:29 PDT
Created attachment 528058 [details] [diff] [review]
fix

Well, it looks like in the heat of bug 613457 I seem to have "refactored" out the OOM check I added in bug 609440.  Sheesh, you'd think if anyone would know about fallible string chars...

Thanks for finding this!  The pesky string limits make it hard to make a fast-running shell test case that doesn't require special flags.
Comment 2 Luke Wagner [:luke] 2011-04-25 11:54:59 PDT
http://hg.mozilla.org/tracemonkey/rev/a3e69f698ce3
Comment 3 Chris Leary [:cdleary] (not checking bugmail) 2011-04-26 15:44:07 PDT
http://hg.mozilla.org/mozilla-central/rev/a3e69f698ce3
Comment 4 Daniel Veditz [:dveditz] 2011-05-26 19:45:54 PDT
Comment on attachment 528058 [details] [diff] [review]
fix

We should consider landing this safe fix in Firefox 5: see bug 659920 comment 7
Comment 5 Luke Wagner [:luke] 2011-05-27 09:13:18 PDT
It's definitely low risk.
Comment 6 Asa Dotzler [:asa] 2011-06-01 12:28:16 PDT
Comment on attachment 528058 [details] [diff] [review]
fix

too late in the Beta cycle for non critical bugs.

Note You need to log in before you can comment on or make changes to this bug.