367888 - regexp empty alt (|) with repeat (?, ??, {}, etc) yields infinite loop and consumes all memory

Status: VERIFIED FIXED

(Core :: JavaScript Engine, defect)

Description

[Jesse Ruderman]

Split from bug 346237.

/(|)??x/g.exec("y");
makes the js engine quickly grow to 1.0 GB and then 1.99 GB.

If I leave it running long enough, I get:
js(1174) malloc: *** vm_allocate(size=4194308096) failed (error code=3)
js(1174) malloc: *** error: can't allocate region
js(1174) malloc: *** set a breakpoint in szone_error to debug
typein:1: out of memory

I didn't leave it running long enough to find out whether it eventually crashes or throws.

Using options("explosive_re_throws") from bug 330569 doesn't seem to help.

Comment 1

[Brian Crowder]

A quick hack I wrote fixed the initial case of this, but I found it very easy to craft a very slight variant which exhibited the same bad behavior.  More to come.

Comment 2

[Brian Crowder]

Attachment: handle REOP_ALT "empty" cases better

What is happening here is that we are succeeding in the ALT match, but the success is on an EMPTY op (which hasn't been added to the instruction stream), and so the success isn't yielding a non-null "result".  This confuses the repeat routine because it relies on result to determine whether to stop matching (in some cases).

Comment 3

[Brian Crowder]

jsregexp.c: 3.134

Comment 4

[Bob Clary [:bc] (inactive)]

/cvsroot/mozilla/js/tests/ecma_3/RegExp/regress-367888.js,v  <--  regress-367888.js
initial revision: 1.1

Comment 5

[Bob Clary [:bc] (inactive)]

verified fixed 1.9.0 20070320 win/mac*/linux