727331 - JS OOM Testing: Assertion failure: !rt->gcRunning, at js/src/jsgc.cpp:2990

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Christian Holler (:decoder)]

The following command asserts on mozilla-central revision d45c7d7b0079:

js -m -n -a -A 1137 -f js/src/tests/shell.js -f js/src/tests/js1_7/shell.js -f js/src/tests/js1_7/geniter/shell.js -f js/src/tests/js1_7/geniter/regress-466206.js

Comment 1

[Christian Holler (:decoder)]

I tried looking at this problem but I don't really understand it. The malloc that fails is in the | createTable | function of the HashTable instance holding RegExpCompartment/RegExpShared*. Here's the backtrace (emitted by my backtrace-on-oom-failure patch):

Forcing artificial memory allocation function failure:
#0 js/src/debug64-bt/js(+0x429f50) (js_malloc at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-bt/shell/../dist/include/js/Utility.h:162)
#1 js/src/debug64-bt/js(+0x42bc8d) (JSRuntime::malloc_(unsigned long, JSContext*) at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-bt/shell/../../jscntxt.h:606)
#2 js/src/debug64-bt/js(+0x58fd95) (js::RuntimeAllocPolicy::malloc_(unsigned long) at /home/decoder/LangFuzz/mozilla-central/js/src/jscntxt.h:1655)
#3 js/src/debug64-bt/js(+0x6a1fee) (js::detail::HashTable<js::HashMapEntry<js::RegExpCompartment::Key, js::RegExpShared*>, js::HashMap<js::RegExpCompartment::Key, js::RegExpShared*, js::RegExpCompartment::Key, js::RuntimeAllocPolicy>::MapHashPolicy, js::RuntimeAllocPolicy>::createTable(js::RuntimeAllocPolicy&, unsigned int) at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-bt/./dist/include/js/HashTable.h:345)
#4 js/src/debug64-bt/js(+0x6a21c4) (js::detail::HashTable<js::HashMapEntry<js::RegExpCompartment::Key, js::RegExpShared*>, js::HashMap<js::RegExpCompartment::Key, js::RegExpShared*, js::RegExpCompartment::Key, js::RuntimeAllocPolicy>::MapHashPolicy, js::RuntimeAllocPolicy>::changeTableSize(int) at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-bt/./dist/include/js/HashTable.h:560)
#5 js/src/debug64-bt/js(+0x6a1abc) (js::detail::HashTable<js::HashMapEntry<js::RegExpCompartment::Key, js::RegExpShared*>, js::HashMap<js::RegExpCompartment::Key, js::RegExpShared*, js::RegExpCompartment::Key, js::RuntimeAllocPolicy>::MapHashPolicy, js::RuntimeAllocPolicy>::checkUnderloaded() at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-bt/./dist/include/js/HashTable.h:604)
#6 js/src/debug64-bt/js(+0x6a0912) (~Enum at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-bt/./dist/include/js/HashTable.h:253)
#7 js/src/debug64-bt/js(+0x69de00) (js::RegExpCompartment::sweep(JSRuntime*) at /home/decoder/LangFuzz/mozilla-central/js/src/vm/RegExpObject.cpp:628)
Assertion failure: !rt->gcRunning, at /home/decoder/LangFuzz/mozilla-central/js/src/jsgc.cpp:3722


So as far as I can tell, the table is supposed to be shrinked which requires temporary space for the new (smaller) table, and that malloc fails. However, nothing about the old table is touched before that malloc succeeds and I don't really know what the GC assert here means either.

Comment 2

[Luke Wagner [:luke]]

changeTableSize handles OOM (by just not resizing), so the assert in onOutOfMemory seems bogus in at least this context.

Comment 3

[Adam]

Attachment: 727331 #1

Comment 4

[Jason Orendorff [:jorendorff]]

Comment on attachment 602995 [details] [diff] [review]
727331 #1

I think this is exactly what billm asked for but I'm redirecting review to him just in case. Sorry for the slowness on reviews recently...

Comment 5

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Comment on attachment 602995 [details] [diff] [review]
727331 #1

Yeah, thanks.

Comment 6

[Christian Holler (:decoder)]

Can this be landed since it has r+? It's a bug that triggers with a high frequency.

Comment 7

[Jason Orendorff [:jorendorff]]

I'll land it when the tree reopens (today, we expect).

Comment 8

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

https://hg.mozilla.org/mozilla-central/rev/963e2482837b