735036 - JS OOM Testing: Assertion failure: table, at ./dist/include/js/HashTable.h:450

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Christian Holler (:decoder)]

The following command aborts/crashes on mozilla-central revision c6f26a8dcd08:

js -m -n -a -A 7493 -f js/src/jit-test/tests/basic/bug704795.js


Backtrace of failed allocation (as outputted when compiling with --enable-oom-backtrace and filtered through addr2line):

#0 js/src/debug64-trunk/js(+0x415121) (PrintBacktrace at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-trunk/shell/../dist/include/js/Utility.h:130)
#1 js/src/debug64-trunk/js(+0x415203) (js_malloc at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-trunk/shell/../dist/include/js/Utility.h:172)
#2 js/src/debug64-trunk/js(+0x415364) (js::SystemAllocPolicy::malloc_(unsigned long) at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-trunk/shell/../../jsalloc.h:66)
#3 js/src/debug64-trunk/js(+0x4c70ed) (js::detail::HashTable<js::HashMapEntry<void*, js::gc::VerifyNode*>, js::HashMap<void*, js::gc::VerifyNode*, js::DefaultHasher<void*>, js::SystemAllocPolicy>::MapHashPolicy, js::SystemAllocPolicy>::createTable(js::SystemAllocPolicy&, unsigned int) at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-trunk/./dist/include/js/HashTable.h:345)
#4 js/src/debug64-trunk/js(+0x4c44e9) (js::detail::HashTable<js::HashMapEntry<void*, js::gc::VerifyNode*>, js::HashMap<void*, js::gc::VerifyNode*, js::DefaultHasher<void*>, js::SystemAllocPolicy>::MapHashPolicy, js::SystemAllocPolicy>::init(unsigned int) at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-trunk/./dist/include/js/HashTable.h:402)
#5 js/src/debug64-trunk/js(+0x4bdf68) (js::HashMap<void*, js::gc::VerifyNode*, js::DefaultHasher<void*>, js::SystemAllocPolicy>::init(unsigned int) at /home/decoder/LangFuzz/mozilla-central/js/src/debug64-trunk/./dist/include/js/HashTable.h:959)
#6 js/src/debug64-trunk/js(+0x4b9fe7) (js::gc::StartVerifyBarriers(JSContext*) at /home/decoder/LangFuzz/mozilla-central/js/src/jsgc.cpp:4226)
#7 js/src/debug64-trunk/js(+0x4bab1c) (js::gc::MaybeVerifyBarriers(JSContext*, bool) at /home/decoder/LangFuzz/mozilla-central/js/src/jsgc.cpp:4436)
#8 js/src/debug64-trunk/js(+0x515adf) (js::Interpret(JSContext*, js::StackFrame*, js::InterpMode) at /home/decoder/LangFuzz/mozilla-central/js/src/jsinterp.cpp:4373)
#9 js/src/debug64-trunk/js(+0x69e1e3) (js::mjit::EnterMethodJIT(JSContext*, js::StackFrame*, void*, JS::Value*, bool) at /home/decoder/LangFuzz/mozilla-central/js/src/methodjit/MethodJIT.cpp:1079)


Stepping through this reveals crash with possible security impact:

Program received signal SIGSEGV, Segmentation fault.
0x00000000004c93d6 in js::detail::HashTableEntry<js::HashMapEntry<void*, js::gc::VerifyNode*> >::isFree (this=0x17ffffffd0) at ./dist/include/js/HashTable.h:88
88          bool isFree() const           { return keyHash == sFreeKey; }
(gdb) x /i $pc
=> 0x4c93d6 <js::detail::HashTableEntry<js::HashMapEntry<void*, js::gc::VerifyNode*> >::isFree() const+12>:     mov    (%rax),%eax
(gdb) info rax
Undefined info command: "rax".  Try "help info".
(gdb) info reg rax
rax            0x17ffffffd0     103079215056
(gdb) bt 8
#0  0x00000000004c93d6 in js::detail::HashTableEntry<js::HashMapEntry<void*, js::gc::VerifyNode*> >::isFree (this=0x17ffffffd0) at ./dist/include/js/HashTable.h:88
#1  0x00000000004c6c6a in js::detail::HashTable<js::HashMapEntry<void*, js::gc::VerifyNode*>, js::HashMap<void*, js::gc::VerifyNode*, js::DefaultHasher<void*>, js::SystemAllocPolicy>::MapHashPolicy, js::SystemAllocPolicy>::lookup (this=0xb87cd0, l=@0x7fffffffcf10, keyHash=4294967294, collisionBit=1) at ./dist/include/js/HashTable.h:458
#2  0x00000000004c435a in js::detail::HashTable<js::HashMapEntry<void*, js::gc::VerifyNode*>, js::HashMap<void*, js::gc::VerifyNode*, js::DefaultHasher<void*>, js::SystemAllocPolicy>::MapHashPolicy, js::SystemAllocPolicy>::lookupForAdd (this=0xb87cd0, l=@0x7fffffffcf10) at ./dist/include/js/HashTable.h:677
#3  0x00000000004bde7f in js::HashMap<void*, js::gc::VerifyNode*, js::DefaultHasher<void*>, js::SystemAllocPolicy>::lookupForAdd (this=0xb87cd0, l=@0x7fffffffcf10)
    at ./dist/include/js/HashTable.h:1018
#4  0x00000000004b9b80 in js::gc::MakeNode (trc=0xb87c70, thing=0x0, kind=JSTRACE_OBJECT) at /home/decoder/LangFuzz/mozilla-central/js/src/jsgc.cpp:4149
#5  0x00000000004b9ffd in js::gc::StartVerifyBarriers (cx=0xb7aaa0) at /home/decoder/LangFuzz/mozilla-central/js/src/jsgc.cpp:4226
#6  0x00000000004bab1c in js::gc::MaybeVerifyBarriers (cx=0xb7aaa0, always=true) at /home/decoder/LangFuzz/mozilla-central/js/src/jsgc.cpp:4436
#7  0x0000000000515adf in js::Interpret (cx=0xb7aaa0, entryFrame=0x7ffff67db030, interpMode=js::JSINTERP_NORMAL) at /home/decoder/LangFuzz/mozilla-central/js/src/jsinterp.cpp:4372
(More stack frames follow...)

Comment 1

[Christian Holler (:decoder)]

You shall not hit enter by accident -.-

Comment 2

[Daniel Veditz [:dveditz]]

uninitialized memory use when the allocation fails?

Comment 3

[Luke Wagner [:luke]]

Looks like StartVerifyBarriers is missing an oom check.  This is a debug-only verification function and thus not s-s, right Bill?

Comment 4

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Yeah, this is debug-only code.

Comment 5

[Tom S [:evilpie]]

Attachment: fix

Obvious fix that Luke already hinted at.
I also added MOZ_WARN_UNUSED_RESULT to init, not sure how useful that is considering how many functions potentially could have it. But because you usually don't need the return value of init functions, it might even make sense.

I have no idea how you test this stuff, because it's probably very flaky.

Comment 6

[Luke Wagner [:luke]]

Comment on attachment 617594 [details] [diff] [review]
fix

Oooh, MOZ_WARN_UNUSED_RESULT... need to start using that.

Thanks!

Comment 7

[Tom S [:evilpie]]

http://hg.mozilla.org/integration/mozilla-inbound/rev/59c95d0f775a

Comment 8

[(no longer active)]

https://hg.mozilla.org/mozilla-central/rev/59c95d0f775a