714545 - Intermittent crash [@ js::gc::Chunk::releaseArena] during shutdown

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Phil Ringnalda (:philor)]

I was hoping (without any actual reason to hope) that this was just another manifestation of bug 714344, but it's still happening after that fix landed.

https://tbpl.mozilla.org/php/getParsedLog.php?id=8263995&tree=Mozilla-Inbound
Rev3 WINNT 6.1 mozilla-inbound pgo test mochitest-other on 2012-01-01 07:05:21 PST for push da6c33eb4b16

PROCESS-CRASH | Shutdown | application crashed (minidump found)
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpzw4v55\minidumps\ebe82e25-01f2-4e00-84c4-1eb1d4376d04.dmp
Operating system: Windows NT
                  6.1.7600 
CPU: x86
     GenuineIntel family 6 model 23 stepping 10
     2 CPUs

Crash reason:  EXCEPTION_ACCESS_VIOLATION_WRITE
Crash address: 0x0

Thread 0 (crashed)
 0  mozjs.dll!js::gc::Chunk::releaseArena(js::gc::ArenaHeader *) [jsgc.cpp:da6c33eb4b16 : 782 + 0xc]
    eip = 0x6d35885f   esp = 0x0020eee8   ebp = 0x111ce000   ebx = 0x015d3000
    esi = 0x11100000   edi = 0x05d15000   eax = 0x05d15038   ecx = 0x00000000
    edx = 0x00000000   efl = 0x00210202
    Found by: given as instruction pointer in context
 1  mozjs.dll!js::gc::ArenaLists::~ArenaLists() [jsgc.h:da6c33eb4b16 : 1088 + 0xd]
    eip = 0x6d340467   esp = 0x0020ef04   ebp = 0x19304004   ebx = 0x015d3000
    Found by: call frame info
 2  mozjs.dll!js_FinishGC(JSRuntime *) [jsgc.cpp:da6c33eb4b16 : 1202 + 0xe]
    eip = 0x6d3589cf   esp = 0x0020ef14   ebp = 0x19304004
    Found by: call frame info
 3  mozjs.dll!JSRuntime::~JSRuntime() [jsapi.cpp:da6c33eb4b16 : 813 + 0x5]
    eip = 0x6d32d80d   esp = 0x0020ef38   ebp = 0x00000000   ebx = 0x00000001
    Found by: call frame info
 4  mozjs.dll!JS_Finish [jsapi.cpp:da6c33eb4b16 : 901 + 0xa]
    eip = 0x6d32fdb9   esp = 0x0020ef58   ebp = 0x00000000   ebx = 0x011c0190
    Found by: call frame info with scanning
 5  xul.dll!XPCJSRuntime::~XPCJSRuntime() [XPCJSRuntime.cpp:da6c33eb4b16 : 1227 + 0x6]
    eip = 0x6a83684c   esp = 0x0020ef60   ebp = 0x00000000
    Found by: call frame info
 6  xul.dll!nsXPConnect::~nsXPConnect() [nsXPConnect.cpp:da6c33eb4b16 : 157 + 0x11]
    eip = 0x6a8445ef   esp = 0x0020ef70   ebp = 0x6a83684c
    Found by: call frame info with scanning
 7  xul.dll!nsXPConnect::`vector deleting destructor'(unsigned int) + 0x7
    eip = 0x6a86a9e6   esp = 0x0020ef84   ebp = 0x00000000
    Found by: call frame info with scanning

https://tbpl.mozilla.org/php/getParsedLog.php?id=8249548&tree=Mozilla-Inbound
Rev3 WINNT 6.1 mozilla-inbound pgo test reftest on 2011-12-31 00:12:35 PST for push ca99dd8313ce

PROCESS-CRASH | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/bugs/98223-1-ref.html | application crashed (minidump found)
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmplee70a\minidumps\ca04c4f1-979a-4606-97b3-8b5e0901eefb.dmp
Operating system: Windows NT
                  6.1.7600 
CPU: x86
     GenuineIntel family 6 model 23 stepping 10
     2 CPUs

Crash reason:  EXCEPTION_ACCESS_VIOLATION_WRITE
Crash address: 0x0

Thread 9 (crashed)
 0  mozjs.dll!js::gc::Chunk::releaseArena(js::gc::ArenaHeader *) [jsgc.cpp:ca99dd8313ce : 782 + 0xc]
    eip = 0x7269881f   esp = 0x04b7fde8   ebp = 0x085d7000   ebx = 0x0a7d1000
    esi = 0x08500000   edi = 0x04915000   eax = 0x04915038   ecx = 0x00000000
    edx = 0x00000000   efl = 0x00010202
    Found by: given as instruction pointer in context
 1  mozjs.dll!js::gc::FinalizeTypedArenas<JSObject>(JSContext *,js::gc::ArenaLists::ArenaList *,js::gc::AllocKind,bool) [jsgc.cpp:ca99dd8313ce : 362 + 0xf]
    eip = 0x72699cea   esp = 0x04b7fe04   ebp = 0x04b7fe48   ebx = 0x00000001
    Found by: call frame info
 2  mozjs.dll!js::gc::FinalizeArenas [jsgc.cpp:ca99dd8313ce : 399 + 0x8]
    eip = 0x72699d61   esp = 0x04b7fe24   ebp = 0x04935330   ebx = 0x0098d100
    Found by: call frame info
 3  mozjs.dll!js::gc::ArenaLists::backgroundFinalize(JSContext *,js::gc::ArenaHeader *) [jsgc.cpp:ca99dd8313ce : 1578 + 0x25]
    eip = 0x72699f2e   esp = 0x04b7fe3c   ebp = 0x04935330
    Found by: call frame info
 4  mozjs.dll!js::GCHelperThread::doSweep() [jsgc.cpp:ca99dd8313ce : 2533 + 0x8]
    eip = 0x7269a0bd   esp = 0x04b7fe54   ebp = 0x04935330   ebx = 0x0098d100
    Found by: call frame info
 5  mozjs.dll!js::GCHelperThread::threadLoop() [jsgc.cpp:ca99dd8313ce : 2394 + 0x6]
    eip = 0x7269af1e   esp = 0x04b7fe78   ebp = 0x04915000   ebx = 0x04915000
    Found by: call frame info
 6  nspr4.dll!_PR_NativeRunThread [pruthr.c:ca99dd8313ce : 426 + 0x8]
    eip = 0x732e2b70   esp = 0x04b7fe94   ebp = 0x04b7feb0   ebx = 0x0090436c
    Found by: call frame info
 7  nspr4.dll!pr_root [w95thred.c:ca99dd8313ce : 122 + 0xc]
    eip = 0x732e3c3d   esp = 0x04b7feb8   ebp = 0x04b7fef0
    Found by: previous frame's frame pointer

https://tbpl.mozilla.org/php/getParsedLog.php?id=8254659&tree=Mozilla-Inbound
Rev3 WINNT 5.1 mozilla-inbound debug test reftest on 2011-12-31 06:57:31 PST for push 196f5b34b6e3

PROCESS-CRASH | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-y-interp-2.svg | application crashed (minidump found)
Crash dump filename: c:\docume~1\cltbld\locals~1\temp\tmpae2zyr\minidumps\8d27e17d-6a4a-4e45-b86c-42092855934d.dmp
Operating system: Windows NT
                  5.1.2600 Service Pack 2
CPU: x86
     GenuineIntel family 6 model 23 stepping 10
     2 CPUs

Crash reason:  EXCEPTION_ACCESS_VIOLATION_WRITE
Crash address: 0x0

Thread 0 (crashed)
 0  mozjs.dll!CrashInJS [jsutil.cpp:196f5b34b6e3 : 89 + 0x2]
    eip = 0x01059572   esp = 0x0012d574   ebp = 0x0012d57c   ebx = 0x04b00000
    esi = 0x10229380   edi = 0x04b78000   eax = 0x00000000   ecx = 0x22aa71b0
    edx = 0x10313d18   efl = 0x00210206
    Found by: given as instruction pointer in context
 1  mozjs.dll!js::gc::Chunk::releaseArena(js::gc::ArenaHeader *) [jsgc.cpp:196f5b34b6e3 : 779 + 0x1c]
    eip = 0x00f74995   esp = 0x0012d584   ebp = 0x0012d57c
    Found by: call frame info with scanning

Comment 1

[Igor Bukanov]

It should be an old bug that was exposed by the changes from the bug 702251. If, during the shutdown, we have a leak and some GC arenas are still marked as having live GC thing, the code that tries to release at least the GC part of the memory races with the background finalization. The bug 702251 could have exposed this race as it moved the chunk release, a slow operation, after all the finalization is done, so it is more likely to race with the js_FinsihGC call on the main thread.

Comment 2

[Igor Bukanov]

I close this as the race in case of the shutdown leak could be exploited even if extremely unlikely.

Comment 3

[Igor Bukanov]

Attachment: v1

The fix moves the helper thread shutdown code before we forcefully release any remaining compartments and their GC arenas in the ArenaLists destructor.

Comment 4

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Comment on attachment 585223 [details] [diff] [review]
v1

It would be nice to track those leaks down :-).

Comment 5

[Igor Bukanov]

https://hg.mozilla.org/integration/mozilla-inbound/rev/9cf396847500

Comment 6

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/9cf396847500

I guess we're calling this fixed, and dealing with the rest in bug 714562?

Comment 7

[Igor Bukanov]

I rename the bug to properly reflect the area it covers

Comment 8

[Daniel Veditz [:dveditz]]

I'm assuming comment 1 means that older version were unaffected by this bug in practical terms.