425594 - new branch top crash [@ js_GC] maybe also crash [@ js_MarkGCThing][@js_GetGCThingFlags]

Status: VERIFIED FIXED

(Core :: JavaScript Engine, defect)

Description

[Samuel Sidler (old account; do not CC)]

There's a new branch topcrash. We've seen crashes @js_GC but they're appearing a *lot* more frequently in Firefox 2.0.0.13. These two are the #3 and #4 topcrashes, respectively.

The trace smart analysis gives is: 

js_GC	[c:/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/js/src/jsgc.c  line 2947] 
JS_GC	[c:/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/js/src/jsapi.c  line 1879] 
nsAppStartup::Run	[c:/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/toolkit/components/startup/src/nsAppStartup.cpp  line 152] 
main	[c:/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/browser/app/nsBrowserApp.cpp  line 61] 
kernel32.dll + 0x16fd7 (0x7c816fd7)   


At a different offset, we also see (which might be the old js_GC crash):

js_GC	[c:/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/js/src/jsgc.c  line 2947]  


And another (Mac-specific):

js_GC()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/js/src//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/js/src/jsgc.c  line 2947] 
js_GC()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/js/src//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/js/src/jsgc.c  line 2904] 
nsJSContext::Notify()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/dom/src/base//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/dom/src/base/nsJSEnvironment.cpp  line 2231] 
nsTimerImpl::Fire()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/xpcom/threads//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/xpcom/threads/nsTimerImpl.cpp  line 398] 
handleTimerEvent()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/xpcom/threads//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/xpcom/threads/nsTimerImpl.cpp  line 462] 
PL_HandleEvent()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/xpcom/threads//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/xpcom/threads/plevent.c  line 689] 
PL_ProcessPendingEvents()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/xpcom/threads//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/xpcom/threads/plevent.c  line 623] 
CoreFoundation.368.33.0 + 0x242ec (0x907df2ec)  
CoreFoundation.368.33.0 + 0x2381c (0x907de81c)  
CoreFoundation.368.33.0 + 0x2329c (0x907de29c)  
HIToolbox.228.0.0 + 0x8b20 (0x932aeb20)  
HIToolbox.228.0.0 + 0x81b4 (0x932ae1b4)  
HIToolbox.228.0.0 + 0x4d348 (0x932f3348)  
HIToolbox.228.0.0 + 0x4d138 (0x932f3138)  
nsAppShell::Run()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/widget/src/mac//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/widget/src/mac/nsAppShell.cpp  line 94] 
nsAppStartup::Run()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/extensions/transformiix/source/xslt/util//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/db/sqlite3/src/pragma.c  line 152] 
XRE_main()	[/builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/build/unifox/ppc/toolkit/xre//builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/mozilla/toolkit/xre/nsAppRunner.cpp  line 2819] 
_start()	[/SourceCache/Csu/Csu-45//SourceCache/Csu/Csu-45/crt.c  line 267] 
start()   


Lots and lots of comments here: http://talkback-public.mozilla.org/reports/firefox/FF20013/smart-analysis.all

Fall out from bug 411025?

That bug is also about to get approval for the 1.8.0 branch. If this is indeed a regression from that bug, this bug should block that release.

Marking as security sensitive since that bug was before release.

Comment 1

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Related to bug 425576?

Comment 2

[Blake Kaplan (:mrbkap) (inactive)]

I think so.

Comment 3

[Daniel Veditz [:dveditz]]

and topcrash #6 js_GetGCThingFlags

these jumped from ~0.3% of crashes each to 2.7-3.2% of crashes (~9% combined)

Are all three likely to be bug 425576, or was there a similar bug in some of the other GC-related fixes? 15 bugs in:
https://bugzilla.mozilla.org/buglist.cgi?quicksearch=ALL%20!fixed1.8.1.13%2Cverified1.8.1.13%20comp%3AJavascript

Comment 4

[Daniel Veditz [:dveditz]]

This bug is the same thing as bug 425576, but in case it's not obvious in the public bug I wanted to flag that a GC crash like this is almost certainly exploitable if you can cause it at will. The concentrated crashes at Excite means someone _could_ figure out a way to cause it at will pretty reliably.

Igor: it would be great (required, really) to have testcases that demonstrate the bug so QA can test this fix and we can prevent future regressions. Ideally at least three, one for each stack we're seeing (js_MarkGCThing, js_GC, js_GetGCThingFlags)

Comment 5

[Josh Bressers]

Let's use CVE-2008-1380 for this one.

Comment 6

[Samuel Sidler (old account; do not CC)]

Igor, the testcase you attached in bug 425576 crashes in js_GC. Any luck getting a crash in js_MarkGCThing or js_GetGCThingFlags?

(And thank you *so much* for finding a testcase for one of these stacks.)

Comment 7

[Igor Bukanov]

(In reply to comment #6)
> Igor, the testcase you attached in bug 425576 crashes in js_GC. Any luck
> getting a crash in js_MarkGCThing or js_GetGCThingFlags?

Do you have the traces with line numbers that indicates js_MarkGCThing? In particular, I would like to know on which line in js_GC the code was when js_MarkGCThing was invoked.

The bug results in code in js_GC walking a linked list where one of the nodes points to already returned native stack frame. Thus the crash can happen either when accessing the next link for the node, or when working with the data stored in the node. In the first case the crash will be in js_GC while in the second case it will be either in js_GC or in js_MarkGCThing or in js_GetGCThingFlags depending on how the the struct from the returned frame was overwritten.

Comment 8

[Samuel Sidler (old account; do not CC)]

Igor, there are stacks available on the smart analysis page: http://talkback-public.mozilla.org/reports/firefox/FF20013/smart-analysis.all

But for reference...

js_MarkGCThing:

Incident ID: 43400672
Stack Signature	js_MarkGCThing 985368a9
Product ID	Firefox2
Build ID	2008031114
Trigger Time	2008-04-02 07:55:11.0
Platform	Win32
Operating System	Windows NT 5.1 build 2600
Module	js3250.dll + (0001f93d)
URL visited	
User Comments	
Since Last Crash	49202 sec
Total Uptime	168383 sec
Trigger Reason	Access violation
Source File, Line No.	c:/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/js/src/jsgc.c, line 2447
Stack Trace 	
js_MarkGCThing  [mozilla/js/src/jsgc.c, line 2447]
JS_GC  [mozilla/js/src/jsapi.c, line 1879]
nsAppStartup::Run  [mozilla/toolkit/components/startup/src/nsAppStartup.cpp, line 152]
main  [mozilla/browser/app/nsBrowserApp.cpp, line 61]
kernel32.dll + 0x16fd7 (0x7c816fd7)



js_GetGCThingFlags:

Incident ID: 43401125
Stack Signature	js_GetGCThingFlags 926e32e5
Product ID	Firefox2
Build ID	2008031114
Trigger Time	2008-04-02 08:05:43.0
Platform	Win32
Operating System	Windows NT 5.1 build 2600
Module	js3250.dll + (0001ec8d)
URL visited	
User Comments	
Since Last Crash	201918 sec
Total Uptime	201918 sec
Trigger Reason	Access violation
Source File, Line No.	c:/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/js/src/jsgc.c, line 492
Stack Trace 	
js_GetGCThingFlags  [mozilla/js/src/jsgc.c, line 492]
JS_GC  [mozilla/js/src/jsapi.c, line 1879]
nsAppStartup::Run  [mozilla/toolkit/components/startup/src/nsAppStartup.cpp, line 152]
main  [mozilla/browser/app/nsBrowserApp.cpp, line 61]
kernel32.dll + 0x16fd7 (0x7c816fd7)

Comment 9

[Samuel Sidler (old account; do not CC)]

Also, fwiw, Al submitted a Talkback report earlier today using the testcase in bug 425576 and it generated the following stack (which has the same first two lines in it):

Incident ID: 43622647
Stack Signature	js_MarkGCThing bafa21bc
Product ID	Firefox2
Build ID	2008031114
Trigger Time	2008-04-07 12:41:43.0
Platform	Win32
Operating System	Windows NT 5.1 build 2600
Module	js3250.dll + (0001f93d)
URL visited	https://bugzilla.mozilla.org/show_bug.cgi?id=425576
User Comments	Testing for Sam.
Since Last Crash	72 sec
Total Uptime	72 sec
Trigger Reason	Access violation
Source File, Line No.	c:/builds/tinderbox/Fx-Mozilla1.8-Release/WINNT_5.2_Depend/mozilla/js/src/jsgc.c, line 2447
Stack Trace 	
js_MarkGCThing  [mozilla/js/src/jsgc.c, line 2447]
JS_GC  [mozilla/js/src/jsapi.c, line 1879]
js_Interpret  [mozilla/js/src/jsinterp.c, line 3171]
js_Invoke  [mozilla/js/src/jsinterp.c, line 1403]
js_InternalInvoke  [mozilla/js/src/jsinterp.c, line 1478]
JS_CallFunctionValue  [mozilla/js/src/jsapi.c, line 4362]
nsJSContext::CallEventHandler  [mozilla/dom/src/base/nsJSEnvironment.cpp, line 1493]
nsJSEventListener::HandleEvent  [mozilla/dom/src/events/nsJSEventListener.cpp, line 195]
nsEventListenerManager::HandleEventSubType  [mozilla/content/events/src/nsEventListenerManager.cpp, line 1655]
nsEventListenerManager::HandleEvent  [mozilla/content/events/src/nsEventListenerManager.cpp, line 1762]
nsGlobalWindow::HandleDOMEvent  [mozilla/dom/src/base/nsGlobalWindow.cpp, line 1733]
DocumentViewerImpl::LoadComplete  [mozilla/layout/base/nsDocumentViewer.cpp, line 1017]
nsDocShell::EndPageLoad  [mozilla/docshell/base/nsDocShell.cpp, line 4873]
nsWebShell::EndPageLoad  [mozilla/docshell/base/nsWebShell.cpp, line 673]
nsDocShell::OnStateChange  [mozilla/docshell/base/nsDocShell.cpp, line 4788]
nsDocLoader::FireOnStateChange  [mozilla/uriloader/base/nsDocLoader.cpp, line 1210]
nsDocLoader::doStopDocumentLoad  [mozilla/uriloader/base/nsDocLoader.cpp, line 844]
nsDocLoader::DocLoaderIsEmpty  [mozilla/uriloader/base/nsDocLoader.cpp, line 744]
nsDocLoader::OnStopRequest  [mozilla/uriloader/base/nsDocLoader.cpp, line 665]
nsLoadGroup::RemoveRequest  [mozilla/netwerk/base/src/nsLoadGroup.cpp, line 732]
nsDocument::UnblockOnload  [mozilla/content/base/src/nsDocument.cpp, line 5199]
nsImageLoadingContent::Event::~Event  [mozilla/content/base/src/nsImageLoadingContent.cpp, line 694]
0x778b0c24
0x0020006c
0x38891445


Note that it crashes in js_MarkGCThing. When I crashed on my Mac (Intel; no Talkback), the Mac crash log showed it as crashing at js_GC.

Comment 10

[Igor Bukanov]

(In reply to comment #8)
> Igor, there are stacks available on the smart analysis page:
> http://talkback-public.mozilla.org/reports/firefox/FF20013/smart-analysis.all

Unfortunately those stack are not usable to check if they all comes from the code that walked that list with a bogus node, the lines 2925-2949 from js/src/jsgc.c. As in the example from the comment 9 the stack traces contains:

(In reply to comment #9)
> js_MarkGCThing  [mozilla/js/src/jsgc.c, line 2447]
> JS_GC  [mozilla/js/src/jsapi.c, line 1879]

JS_GC never calls js_MarkGCThing directly, it is js_GC invoked from JS_GC, that can call js_MarkGCThing. So this stack trace is incomplete and comes from compiler optimizations.

> Note that it crashes in js_MarkGCThing. When I crashed on my Mac (Intel; no
> Talkback), the Mac crash log showed it as crashing at js_GC.

This is not surprising. Depending on the compiler/os/script, the crashes can come from js_GC/js_MarkgCThing/js_GetGCThingFlgas. 

Comment 11

[Bob Clary [:bc] (inactive)]

verified1.8.1.15 by bug 425576 comment 51