494617 - Crash [@ memmove | nsTArray_base::ShiftData | nsXULDocument::ResolveForwardReferences] with overlay, observes, event handlers removing window and xmlhttprequest

Status: VERIFIED FIXED

(Core :: XUL, defect)

Description

[Martijn Wargers (dead)]

Attachment: zipped up testase

See zipped up testcase, which crashes current trunk build and Firefox3.0.x after 1 second.
To reproduce, extract the zipped up testcase, then open up the file named 'parentframe.htm'.

http://crash-stats.mozilla.com/report/index/d45a612e-9960-4f8a-9244-833ef2090523?p=1
0  	mozcrt19.dll  	memmove  	 MEMCPY.ASM:188
1 	xul.dll 	nsTArray_base::ShiftData 	obj-firefox/xpcom/build/nsTArray.cpp:173
2 	xul.dll 	nsXULDocument::ResolveForwardReferences 	content/xul/document/src/nsXULDocument.cpp:1210
3 	xul.dll 	nsXULDocument::ResumeWalk 	content/xul/document/src/nsXULDocument.cpp:3136
4 	xul.dll 	nsXULDocument::OnPrototypeLoadDone 	content/xul/document/src/nsXULDocument.cpp:625
5 	xul.dll 	nsXULDocument::EndLoad 	content/xul/document/src/nsXULDocument.cpp:608
6 	xul.dll 	XULContentSinkImpl::DidBuildModel 	content/xul/document/src/nsXULContentSink.cpp:278
7 	xul.dll 	nsExpatDriver::DidBuildModel 	parser/htmlparser/src/nsExpatDriver.cpp:1319
8 	xul.dll 	nsParser::DidBuildModel 	parser/htmlparser/src/nsParser.cpp:1563
9 	xul.dll 	nsParser::ResumeParse 	parser/htmlparser/src/nsParser.cpp:2312
10 	nspr4.dll 	_MD_CURRENT_THREAD 	nsprpub/pr/src/md/windows/w95thred.c:308
11 	xul.dll 	nsBaseChannel::OnStopRequest 	netwerk/base/src/nsBaseChannel.cpp:680
12 	xul.dll 	nsInputStreamPump::OnStateStop 	netwerk/base/src/nsInputStreamPump.cpp:576
13 	xul.dll 	nsInputStreamPump::OnInputStreamReady 	netwerk/base/src/nsInputStreamPump.cpp:401
14 	xul.dll 	nsOutputStreamReadyEvent::Run 	xpcom/io/nsStreamUtils.cpp:190
15 	xul.dll 	nsThread::ProcessNextEvent 	xpcom/threads/nsThread.cpp:510
16 	xul.dll 	NS_ProcessNextEvent_P 	obj-firefox/xpcom/build/nsThreadUtils.cpp:230
17 	xul.dll 	xul.dll@0x3dac0f 	
18 	xul.dll 	NS_InvokeByIndex_P 	xpcom/reflect/xptcall/src/md/win32/xptcinvoke.cpp:101
19 	xul.dll 	XPCWrappedNative::CallMethod 	js/src/xpconnect/src/xpcwrappednative.cpp:2491

Firefox3 breakpad report:
http://crash-stats.mozilla.com/report/index/b972a51b-44bf-401c-8131-026ae2090523?p=1
0  	 	@0x325dc0

Comment 1

[Martijn Wargers (dead)]

I don't seem to be able to crash in a mac build. In a windows debug build, I see these assertions prior to the crash:
###!!! ASSERTION: Already have a document.  Unbind first!: '!GetCurrentDoc()', f
ile c:/mozilla-build-1.3/src/content/base/src/nsGenericElement.cpp, line 2520
###!!! ASSERTION: Already have an undisplayed context entry for aContent: '!GetU
ndisplayedContent(aContent)', file c:/mozilla-build-1.3/src/layout/base/nsFrameM
anager.cpp, line 593

Comment 2

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Is there any particular reason why this should block 1.9.1?

Comment 3

[Daniel Veditz [:dveditz]]

In Firefox 3.0.x I got a similar stack (single-frame, 0x31fe50) but windbg had slightly more information (fonts? Something Cycle-collected?). It was a DEP violation which are usually pretty trivially exploited with heapsprays.

After changing the MIME type to application/java-archive I was able to get the testcase to work without saving it:
 jar:https://bugzilla.mozilla.org/attachment.cgi?id=379372!/parentframe.htm

Had to refresh a few times before it crashed.

0:000> !exploitable -v
Event Type: Exception
Exception Faulting Address: 0x31fe50
First Chance Exception Type: STATUS_ACCESS_VIOLATION (0xC0000005)
Exception Sub-Type: Data Execution Protection (DEP) Violation

Exception Hash (Major/Minor): 0x2b336c41.0x66143413

Stack Trace:
Unknown
xul!gfxTextRun::HasDetailedGlyphs+0xac26
xul!NS_CycleCollectorForget_P+0xb60
xul!gfxTextRun::Create+0x4d2e
xul!gfxWindowsPlatform::ResolveFontName+0x61e7
xul!gfxTextRun::Create+0x4e06
xul!gfxTextRun::gfxTextRun+0x2fd1
xul!gfxWindowsFontGroup::GetFontAt+0x6c0
Instruction Address: 0x31fe50

Description: Data Execution Prevention Violation
Short Description: DEPViolation
Exploitability Classification: EXPLOITABLE
Recommended Bug Title: Exploitable - Data Execution Prevention Violation starting at Unknown Symbol @ 0x92e1dcffff000a (Hash=0x2b336c41.0x66143413)

User mode DEP access violations are exploitable.

Comment 4

[Martijn Wargers (dead)]

I thought this might be related to bug 485799, and according to bug 485799, comment 2, it was a topcrash (at least back then).

Comment 5

[Martijn Wargers (dead)]

Attachment: testcase2, zipped, using enhanced privileges

I also crash with this testcase (open the file named 'parentframe2.htm'). It seems like the same kind of crash to me, because it also uses xmlhttprequest and the first part of the stack is the same. Although, it doesn't seem to crash in Firefox 3.0.x.

http://crash-stats.mozilla.com/report/index/d36f7528-9c0d-4158-8459-08b6c2090604?p=1
0  	mozcrt19.dll  	memmove  	 MEMCPY.ASM:188
1 	xul.dll 	nsTArray_base::ShiftData 	obj-firefox/xpcom/build/nsTArray.cpp:173
2 	xul.dll 	nsTArray<nsAutoPtr<PresShell::nsDelayedEvent> >::RemoveElementsAt 	obj-firefox/dist/include/nsTArray.h:664
3 	xul.dll 	nsTArray<nsAutoPtr<PresShell::nsDelayedEvent> >::RemoveElementAt 	obj-firefox/dist/include/nsTArray.h:669
4 	xul.dll 	xul.dll@0x332e9f 	
5 	xul.dll 	FireOrClearDelayedEvents 	content/base/src/nsDocument.cpp:7508
6 	xul.dll 	nsDelayedEventDispatcher::Run 	content/base/src/nsDocument.cpp:7525
7 	xul.dll 	nsThread::ProcessNextEvent 	xpcom/threads/nsThread.cpp:510
8 	xul.dll 	nsBaseAppShell::Run 	widget/src/xpwidgets/nsBaseAppShell.cpp:170
9 	xul.dll 	nsAppStartup::Run 	toolkit/components/startup/src/nsAppStartup.cpp:193
10 	nspr4.dll 	PR_GetEnv 	
11 	firefox.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp:110
12 	firefox.exe 	firefox.exe@0x21a7 	
13 	kernel32.dll 	kernel32.dll@0x17076

Comment 6

[Mats Palmgren (inactive)]

Attachment: trace + stack

There's a nested call to nsXULDocument::ResolveForwardReferences() for the
same document.

Comment 7

[Mats Palmgren (inactive)]

Attachment: wip

This fixes the first testcase.  I can't reproduce the second
(had to fix the file suffix to .htm -- maybe wrong file? ...
got sendKeyEvent exception even with right priviligies...)

Comment 8

[Martijn Wargers (dead)]

(In reply to comment #7)
> I can't reproduce the second
> (had to fix the file suffix to .htm -- maybe wrong file? ...

Sorry, my description of how to reproduce was insufficient.
While you have the 'parentframe.htm' file opened, you have to keep the tab key pressed.

Comment 9

[Martijn Wargers (dead)]

Comment on attachment 381527 [details]
testcase2, zipped, using enhanced privileges

It turns out this is bug 498530 (and I screwed up with this testcase anyway, adding the wrong file).
I attached the correct testcase in bug 498530.

Comment 10

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

(In reply to comment #2)
> Is there any particular reason why this should block 1.9.1?

Topcrash for the past week at #2 for "Results within 1 weeks of now, and the
product is one of Firefox, and the version is one of Firefox:3.5." - now numbers >4,400 crashes.

I checked a few stacks at:

http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A3.5&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=memmove

and their first and second lines seem to be at memmove and nsTArray_base::ShiftData - nominating blocking1.9.1.1? just-in-case. It's been a month ever since any activity on this bug.

Comment 11

[Samuel Sidler (old account; do not CC)]

Mats: Any update on this bug? Given our focus on stability for Firefox 3.5.1, it's now blocking that release.

Comment 12

[Mike Beltzner [:beltzner, not reading bugmail]]

Mats: are you still working on this issue?
jst: can you either follow up with mats or reassign? this is a topcrash which we need to fix for the 3.5.2 release at the end of the month

Comment 13

[Martijn Wargers (dead)]

I don't think this is a topcrash, the topcrash seems to be something with a "too low memory" alert or something.

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #13)
> I don't think this is a topcrash, the topcrash seems to be something with a
> "too low memory" alert or something.

Those stacks looks indeed different: http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A3.5.1&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=memmove

Martijn, shall I file a new bug? I can't find a related one yet.

Comment 15

[Martijn Wargers (dead)]

Sure, you can do that. But a testcase for it, would be nice.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I have taken a look at several of those top crashers and all have the same signature in the first two frames:

0  	mozcrt19.dll  	memmove  	MEMCPY.ASM:188
1 	xul.dll 	nsTArray_base::ShiftData 	obj-firefox/xpcom/build/nsTArray.cpp:173

I'll not create a new bug for now. Anyone who can tell for sure how reliable the 2nd frame is to determine if both are the same issue or not? timeless?

Comment 17

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

If it's only those two frames that match, you should definitely file a separate bug.  If it's also from nsXULDocument::ResolveForwardReferences there's a chance it could be the same as this, but it's probably still worth a separate bug.

Comment 18

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I have filed bug 507114 on the topcrasher.

Comment 19

[Samuel Sidler (old account; do not CC)]

This will have to wait.

Mats/jst: Who's working on this?

Comment 20

[Johnny Stenback (:jst)]

Mats, feel free to jump in here if you have cycles to spend on this, but we need to get this moving. Smaug, do you think you could spend some time looking at this topcrasher?

Comment 21

[Olli Pettay [:smaug][[e-mail]]]

I can look at this during this weekend.

Comment 22

[Olli Pettay [:smaug][[e-mail]]]

I can't reproduce this on trunk using the first testcase.

Comment 23

[Olli Pettay [:smaug][[e-mail]]]

Can't reproduce on 1.9.1 (OSX) either. Will try linux.

Comment 24

[Olli Pettay [:smaug][[e-mail]]]

Ok, crashes on linux.

Comment 25

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 381988 [details] [diff] [review]
wip

I think we should take Mats' patch.

Comment 26

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 381988 [details] [diff] [review]
wip

Or Mats, what do you say?

Comment 27

[Johnny Stenback (:jst)]

Comment on attachment 381988 [details] [diff] [review]
wip

Seems fine to me.

Comment 28

[Samuel Sidler (old account; do not CC)]

Mats?

Comment 29

[Mats Palmgren (inactive)]

Comment on attachment 381988 [details] [diff] [review]
wip

This patch still makes sense to me, but I have to admit I don't know
this part of the code very well.  FWIW, it fixes the crash and pass
unit tests on TryServer.  I'll push it to m-c tomorrow.

Comment 30

[Mats Palmgren (inactive)]

http://hg.mozilla.org/mozilla-central/rev/6d9356680430

Comment 31

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 381988 [details] [diff] [review]
wip

a1.9.2=dbaron

Comment 32

[Mats Palmgren (inactive)]

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/7f320fdcea0a

Comment 33

[Daniel Veditz [:dveditz]]

Comment on attachment 381988 [details] [diff] [review]
wip

This doesn't appear to fix the topcrash so we don't need to cram this in after the code freeze. We'll catch it in the next round (tree opens in early Oct)

Comment 34

[Daniel Veditz [:dveditz]]

Comment on attachment 381988 [details] [diff] [review]
wip

Approved for 1.9.1.5 and 1.9.0.16, a=dveditz for release-drivers

Comment 35

[Mats Palmgren (inactive)]

http://hg.mozilla.org/releases/mozilla-1.9.1/rev/8f3cbd759f80

On CVS trunk:
mozilla/content/xul/document/src/nsXULDocument.cpp 	1.838

Comment 36

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Verified fixed on trunk and 1.9.2 with:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20091108 Minefield/3.7a1pre (.NET CLR 3.5.30729) ID:20091108045014

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2b2pre) Gecko/20091106 Namoroka/3.6b2pre (.NET CLR 3.5.30729) ID:20091106050124

Comment 37

[Al Billings [:abillings - ex-MoCo]]

Verified for 1.9.1 with:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6pre) Gecko/20091119 Shiretoko/3.5.6pre (.NET CLR 3.5.30729)

Verified for 1.9.0. with:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.16pre) Gecko/2009111921 GranParadiso/3.0.16pre (.NET CLR 3.5.30729)

using attached testcase after verifying crash with testcase in 1.9.1.5 and 1.9.0.15.