Last Comment Bug 719117 - Crash @ mozalloc_abort | NS_DebugBreak_P | nsIFrame::GetOffsetToCrossDoc with abort message: "###!!! ABORT: trying to get the offset between frames in different document hierarchies?"
: Crash @ mozalloc_abort | NS_DebugBreak_P | nsIFrame::GetOffsetToCrossDoc with...
Status: VERIFIED FIXED
[qa+:ashughes]
: crash, regression, reproducible, topcrash
Product: Core
Classification: Components
Component: Layout (show other bugs)
: 10 Branch
: All All
: -- critical (vote)
: mozilla15
Assigned To: Mats Palmgren (:mats)
:
Mentors:
Depends on: 760190 759788
Blocks: 90268 724781
  Show dependency treegraph
 
Reported: 2012-01-18 09:23 PST by Swarnava Sengupta (:Swarnava)
Modified: 2012-05-31 19:17 PDT (History)
23 users (show)
ryanvm: in‑testsuite-
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
affected
affected
+
verified
+
verified


Attachments
wallpaper fix (1.34 KB, patch)
2012-05-20 16:55 PDT, Robert O'Callahan (:roc) (email my personal email if necessary)
mats: review+
Details | Diff | Splinter Review
InstantiatePluginInstance from StopPluginInstance (13.74 KB, text/html)
2012-05-24 16:06 PDT, Mats Palmgren (:mats)
no flags Details
part 1, backout bug 724781 (1.23 KB, patch)
2012-05-25 12:31 PDT, Mats Palmgren (:mats)
roc: review+
benjamin: review+
akeybl: approval‑mozilla‑aurora+
akeybl: approval‑mozilla‑beta+
Details | Diff | Splinter Review
part 2, fix bug 724781 by preventing reentry to DoStopPlugin with a flag (7.01 KB, patch)
2012-05-25 12:34 PDT, Mats Palmgren (:mats)
roc: review+
benjamin: review+
Details | Diff | Splinter Review
part 3, style nits (1.96 KB, patch)
2012-05-25 17:26 PDT, Mats Palmgren (:mats)
roc: review+
Details | Diff | Splinter Review

Description Swarnava Sengupta (:Swarnava) 2012-01-18 09:23:19 PST
This bug was filed from the Socorro interface and is 
report bp-1942a1c2-27bf-4c7e-a036-7233e2120116 .
============================================================= 

0 	mozalloc.dll 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:77
1 	xul.dll 	NS_DebugBreak_P 	xpcom/base/nsDebugImpl.cpp:375
2 	xul.dll 	nsIFrame::GetOffsetToCrossDoc 	
3 	xul.dll 	nsIFrame::GetOffsetToCrossDoc 	layout/generic/nsFrame.cpp:4128
4 	xul.dll 	PluginBoundsEnumerator 	layout/base/nsPresContext.cpp:2354
5 	xul.dll 	nsTHashtable<nsPtrHashKey<nsFontFaceLoader> >::s_EnumStub 	obj-firefox/dist/include/nsTHashtable.h:472
6 	xul.dll 	PL_DHashTableEnumerate 	obj-firefox/xpcom/build/pldhash.cpp:754
7 	xul.dll 	nsTHashtable<nsCookieEntry>::EnumerateEntries 	obj-firefox/dist/include/nsTHashtable.h:241
8 	xul.dll 	nsRootPresContext::GetPluginGeometryUpdates 	layout/base/nsPresContext.cpp:2450
9 	xul.dll 	nsRootPresContext::UpdatePluginGeometry 	
10 	xul.dll 	PresShell::FlushPendingNotifications 	layout/base/nsPresShell.cpp:4128
11 	xul.dll 	nsDocument::FlushPendingNotifications 	content/base/src/nsDocument.cpp:6291
12 	xul.dll 	nsDocument::FlushPendingNotifications 	content/base/src/nsDocument.cpp:6269
13 	xul.dll 	nsDocument::FlushPendingNotifications 	content/base/src/nsDocument.cpp:6269
14 	xul.dll 	nsDocument::FlushPendingNotifications 	content/base/src/nsDocument.cpp:6269
15 	xul.dll 	nsDocument::FlushPendingNotifications 	content/base/src/nsDocument.cpp:6269
16 	xul.dll 	nsCOMPtr<nsIContent>::nsCOMPtr<nsIContent> 	obj-firefox/dist/include/nsCOMPtr.h:583
17 	xul.dll 	nsObjectLoadingContent::GetExistingFrame 	content/base/src/nsObjectLoadingContent.cpp:1803
18 	xul.dll 	nsObjectLoadingContent::EnsureInstantiation 	content/base/src/nsObjectLoadingContent.cpp:919
19 	xul.dll 	nsCOMPtr<nsIObjectLoadingContent>::nsCOMPtr<nsIObjectLoadingContent> 	obj-firefox/dist/include/nsCOMPtr.h:583
20 	xul.dll 	nsHTMLPluginObjElementSH::GetPluginInstanceIfSafe 	dom/base/nsDOMClassInfo.cpp:9415
21 	xul.dll 	nsHTMLPluginObjElementSH::SetupProtoChain 	dom/base/nsDOMClassInfo.cpp:9502
22 	xul.dll 	nsDocLoader::OnStartRequest 	uriloader/base/nsDocLoader.cpp:591
23 	xul.dll 	nsRefPtr<nsDocLoader>::~nsRefPtr<nsDocLoader> 	obj-firefox/dist/include/nsAutoPtr.h:907
24 	xul.dll 	xpc_UnmarkGrayObject 	js/xpconnect/src/xpcpublic.h:181
25 	xul.dll 	XPCWrappedNative::GetJSObject 	js/xpconnect/src/XPCWrappedNative.cpp:2931
26 	xul.dll 	nsPluginProtoChainInstallRunner::Run 	dom/base/nsDOMClassInfo.cpp:9469
27 	xul.dll 	nsContentUtils::RemoveScriptBlocker 	content/base/src/nsContentUtils.cpp:4414
28 	xul.dll 	PresShell::InitialReflow 	layout/base/nsPresShell.cpp:1979
29 	xul.dll 	DocumentViewerImpl::InitPresentationStuff 	
30 	xul.dll 	DocumentViewerImpl::Show 	
31 	xul.dll 	nsFrameLoader::Show 	content/base/src/nsFrameLoader.cpp:814
32 	xul.dll 	nsDocShell::SetVisibility 	docshell/base/nsDocShell.cpp:4932
33 	xul.dll 	nsRefPtr<nsDocLoader>::~nsRefPtr<nsDocLoader> 	obj-firefox/dist/include/nsAutoPtr.h:907
34 	xul.dll 	nsDocLoader::FireOnStateChange 	uriloader/base/nsDocLoader.cpp:1322
35 	xul.dll 	nsGenericElement::AddRef 	content/base/src/nsGenericElement.cpp:4371
36 	xul.dll 	nsGenericHTMLFrameElement::QueryInterface 	content/html/content/src/nsGenericHTMLElement.cpp:3231
37 	xul.dll 	NS_IsMainThread_P 	obj-firefox/xpcom/build/nsThreadUtils.cpp:138
38 	xul.dll 	nsGenericElement::Release 	content/base/src/nsGenericElement.cpp:4373
39 	xul.dll 	nsSubDocumentFrame::ShowViewer 	layout/generic/nsSubDocumentFrame.cpp:231
40 	xul.dll 	AsyncFrameInit::Run 	layout/generic/nsSubDocumentFrame.cpp:148
41 	xul.dll 	nsContentUtils::RemoveScriptBlocker 	content/base/src/nsContentUtils.cpp:4414
42 	xul.dll 	PresShell::InitialReflow 	layout/base/nsPresShell.cpp:1979
43 	xul.dll 	DocumentViewerImpl::InitPresentationStuff 	
44 	xul.dll 	DocumentViewerImpl::Show 	
45 	xul.dll 	nsFrameLoader::Show 	content/base/src/nsFrameLoader.cpp:814
46 	xul.dll 	nsDocShell::SetVisibility 	docshell/base/nsDocShell.cpp:4932
47 	xul.dll 	nsRefPtr<nsDocLoader>::~nsRefPtr<nsDocLoader> 	obj-firefox/dist/include/nsAutoPtr.h:907
48 	xul.dll 	nsDocLoader::FireOnStateChange 	uriloader/base/nsDocLoader.cpp:1322
49 	xul.dll 	nsGenericElement::AddRef 	content/base/src/nsGenericElement.cpp:4371
50 	xul.dll 	nsGenericHTMLFrameElement::QueryInterface 	content/html/content/src/nsGenericHTMLElement.cpp:3231
51 	xul.dll 	NS_IsMainThread_P 	obj-firefox/xpcom/build/nsThreadUtils.cpp:138
52 	xul.dll 	nsGenericElement::Release 	content/base/src/nsGenericElement.cpp:4373
53 	xul.dll 	nsSubDocumentFrame::ShowViewer 	layout/generic/nsSubDocumentFrame.cpp:231
54 	xul.dll 	AsyncFrameInit::Run 	layout/generic/nsSubDocumentFrame.cpp:148
55 	xul.dll 	nsContentUtils::RemoveScriptBlocker 	content/base/src/nsContentUtils.cpp:4414
56 	xul.dll 	PresShell::InitialReflow 	layout/base/nsPresShell.cpp:1979
57 	xul.dll 	DocumentViewerImpl::InitPresentationStuff 	
58 	xul.dll 	DocumentViewerImpl::Show 	
59 	xul.dll 	nsFrameLoader::Show 	content/base/src/nsFrameLoader.cpp:814
60 	xul.dll 	nsDocShell::SetVisibility 	docshell/base/nsDocShell.cpp:4932
61 	xul.dll 	nsRefPtr<nsDocLoader>::~nsRefPtr<nsDocLoader> 	obj-firefox/dist/include/nsAutoPtr.h:907
62 	xul.dll 	nsDocLoader::FireOnStateChange 	uriloader/base/nsDocLoader.cpp:1322
63 	xul.dll 	nsGenericElement::AddRef 	content/base/src/nsGenericElement.cpp:4371
64 	xul.dll 	nsGenericHTMLFrameElement::QueryInterface 	content/html/content/src/nsGenericHTMLElement.cpp:3231
65 	xul.dll 	NS_IsMainThread_P 	obj-firefox/xpcom/build/nsThreadUtils.cpp:138
66 	xul.dll 	nsGenericElement::Release 	content/base/src/nsGenericElement.cpp:4373
67 	xul.dll 	nsSubDocumentFrame::ShowViewer 	layout/generic/nsSubDocumentFrame.cpp:231
68 	xul.dll 	AsyncFrameInit::Run 	layout/generic/nsSubDocumentFrame.cpp:148
69 	xul.dll 	nsContentUtils::RemoveScriptBlocker 	content/base/src/nsContentUtils.cpp:4414
70 	xul.dll 	PresShell::InitialReflow 	layout/base/nsPresShell.cpp:1979
71 	xul.dll 	DocumentViewerImpl::InitPresentationStuff 	
72 	xul.dll 	DocumentViewerImpl::Show 	
73 	xul.dll 	nsFrameLoader::Show 	content/base/src/nsFrameLoader.cpp:814
74 	xul.dll 	nsDocShell::SetVisibility 	docshell/base/nsDocShell.cpp:4932
75 	xul.dll 	nsXBLBinding::ChangeDocument 	content/xbl/src/nsXBLBinding.cpp:1173
76 	xul.dll 	nsCycleCollectingAutoRefCnt::decr 	obj-firefox/dist/include/nsISupportsImpl.h:211
77 	xul.dll 	NS_IsMainThread_P 	obj-firefox/xpcom/build/nsThreadUtils.cpp:138
78 	xul.dll 	AbortIfOffMainThreadIfCheckFast 	xpcom/base/nsCycleCollector.cpp:1287
79 	xul.dll 	nsCycleCollectingAutoRefCnt::incr 	obj-firefox/dist/include/nsISupportsImpl.h:172
80 	xul.dll 	nsGenericElement::AddRef 	content/base/src/nsGenericElement.cpp:4371
81 	xul.dll 	NS_TableDrivenQI 	obj-firefox/xpcom/build/nsISupportsImpl.cpp:49
82 	xul.dll 	nsGenericHTMLFrameElement::QueryInterface 	content/html/content/src/nsGenericHTMLElement.cpp:3231
83 	xul.dll 	NS_IsMainThread_P 	obj-firefox/xpcom/build/nsThreadUtils.cpp:138
84 	xul.dll 	nsGenericElement::Release 	content/base/src/nsGenericElement.cpp:4373
85 	xul.dll 	nsSubDocumentFrame::ShowViewer 	layout/generic/nsSubDocumentFrame.cpp:231
86 	xul.dll 	AsyncFrameInit::Run 	layout/generic/nsSubDocumentFrame.cpp:148
87 	xul.dll 	nsContentUtils::RemoveScriptBlocker 	content/base/src/nsContentUtils.cpp:4414
88 	xul.dll 	PresShell::FlushPendingNotifications 	layout/base/nsPresShell.cpp:4070
89 	xul.dll 	PresShell::WillPaint 	layout/base/nsPresShell.cpp:6904
90 	xul.dll 	nsViewManager::CallWillPaintOnObservers 	view/src/nsViewManager.cpp:1578
91 	xul.dll 	nsViewManager::DispatchEvent 	view/src/nsViewManager.cpp:876
92 	xul.dll 	AttachedHandleEvent 	view/src/nsView.cpp:191
93 	xul.dll 	nsWindow::DispatchEvent 	widget/src/windows/nsWindow.cpp:3597
94 	xul.dll 	nsWindow::DispatchWindowEvent 	widget/src/windows/nsWindow.cpp:3620
95 	xul.dll 	nsWindow::OnPaint 	widget/src/windows/nsWindowGfx.cpp:261
96 	xul.dll 	nsWindow::ProcessMessage 	widget/src/windows/nsWindow.cpp:4885
97 	xul.dll 	nsWindow::WindowProcInternal 	widget/src/windows/nsWindow.cpp:4453
98 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp:65
99 	xul.dll 	nsWindow::WindowProc 	widget/src/windows/nsWindow.cpp:4395
100 	user32.dll 	InternalCallWinProc 	
143 	OFFICE.ODF 	OFFICE.ODF@0x1f6562 	
144 		@0x690fffff 	
145 	dui70.dll 	DirectUI::CallstackTracker::GetModuleBase 	
146 		@0x70f5ffff 	
147 	GROOVEEX.DLL 	GROOVEEX.DLL@0x165273 	
148 	GrooveIntlResource.dll 	GrooveIntlResource.dll@0x604175 	
149 	xul.dll 	mozilla::places::MatchAutoCompleteFunction::OnFunctionCall 	toolkit/components/places/SQLFunctions.cpp:386
150 	mozsqlite3.dll 	sqlite3EndTable 	db/sqlite3/src/sqlite3.c:78959
151 	xul.dll 	mozilla::places::MatchAutoCompleteFunction::OnFunctionCall 	toolkit/components/places/SQLFunctions.cpp:386
152 	xul.dll 	mozilla::places::MatchAutoCompleteFunction::OnFunctionCall 	toolkit/components/places/SQLFunctions.cpp:386
Comment 1 Scoobidiver (away) 2012-01-18 14:42:32 PST
Please be more specific: rank in different versions, different stacks, component, interesting comments, STR, regression range...
Comment 2 Timothy Nikkel (:tnikkel) 2012-02-07 13:01:40 PST
Dupe of bug 621551 ?
Comment 3 Scoobidiver (away) 2012-02-08 00:05:24 PST
(In reply to Timothy Nikkel (:tn) from comment #2)
> Dupe of bug 621551 ?
No. This one is an abort and a regression from 10.0.

It's #180 top crasher in 11.0b1.
On Mac, it first appeared in 10.0a1/20111028 and, on Windows, in 10.0a1/20111011.

More reports at:
https://crash-stats.mozilla.com/report/list?signature=mozalloc_abort%28char%20const*%20const%29%20|%20NS_DebugBreak_P%20|%20nsIFrame%3A%3AGetOffsetToCrossDoc%28nsIFrame%20const*%2C%20int%29
https://crash-stats.mozilla.com/report/list?signature=mozalloc_abort%20|%20NS_DebugBreak_P%20|%20nsIFrame%3A%3AGetOffsetToCrossDoc
Comment 4 Timothy Nikkel (:tnikkel) 2012-02-08 00:09:50 PST
(In reply to Scoobidiver from comment #3)
> (In reply to Timothy Nikkel (:tn) from comment #2)
> > Dupe of bug 621551 ?
> No. This one is an abort and a regression from 10.0.

We changed the code to abort when it detects a bad situation instead of continuing and crashing later in hopes of shedding more light on the issue (it didn't help much). So you should consider the aborts and the crashes as the same. With that understanding has anything changed here (frequency, etc)?
Comment 5 Scoobidiver (away) 2012-02-08 01:02:31 PST
(In reply to Timothy Nikkel (:tn) from comment #4)
> With that understanding has anything changed here (frequency, etc)?
bug 621551 is #99 top crasher in 9.0.1, #511 in the first days of 10.0, and #749 in 11.0b1.
This bug is not present in the first days of 10.0 (maybe hidden by new Layout bugs) and is #180 top crasher in 11.0b1.
Comment 6 Scoobidiver (away) 2012-04-17 11:34:55 PDT
It's #2 top crasher in 13.0a2 and #6 in 14.0a1 on Mac OS X.
64-bit processors are more affected.
Comment 7 Timothy Nikkel (:tnikkel) 2012-04-18 15:53:40 PDT
Last I checked this wasn't super frequent. So it jumped up in frequency?
Comment 8 Scoobidiver (away) 2012-04-19 02:18:49 PDT
(In reply to Timothy Nikkel (:tn) from comment #7)
> Last I checked this wasn't super frequent. So it jumped up in frequency?
Yes. It's #49 top browser crasher in 11.0, #175 in 12.0b5, #2 in 13.0a2 and #4 in 14.0a1 on Mac OS X.
Comment 9 Robert Kaiser 2012-04-19 09:32:10 PDT
(In reply to Timothy Nikkel (:tn) from comment #4)
> We changed the code to abort when it detects a bad situation instead of
> continuing and crashing later in hopes of shedding more light on the issue
> (it didn't help much). So you should consider the aborts and the crashes as
> the same. With that understanding has anything changed here (frequency, etc)?

So, this (mozalloc_abort | NS_DebugBreak_P | nsIFrame::GetOffsetToCrossDoc) changed in frequency between 12 and 13, rising much higher than before. When did the change to abort happen? If it's related to that change, maybe we just shifted signatures around, but if not, it might be some regression.
Comment 10 Timothy Nikkel (:tnikkel) 2012-04-20 00:18:46 PDT
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #9)
> So, this (mozalloc_abort | NS_DebugBreak_P | nsIFrame::GetOffsetToCrossDoc)
> changed in frequency between 12 and 13, rising much higher than before. When
> did the change to abort happen? If it's related to that change, maybe we
> just shifted signatures around, but if not, it might be some regression.

http://hg.mozilla.org/mozilla-central/rev/c52ed5e53db9 made it crash immediately and then http://hg.mozilla.org/mozilla-central/rev/a16971b9a582 made it into an abort. This was May and October of 2011.
Comment 11 Scoobidiver (away) 2012-04-20 01:18:51 PDT
It started spiking on Windows in 13.0a2/20120319 and started on Mac OS X in 13.0a2/20120320, but 13.0a2 has been blocked at the beginning during several days. The regression range is:
http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?fromchange=f1135f99a0e4&tochange=5e80b801b695
Comment 12 Robert Kaiser 2012-04-20 04:48:33 PDT
Timothy, so your dates don't match what Scoobidiver points out, which means the regression in 13 is not directly related with those changes. We'll need to look for something in that range.
Comment 13 Scoobidiver (away) 2012-04-20 23:01:05 PDT
Searching in Nightly I found a spike in crashes from 13.0a1/20120303 on Windows. The regression range for the spike is:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=3a7b9e61c263&tochange=343ec916dfd5
Comment 14 Scoobidiver (away) 2012-05-01 01:59:23 PDT
It's #1 top browser crasher on Mac OS X in 13.0b1 with 7% of all crashes.
Comment 15 Alex Keybl [:akeybl] 2012-05-02 13:23:51 PDT
Jet, we're looking for an assignee on the engineering side to look into why this has spiked recently (not #1 FF13 top crasher!). Comment 13 suggests a possible regression range while FF13 was on nightly.

Scoobidiver, are there any interesting correlations for this crash, or are we confident this most recent spike was caused by an in-product change based upon the spike?
Comment 16 Timothy Nikkel (:tnikkel) 2012-05-02 14:26:38 PDT
If anyone wanted to try to reproduce, things that are mentioned often in the comments are pogo, pressing the back button, and sometimes flashblock.
Comment 17 Timothy Nikkel (:tnikkel) 2012-05-02 14:45:36 PDT
A lot of the crashes, but not all, are in plugin bounds updating. I've looked and can't figure out how we could hit this case during plugin bounds updating. We could paper over this crash by checking what should already be true but we might just move the crash elsewhere, or we might make it crash worse, or we might make things behave improperly.
Comment 18 Scoobidiver (away) 2012-05-03 00:16:08 PDT
(In reply to Alex Keybl [:akeybl] from comment #15)
> Scoobidiver, are there any interesting correlations for this crash, or are
> we confident this most recent spike was caused by an in-product change based
> upon the spike?
According to me, it's a change in the Firefox code that made it spike (see comment 11 and comment 13).

94% of Mac crashes happen on 64-build processors.
In 13.0, it's correlated to the Java plugin on Mac OS X:
  TouchBadMemory | mozalloc_abort | NS_DebugBreak_P | nsIFrame::GetOffsetToCrossDoc|EXC_BAD_ACCESS / KERN_INVALID_ADDRESS (26 crashes)
     58% (15/26) vs.   8% (26/312) JavaAppletPlugin
          0% (0/26) vs.   1% (3/312) 014569F00BCF39E1BB61C7A705C885D90
          0% (0/26) vs.   0% (1/312) 23E077DB9E653C469F9453F29A17E9E30
         42% (11/26) vs.   6% (18/312) 42790EC844EE3511B50CE7B243CC9E330
         15% (4/26) vs.   1% (4/312) EB16CBBDAC7D3EB2B62CB2D2CF720DCA0

Two different users say it occurs while clicking the back button.
Comment 19 Jet Villegas (:jet) 2012-05-03 14:32:35 PDT
+cc: Bas

I started looking for possible suspects in the regression range and found this one:

https://bugzilla.mozilla.org/attachment.cgi?id=598663&action=diff

Can you have a look and see if it's related?
Comment 20 Bas Schouten (:bas.schouten) 2012-05-14 14:28:01 PDT
(In reply to Jet Villegas (:jet) from comment #19)
> +cc: Bas
> 
> I started looking for possible suspects in the regression range and found
> this one:
> 
> https://bugzilla.mozilla.org/attachment.cgi?id=598663&action=diff
> 
> Can you have a look and see if it's related?

I can't see that it's obviously related, but it did touch Mac plugin code, BenWa helped me with that though, and did a bunch of the code. Maybe he has ideas?
Comment 21 Scoobidiver (away) 2012-05-17 01:14:17 PDT
It represents 28% of Mac crashes in 13.0b3.
Another comment says: "Joining WebEx meeting."
Comment 22 Alex Keybl [:akeybl] 2012-05-18 16:13:03 PDT
(In reply to Scoobidiver from comment #21)
> It represents 28% of Mac crashes in 13.0b3.
> Another comment says: "Joining WebEx meeting."

Adding qawanted to try and get our hands on WebEx to help with STR.
Comment 23 Robert O'Callahan (:roc) (email my personal email if necessary) 2012-05-20 16:51:51 PDT
The Mac stacks are more helpful than the Windows stacks:
https://crash-stats.mozilla.com/report/list?product=Firefox&platform=mac&query_search=signature&query_type=contains&query=nsIFrame%3A%3AGetOffsetToCrossDoc&reason_type=contains&date=05%2F20%2F2012%2022%3A44%3A14&range_value=1&range_unit=weeks&hang_type=any&process_type=any&do_query=1&signature=TouchBadMemory%20|%20mozalloc_abort%20|%20NS_DebugBreak_P%20|%20nsIFrame%3A%3AGetOffsetToCrossDoc

None of the reports I looked suggest tab dragging could be involved, so I don't think it's anything to do with Begin/EndSwapDocShells. That means the most likely scenario is that the plugin's frame's GetRootPresContext returns null.

Many of the reports I looked at show the crash happening during  	nsDocShell::RestorePresentationEvent::Run. This happens when the back button triggers a restore from bfcache. I think that could be an important clue. It certainly wouldn't be surprising to have that involved when GetRootPresContext returns null.

However, it seems to me that before any crash in FinishRestore we should have destroyed the previous viewer, which should have calls PresShell::Freeze for the previous document's presentation and all its descendants, which should have called nsObjectLoadingContent::DisconnectFrame on all plugin instances, calling nsObjectFrame::SetInstanceOwner(nsnull) on their frames, which calls UnregisterPluginForGeometryUpdates. So I don't see where the bug is.

If we can't get steps to reproduce, a simple and safe wallpaper patch that would probably fix this would be to have PluginBoundsEnumerator check f->PresContext()->GetRootPresContext(); if that's null, don't do anything.
Comment 24 Robert O'Callahan (:roc) (email my personal email if necessary) 2012-05-20 16:55:52 PDT
Created attachment 625526 [details] [diff] [review]
wallpaper fix

There's still an underlying bug here, but it will be very difficult to find without steps to reproduce.
Comment 25 Simon Howes 2012-05-20 17:12:18 PDT
I've experienced this bug twice in OS X on two machines when clicking to enable Java in the applet.

I'm unsure how to remove the permission for each app to make the enable button reappear. 

You have to enable Java for each app in OS X since http://support.apple.com/kb/DL1515
Comment 26 Mats Palmgren (:mats) 2012-05-20 23:57:41 PDT
(In reply to Simon Howes from comment #25)
> I've experienced this bug twice in OS X on two machines when clicking to
> enable Java in the applet.

Please let us know the URL of the page and which applet you clicked on the page.
Is the crash reproducible?  Please submit the crash data and let us know the
crash report ID (http://kb.mozillazine.org/Breakpad#Viewing_crash_reports).
Thanks.
Comment 27 Mats Palmgren (:mats) 2012-05-21 00:17:14 PDT
Comment on attachment 625526 [details] [diff] [review]
wallpaper fix

As tn@ notes above, the risk is that we continue and crash in a worse way.
But given this is a topcrash, it's probably worth taking.

s/NS_ERROR/NS_ABORT_IF_FALSE(false, / to crash debug builds?

We probably shouldn't take this wallpaper in the Nightly channel as we
still need the crash report feedback with URLs and comments to help us
find the real bug.
Comment 28 Mats Palmgren (:mats) 2012-05-21 00:36:38 PDT
BTW, can someone post a list of the most common URLs from crash data please?

Given that this is a topcrash, it seems that loading a few of those, starting
a plugin in each, and then going back/forward over them could give us a STR.
Comment 29 Benjamin Smedberg [:bsmedberg] 2012-05-23 09:16:42 PDT
This bug may have a similar root cause to bug 757262 (youtube keeps playing after navigation/back), if we're not disconnecting/stopping the plugin instances correctly. As far as I know, pages with plugins are not supposed to be eligible for bfcache, are they?

The regression range in comment 13 points to bug 651192, plugin async drawing model, but reading through the commits there I didn't see any red flags.
Comment 30 Boris Zbarsky [:bz] (TPAC) 2012-05-23 09:31:13 PDT
Pages with plugins are eligible for bfcache; otherwise pretty much no page would ever be (flash ads!).
Comment 31 Robert O'Callahan (:roc) (email my personal email if necessary) 2012-05-23 12:14:56 PDT
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #29)
> As far as I know, pages with plugins are not supposed
> to be eligible for bfcache, are they?

They sure are, but we explicitly shut down all plugin instances when a page goes into the bfcache.
Comment 32 Marcia Knous [:marcia - use ni] 2012-05-24 10:01:34 PDT
Here are some URLs for the first signature:

55 about:blank
26 http://am10.ru/cu_out.php
22 http://www.facebook.com/
17 https://secure.usverify.com/hrmgr/eob?p_action=doLoop
15 https://www.bred.fr//portal_BRED/quitterSession
14 http://www.bearforest.com/tsignout.htm
11 http://www.mameworld.info/ubbthreads/dosearch.php?Cat=&displaytype=&Forum=All_Fo
10 http://oasis.mkcl.org/rayat/PhotoSign/PhotoSign.aspx
10 http://www.google.com/
10 about:home
8  http://www.reddit.com/
8  https://www.facebook.com/
8  http://www.sonymobile.com/global-en/tools/pc-companion/
7  https://www.usvisa-germany.com/germany/index.jsp
7  about:newtab
6  http://evserver:9090/vapost139/
6  http://www.sonymobile.com/de/tools/pc-companion/
6  http://www.ma-config.com/fr/services/60_trouver-les-pilotes.html
6  http://www.digiturkplay.com.tr/CANLI-TV/SHOW-TV/
6  http://www.zamradio.com/besplatna_muzika_4_1.php
5  http://manager.penguinpos.com/~sicom/mgrng/login.php
5  http://www.mercadopago.com/mlb/accountSummary
5  http://media.radiosai.org/www/Telugu.html
4  http://www.commbank.com.au/
4  http://media.radiosai.org/www/Bhajan.html
4  http://de.thewrestlinggame.com/arena/
4  http://www.thehun.net/
4  http://www.tumblr.com/dashboard
4  http://evserver:9090/vapost139/login
4  http://www.youtube.com/
3  http://chelsea.in.th/c_home.php
3  http://mixi.jp/home.pl?from=global
3  http://www.nvidia.de/object/win7-winvista-64bit-301.42-whql-driver-de.html
3  http://trailers.apple.com/trailers/
3  http://www.varzesh3.com/
3  http://bearforest.com/tsignout.htm
3  http://ro.thewrestlinggame.com/arena/
3  http://www.lsusports.net/
3  https://vidis.vid.gov.lv/Alr_User/Pages/Login.aspx
3  http://poll.pollcode.com/qfce
3  http://www.digiturkplay.com.tr/Anasayfa/
3  http://www.5278.cc/forum-23-1.html
Comment 33 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-24 12:07:18 PDT
So far I've been unable to reproduce a crash using the top URL: http://am10.ru/cu_out.php
That said, there has been no plug-ins loaded so I'm not sure how this crash would trigger in the first place (assuming plug-ins are involved).

Further, I tested the top-10 URLs from comment 32, excluding about: pages, and so far have yet to stumble upon a crash. I've been using Firefox 13.0b5, going to the pages, clicking around links trying to get plug-ins to load, using the plug-in for a few minutes when it does, then click back and forward a few times.

So far, no luck. Are there any other leads QA can pursue?
Comment 34 Alice0775 White 2012-05-24 12:29:28 PDT
I can reproduce to crash in
http://hg.mozilla.org/releases/mozilla-beta/rev/eb5336a48ab0
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20100101 Firefox/13.0 ID:20120516113045

STR
1. Open http://bearforest.com/tsignout.htm
2. Click Asia Stream
Comment 35 Alice0775 White 2012-05-24 12:31:47 PDT
(In reply to Alice0775 White from comment #34)
> I can reproduce to crash in
> http://hg.mozilla.org/releases/mozilla-beta/rev/eb5336a48ab0
> Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20100101 Firefox/13.0
> ID:20120516113045
> 
> STR
> 1. Open http://bearforest.com/tsignout.htm
> 2. Click Asia Stream

Err
1. Open http://media.radiosai.org/www/Bhajan.html
Comment 36 Benjamin Smedberg [:bsmedberg] 2012-05-24 12:34:28 PDT
Since this bug may involve document navigation, the URL listed for the crash report may not be relevant to the page which actually caused the crash, which would likely be the previous page in history.

I can't reproduce a crash with Alice's links in nightly builds... :-(
Comment 37 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-24 13:11:41 PDT
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #36)
> I can't reproduce a crash with Alice's links in nightly builds... :-(

Nor can I in Firefox 13.0b5...
Comment 38 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-24 13:23:40 PDT
Okay, I just hit this in Windows 7.

1) Install Flash Player 11.2
2) Start Firefox 13.0b5
3) Open http://media.radiosai.org/www/Bhajan.html
4) Click Asia Stream
> https://crash-stats.mozilla.com/report/index/68f23cc2-a0ea-4ebb-bf62-2f15b2120524

Crash appears to be 100% reproducible.

Dropping qawanted and adding reproducible. Please let me know if there is something more QA can do.
Comment 39 Timothy Nikkel (:tnikkel) 2012-05-24 13:25:58 PDT
I can reproduce on Alice's crash nightly on mac! Wonderful, I've been waiting for a reproducible testcase for this for a long time!

Thanks so much Alice!
Comment 40 Alice0775 White 2012-05-24 13:49:44 PDT
bp-14955002-81ba-485a-aa38-0a1cb2120524

STR
1. Open http://media.radiosai.org/www/Bhajan.html
2. Wait to start sound
3. Open about:crashes in the same tab


Regression window(m-c)
Not crash:
http://hg.mozilla.org/mozilla-central/rev/7672adec56b9
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120301 Firefox/13.0a1 ID:20120301052609
Crash:
http://hg.mozilla.org/mozilla-central/rev/04caf36509e7
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120301 Firefox/13.0a1 ID:20120301052909
Pushlog:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=7672adec56b9&tochange=04caf36509e7

Regression window(m-i)
Not crash:
http://hg.mozilla.org/integration/mozilla-inbound/rev/57cf4086191d
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120229 Firefox/13.0a1 ID:20120229051709
Crash:
http://hg.mozilla.org/integration/mozilla-inbound/rev/d0477ba39a89
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120229 Firefox/13.0a1 ID:20120229081509
Pushlog:
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=57cf4086191d&tochange=d0477ba39a89
Comment 41 Timothy Nikkel (:tnikkel) 2012-05-24 13:54:45 PDT
(In reply to Alice0775 White from comment #40)
> Regression window(m-i)
> Not crash:
> http://hg.mozilla.org/integration/mozilla-inbound/rev/57cf4086191d
> Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120229 Firefox/13.0a1
> ID:20120229051709
> Crash:
> http://hg.mozilla.org/integration/mozilla-inbound/rev/d0477ba39a89
> Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120229 Firefox/13.0a1
> ID:20120229081509
> Pushlog:
> http://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?fromchange=57cf4086191d&tochange=d0477ba39a89

That suggests bug 724781.
Comment 42 Alex Keybl [:akeybl] 2012-05-24 15:26:21 PDT
(In reply to Timothy Nikkel (:tn) from comment #41)
> That suggests bug 724781.

Can we create a try build with bug 724781 backed out? If that resolves the STR in comment 38, we'll do the same for our 6th beta (going to build on Monday).
Comment 43 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-24 15:47:01 PDT
I can test this once a try build is available -- just let me know.
Comment 44 Mats Palmgren (:mats) 2012-05-24 16:06:59 PDT
Created attachment 627002 [details]
InstantiatePluginInstance from StopPluginInstance

It looks like the plugin's Destroy method pokes at something
that triggers nsHTMLPluginObjElementSH::NewResolve to instantiate
the plugin again.  This is because GetPluginInstance() returned
null at line 9577 (because DoStopPlugin nulled mInstanceOwner):
http://mxr.mozilla.org/mozilla-central/source/dom/base/nsDOMClassInfo.cpp#9565
Comment 45 Mats Palmgren (:mats) 2012-05-24 16:18:24 PDT
The patch in bug 724781 seems rather risky to me.  There's many functions
that are protected by "if (!mInstanceOwner)" at entry.  Can we fix bug
724781 with a simple bool flag that just protects StopPluginInstance
from re-entry instead?  That should discard further StopPluginInstance
calls from InDocCheckEvent:
http://mxr.mozilla.org/mozilla-central/source/content/base/src/nsObjectLoadingContent.cpp#146
Comment 46 Robert O'Callahan (:roc) (email my personal email if necessary) 2012-05-24 17:04:19 PDT
You mean don't null out mInstanceOwner before calling DoStopPlugin, and instead have an isDestroying flag to prevent reentry? That sounds good to me. I'd put the flag setting and checking in DoStopPlugin though.
Comment 47 Benjamin Smedberg [:bsmedberg] 2012-05-25 10:52:44 PDT
I don't think it makes sense to back out bug 724781 without some other mitigation, since that is a use-after-free bug. I'll take a stab at writing the comment 46 patch now unless Mats has already done that...
Comment 48 Mats Palmgren (:mats) 2012-05-25 12:15:30 PDT
Yes, I already have a patch for that...
Comment 49 Mats Palmgren (:mats) 2012-05-25 12:31:15 PDT
Created attachment 627321 [details] [diff] [review]
part 1, backout bug 724781

mozilla-beta Try build with part 1 only:
https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mpalmgren@mozilla.com-10cae880234a/
Comment 50 Mats Palmgren (:mats) 2012-05-25 12:34:18 PDT
Created attachment 627324 [details] [diff] [review]
part 2, fix bug 724781 by preventing reentry to DoStopPlugin with a flag

mozilla-beta Try builds with part 1 + 2:
https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mpalmgren@mozilla.com-174800d84600/
Comment 51 Mats Palmgren (:mats) 2012-05-25 12:37:49 PDT
I tried to verify that part 1 crash bug 724781 on OSX, and that part 1 + 2 doesn't,
but it seems the testcase in that bug crashes all my builds, even a current
Nightly build.  So if someone could verify the Try builds I'd appreciate it.
Comment 52 Benjamin Smedberg [:bsmedberg] 2012-05-25 14:21:08 PDT
Comment on attachment 627324 [details] [diff] [review]
part 2, fix bug 724781 by preventing reentry to DoStopPlugin with a flag

FWIW, I'm pretty sure this is right and I think we should get it into nightly ASAP.
Comment 53 Mats Palmgren (:mats) 2012-05-25 15:37:08 PDT
OK, I'll address roc's comments as a follow-up patch if needed.
https://hg.mozilla.org/integration/mozilla-inbound/rev/2067107a71ca
https://hg.mozilla.org/integration/mozilla-inbound/rev/8915b49bc1bc
Comment 54 Robert O'Callahan (:roc) (email my personal email if necessary) 2012-05-25 16:49:40 PDT
Comment on attachment 627324 [details] [diff] [review]
part 2, fix bug 724781 by preventing reentry to DoStopPlugin with a flag

Review of attachment 627324 [details] [diff] [review]:
-----------------------------------------------------------------

Just fix that comment I guess.

::: content/base/src/nsObjectLoadingContent.cpp
@@ +356,5 @@
>  private:
>    nsCOMPtr<nsITimer> mTimer;
>    nsRefPtr<nsPluginInstanceOwner> mInstanceOwner;
> +  nsCOMPtr<nsIObjectLoadingContent> mContentKungFuDeathGrip;
> +  nsObjectLoadingContent* mContent;

Why both? Why not just an nsRefPtr<nsObjectLoadingContent>?
Comment 55 Mats Palmgren (:mats) 2012-05-25 17:02:01 PDT
> Why both? Why not just an nsRefPtr<nsObjectLoadingContent>?

Compile error - something about ambiguous AddRef methods.
I can add the methods to the class, but it didn't seem worth
saving a word on this transient object.  I considered down-
casting but again it didn't seem worth the ugliness.
Comment 56 Robert O'Callahan (:roc) (email my personal email if necessary) 2012-05-25 17:03:11 PDT
It's confusing because it looks like they might be different objects.

I would do the downcast instead.
Comment 57 Mats Palmgren (:mats) 2012-05-25 17:26:33 PDT
Created attachment 627412 [details] [diff] [review]
part 3, style nits
Comment 60 Mats Palmgren (:mats) 2012-05-26 07:34:27 PDT
Comment on attachment 627321 [details] [diff] [review]
part 1, backout bug 724781

[Approval Request Comment]
Bug caused by (feature/regressing bug #): bug 724781
User impact if declined: topcrash, possibly also the source of other problems on youtube (bug 757262), netflix (bug 756250)
Testing completed (on m-c, etc.): not much yet
Risk to taking this patch (and alternatives if risky): the code change as such is low-risk, partly going back to old behaviour, but this is super-regression-prone plugin land so overall I'd say medium risk.
String or UUID changes made by this patch: none

This request is for part 1 + 2 + 3.  I'd like to land this ASAP on
branches to enable QA and others to test the fix and to determine if
bug 757262 / bug 756250 needs additional work.
Comment 61 Mats Palmgren (:mats) 2012-05-26 14:41:03 PDT
It appears this doesn't fix bug 757262.
Comment 62 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-28 11:45:26 PDT
(In reply to Mats Palmgren [:mats] from comment #49)
> Created attachment 627321 [details] [diff] [review]
> part 1, backout bug 724781
> 
> mozilla-beta Try build with part 1 only:
> https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mpalmgren@mozilla.
> com-10cae880234a/

Crash unreproducible in this try build using my test from comment 38 (which is 100% reproducible in Firefox 13 Beta).
Comment 63 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-28 11:47:34 PDT
(In reply to Mats Palmgren [:mats] from comment #50)
> Created attachment 627324 [details] [diff] [review]
> part 2, fix bug 724781 by preventing reentry to DoStopPlugin with a flag
> 
> mozilla-beta Try builds with part 1 + 2:
> https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mpalmgren@mozilla.
> com-174800d84600/

Crash unreproducible in this try build using my test from comment 38 (which is 100% reproducible in Firefox 13 Beta).
Comment 64 Alex Keybl [:akeybl] 2012-05-28 12:04:32 PDT
Comment on attachment 627321 [details] [diff] [review]
part 1, backout bug 724781

[Triage Comment]
We've weighed the risk/reward extensively.

Reasons to take this in order of priority:
* It's the  #1 top crasher on OS X
* This is a new regression in FF13, so we don't know how the signature will move post-release (but my intuition tells me higher)
* Affected plugins/URLs appear to crash reproducibly, which means users may not be able to use that content ever in FF13
* It's the #17 top crasher on Windows
* We do have QA testing around plugins/platforms, and can ask our Romanian overnight QA to do extensive testing of plugins/platforms

Reasons not to take this:
* We're fairly confident this won't fix bugs 757262/719117
* Even with QA testing, you suspect that this would be a low to medium risk landing

Given that, we've approved for both Aurora 14 and Beta 13.
Comment 66 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-29 10:29:11 PDT
Using testcase in comment 38:
* Firefox 13.0b6 does not crash
* Firefox 14.0a2 2012-05-29 does not crash
* Firefox 15.0a1 2012-05-29 does not crash

Marking this bug verified. Please re-add qawanted if there is something more QA can do here.
Comment 67 juan becerra [:juanb] 2012-05-29 11:54:32 PDT
Verified on Mac OS 10.6.8 using the latest beta, aurora, and nightly. Week-old versions crashed, most recent versions (May 29th) did not.

For whatever reason, I was not able to reproduce the problem on Fx12.
Comment 68 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-29 12:00:57 PDT
Thanks Juan!

Note You need to log in before you can comment on or make changes to this bug.