591879 - Crash in [@ RunnableMethod<mozilla::ipc::SyncChannel, void ( mozilla::ipc::SyncChannel::*)(IPC::Message const&), Tuple1<IPC::Message> >::ReleaseCallee() ]

Status: RESOLVED WORKSFORME

(Core Graveyard :: Plug-ins, defect)

Description

[Scoobidiver (away)]

Build : Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b5pre) Gecko/20100828
Firefox/4.0b5pre

This is a new crash signature which has been introduced by this build and it is
still the #5 crasher for this build.
It appears to be mainly a start-up crash, probably because there is a plugin in the home page.

Signature	RunnableMethod<mozilla::ipc::SyncChannel, void ( mozilla::ipc::SyncChannel::*)(IPC::Message const&), Tuple1<IPC::Message> >::ReleaseCallee()
UUID	4023048a-413b-4189-b82e-6d1182100829
Time 	2010-08-29 05:27:14.98631
Uptime	1466
Last Crash	1471 seconds (24.5 minutes) before submission
Install Age	26537 seconds (7.4 hours) since version was first installed.
Product	Firefox
Version	4.0b5pre
Build ID	20100828040640
Branch	2.0
OS	Windows NT
OS Version	6.1.7600
CPU	x86
CPU Info	GenuineIntel family 6 model 15 stepping 13
Crash Reason	EXCEPTION_ACCESS_VIOLATION
Crash Address	0x669de20c

Crashing Thread
Frame 	Module 	Signature [Expand] 	Source
0 	xul.dll 	RunnableMethod<mozilla::ipc::SyncChannel,void 	ipc/chromium/src/base/task.h:318
1 	xul.dll 	RunnableMethod<mozilla::ipc::SyncChannel,void 	ipc/chromium/src/base/task.h:311
2 	xul.dll 	nsRefPtr<nsPresContext>::~nsRefPtr<nsPresContext> 	obj-firefox/xpcom/build/nsCOMPtr.cpp:81
3 	xul.dll 	nsPluginHost::NewPluginURLStream 	modules/plugin/base/src/nsPluginHost.cpp:2932
4 	xul.dll 	nsPluginHost::GetURLWithHeaders 	modules/plugin/base/src/nsPluginHost.cpp:653
5 	xul.dll 	nsPluginHost::GetURL 	modules/plugin/base/src/nsPluginHost.cpp:612
6 	xul.dll 	MakeNewNPAPIStreamInternal 	modules/plugin/base/src/nsNPAPIPlugin.cpp:586

The regression window is:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=e1d55bbd1d1d&tochange=6e3f6d18c124

Comment 1

[Boris Zbarsky [:bzbarsky]]

This seems likely to be the same issue as bug 591880....

Comment 2

[Benjamin Smedberg]

I suspect this is different. The RunnableMethod at the top is just a comdat-folded symbol, and the nsRefPtr<nsPresContext>::~nsRefPtr<nsPresContext> frame is probably the nsRefPtr<nsPluginStreamListenerPeer> listenerPeer.

I bet loading the minidump into MSVC might give us better data.

Comment 3

[Benjamin Smedberg]

 	xul.dll!RunnableMethod<mozilla::ipc::AsyncChannel,void (__thiscall mozilla::ipc::AsyncChannel::*)(void),Tuple0>::ReleaseCallee()  Line 318	C++
 	xul.dll!RunnableMethod<mozilla::ipc::AsyncChannel,void (__thiscall mozilla::ipc::AsyncChannel::*)(void),Tuple0>::Cancel()  Line 312	C++
 	xul.dll!nsCOMPtr_base::~nsCOMPtr_base()  Line 82	C++
>	xul.dll!nsPluginHost::NewPluginURLStream(aURL={...}, aInstance=0x07ce7150, aListener=0x06e8c900, aPostStream=0x00000000, aHeadersData=0x00000000, aHeadersDataLen=0x07ce7150)  Line 2932	C++
 	xul.dll!nsPluginHost::GetURLWithHeaders(pluginInst=0x0012d050, url=0x07ce7150, target=0x06e8c900, streamListener=0x00000000, altHost=0x07ce7150, referrer=0x07ce7150, forceJSEnabled=0x07ce7150, getHeadersLength=0x07ce7150, getHeaders=0x07ce7150)  Line 653	C++
 	xul.dll!nsPluginHost::GetURL(pluginInst=0x07ce7150, url=0x06e40678, target=0x00000000, streamListener=0x06e8c900, altHost=0x00000000, referrer=0x00000000, forceJSEnabled=0x00000000)  Line 613	C++
 	xul.dll!MakeNewNPAPIStreamInternal(npp=0x07c58200, relativeURL=0x06e40678, target=0x00000000, type=eNPPStreamTypeInternal_Get, bDoNotify=0x00000001, notifyData=0x042d5c00, len=0x00000000, buf=0x00000000, file=0x00)  Line 586	C++
 	xul.dll!mozilla::plugins::parent::_geturlnotify(npp=0x07ce7158, relativeURL=0x06e40678, target=0x00000000, notifyData=0x042d5c00)  Line 968	C++
 	xul.dll!mozilla::plugins::PluginInstanceParent::AnswerPStreamNotifyConstructor(actor=0x042d5c00, url={...}, target={...}, post=false, buffer={...}, file=false, result=0x0012d210)  Line 428	C++
 	xul.dll!mozilla::plugins::PPluginInstanceParent::OnCallReceived(__msg={...}, __reply=0x00000000)  Line 1334	C++
 	xul.dll!mozilla::plugins::PPluginModuleParent::OnCallReceived(__msg={...}, __reply=0x00000000)  Line 596	C++
 	xul.dll!mozilla::ipc::RPCChannel::DispatchIncall(call={...})  Line 511	C++
 	xul.dll!mozilla::ipc::RPCChannel::Incall(call={...}, stackDepth=0x00000001)  Line 497	C++
 	xul.dll!mozilla::ipc::RPCChannel::Call(msg=0x002d5c00, reply=0x0012d3d0)  Line 312	C++
 	xul.dll!mozilla::plugins::PPluginInstanceParent::CallNPP_HandleEvent(event={...}, handled=0x0012d410)  Line 366	C++
 	xul.dll!mozilla::plugins::PluginInstanceParent::NPP_HandleEvent(event=0x0012d898)  Line 807	C++
 	xul.dll!mozilla::plugins::PluginModuleParent::NPP_HandleEvent(instance=0x07ce7158, event=0x0012d898)  Line 507	C++
 	xul.dll!nsNPAPIPluginInstance::HandleEvent(event=0x0012d898, result=0x0012d510)  Line 588	C++
 	xul.dll!nsPluginInstanceOwner::ProcessEvent(anEvent={...})  Line 4738	C++
 	xul.dll!nsPluginInstanceOwner::MouseMove(aMouseEvent=0x01b35408)  Line 3985	C++
 	xul.dll!nsEventListenerManager::HandleEventInternal(aPresContext=0x04511400, aEvent=0x0012d8c0, aDOMEvent=0x0012d690, aCurrentTarget=0x06eedf80, aFlags=0x00000006, aEventStatus=0x0012d694, aPusher=0x0012d67c)  Line 1204	C++
 	xul.dll!nsEventTargetChainItem::HandleEvent(aVisitor={...}, aFlags=0x00000006, aMayHaveNewListenerManagers=0x00000000, aPusher=0x0012d67c)  Line 217	C++
 	xul.dll!nsEventTargetChainItem::HandleEventTargetChain(aVisitor={...}, aFlags=0x00000006, aCallback=0x0012d720, aMayHaveNewListenerManagers=0x00000000, aPusher=0x0012d67c)  Line 343	C++
 	xul.dll!nsEventDispatcher::Dispatch(aTarget=, aPresContext=, aEvent=, aDOMEvent=, aEventStatus=, aCallback=, aTargets=)  Line 632	C++
 	xul.dll!PresShell::HandleEventInternal()  Line 6744	C++
 	xul.dll!PresShell::HandlePositionedEvent(aView=0x02386e20, aTargetFrame=0x00000000, aEvent=0x0012d8c0, aEventStatus=0x0012d7b8)  Line 6586	C++
 	xul.dll!PresShell::HandleEvent(aView=0x039e9a00, aEvent=0x0012d8c0, aEventStatus=0x0012d7b8)  Line 6436	C++
 	xul.dll!nsViewManager::HandleEvent(aView=0x00000000, aEvent=0x0012d8c0)  Line 1123	C++
 	xul.dll!nsViewManager::DispatchEvent(aEvent=0x0012d8c0, aView=0x039e9a00, aStatus=0x0012d848)  Line 1098	C++
 	xul.dll!AttachedHandleEvent(aEvent=0x039e6ce0)  Line 194	C++
 	xul.dll!nsWindow::DispatchEvent(event=0x0012d8c0, aStatus=nsEventStatus_eIgnore)  Line 3536	C++
 	xul.dll!nsWindow::DispatchWindowEvent(event=0x00000000)  Line 3560	C++
 	xul.dll!nsWindow::DispatchMouseEvent(aEventType=0x0000012c, wParam=0x00000000, lParam=0x01460163, aIsContextMenuKey=0x00000000, aButton=0x0000, aInputSource=0x0001)  Line 3982	C++
 	xul.dll!nsWindow::ProcessMessage(msg=0x00000200, wParam=0x00000000, lParam=0x01460163, aRetValue=0x0012da8c)  Line 4815	C++
 	xul.dll!nsWindow::WindowProcInternal(hWnd=0x02b52ca0, msg=0x00000001, wParam=0x00000000, lParam=0x01460163)  Line 4335	C++
 	xul.dll!nsWindow::WindowProc(hWnd=0x000e08b0, msg=0x00000200, wParam=0x00000000, lParam=0x01460163)  Line 4291	C++

From https://crash-stats.mozilla.com/report/index/94fddd74-b596-4caf-8ebd-042d92100830

This points to an over-release of something in that frame. The candidates are:

nsCOMPtr<nsIURI> url;
nsCOMPtr<nsIDocument> doc;
nsCOMPtr<nsIPluginInstanceOwner> owner;
nsCOMPtr<nsIPluginTagInfo> pti
nsCOMPtr<nsIDOMElement> element
nsRefPtr<nsPluginStreamListenerPeer> listenerPeer
nsCOMPtr<nsIInterfaceRequestor> callbacks;
nsCOMPtr<nsIChannel> channel;
nsCOMPtr<nsIHttpChannel> httpChannel

From the disassembly, this appears to be the first nsCOMPtr to be released, which points to `httpChannel`.

Candidates: Bug 589025, bug 589292, which happened to change the IID of nsIChannel, which seems like a bad idea this late in the beta cycle :-(

Scoobidiver, can you tell whether there are any common extensions installed that correlate with these crashes?

Comment 4

[Scoobidiver (away)]

These crashes happens only with build 20100828040640.
It was fixed in build 20100829040614

A new crash signature appears in build 20100829040614 :
RunnableMethod<mozilla::ipc::GeckoChildProcessHost, void ( mozilla::ipc::GeckoChildProcessHost::*)(void), Tuple0>::ReleaseCallee()
UUID    4e8f4f7a-3052-475f-953d-0684d2100830
It was fixed in build 20100830040705

A new crash signature appears in build 20100830040705 :
RunnableMethod<mozilla::ipc::AsyncChannel, void ( mozilla::ipc::AsyncChannel::*)(IPC::Message const&), Tuple1<IPC::Message> >::ReleaseCallee()
UUID	39990e9e-1802-44cf-8dfd-5d1e52100830

Comment 5

[Benjamin Smedberg]

The specific symbol that this crash is charged to will probably change from day
to day, because of differences in comdat folding and PGO generation. That makes
it harder to track, but no less serious.

Comment 6

[Benjamin Smedberg]

Handing this to dwitte for the moment, although without STR this is oogly. Maybe a valgrind run can help.

Comment 7

[Mike Beltzner [:beltzner, not reading bugmail]]

Where are we on this? Blocks beta5+ and not sure what we can do about that besides backing out the patches mentioned in comment 3 ...

Comment 8

[[e-mail]]

The timing of this matches up with the nsIChannel change. We can look at tonight's nightly and see if it goes away. (How fast does that data show up on crash-stats? Within a few hours?)

Comment 9

[[e-mail]]

8/10 reports I looked at had the IDM extension. The other two had none, which could just mean the information didn't get reported. So I think this is down to IDM like the other two.

Comment 10

[Mike Beltzner [:beltzner, not reading bugmail]]

Do we want to make this depend on bug 592125 as well?

Comment 11

[chris hofmann]

the data from the 28th shows only 14 crashes inside 30 seconds so its later in start up, or after the browser is up for a bit.

Correlation to startup or time of session
55 total crashes for RunnableMethod.mozilla::ipc::SyncChannel..void on 20100828-crashdata.csv
14 startup crashes inside 30 sec.
41 startup crashes inside 3 min.


(In reply to comment #9)
> 8/10 reports I looked at had the IDM extension. The other two had none, which
> could just mean the information didn't get reported. So I think this is down 
> to IDM like the other two.

it appears there is a low volume crash that's been around for a while, maybe unconnected to IDM.  Then an explosion of crashes got added to the lower volume. 

20100816 1
20100817 1
20100818 0
20100819 0
20100820 0
20100821 0
20100822 0
20100823 0
20100824 0
20100825 2
20100826 0
20100827 0
20100828 55
20100829 46

> The timing of this matches up with the nsIChannel change. We can look at
> tonight's nightly and see if it goes away. (How fast does that data show up on
> crash-stats? Within a few hours?)

it might take a day or so to see what is going on and watch for the absence of this signature.  build 201008280 is running at about 50 crashes per day on the 28th, and 29th, so we can watch for those numbers to go to zero on new builds if the fix is in.

Comment 12

[[e-mail]]

Not seeing any crashes on the 20100831 nightly. This looks fixed.

Comment 13

[Joshua Mitchell (Inactive)]

Issue is resolved - clearing old keywords - qa-wanted clean-up