Closed Bug 1021053 Opened 8 years ago Closed 8 years ago

abort/crash in mozilla::ipc::MessageChannel::Send with "ABORT: nested sync messages are not supported" with OMTC enabled


(Core :: Graphics: Layers, defect)

32 Branch
Windows NT
Not set



Tracking Status
firefox32 --- unaffected
firefox33 + verified
firefox34 + verified


(Reporter: jbecerra, Assigned: jimm)



(Keywords: crash, topcrash-thunderbird, topcrash-win)

Crash Data

This bug was filed from the Socorro interface and is 
report bp-457d54c1-0e63-41c6-a535-171f32140529.

This signature has been around for a while, but it spiked around 5/21, and it is the #1 top crasher on nightly. It happens mostly on Windows XP, and there are a handful of comments, one of which says "Apparently using dexpot to change a private session window makes nightly crash almost at each time", which I haven't tried yet.

More reports at:

0 	mozalloc.dll 	mozalloc_abort(char const * const) 	memory/mozalloc/mozalloc_abort.cpp
1 	xul.dll 	NS_DebugBreak 	xpcom/base/nsDebugImpl.cpp
2 	xul.dll 	mozilla::ipc::MessageChannel::DebugAbort(char const *,int,char const *,char const *,bool) 	ipc/glue/MessageChannel.cpp
3 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message *,IPC::Message *) 	ipc/glue/MessageChannel.cpp
4 	xul.dll 	mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,mozilla::layers::TargetConfig const &,bool const &,bool const &,unsigned int const &,nsTArray<mozilla::layers::EditReply> *) 	obj-firefox/ipc/ipdl/PLayerTransactionChild.cpp
5 	xul.dll 	mozilla::layers::ShadowLayerForwarder::EndTransaction(nsTArray<mozilla::layers::EditReply> *,nsIntRegion const &,bool,unsigned int,bool *) 	gfx/layers/ipc/ShadowLayers.cpp
6 	xul.dll 	mozilla::layers::ClientLayerManager::ForwardTransaction(bool) 	gfx/layers/client/ClientLayerManager.cpp
7 	xul.dll 	mozilla::layers::ClientLayerManager::EndTransaction(void (*)(mozilla::layers::ThebesLayer *,gfxContext *,nsIntRegion const &,mozilla::layers::DrawRegionClip,nsIntRegion const &,void *),void *,mozilla::layers::LayerManager::EndTransactionFlags) 	gfx/layers/client/ClientLayerManager.cpp
8 	xul.dll 	nsDisplayList::PaintForFrame(nsDisplayListBuilder *,nsRenderingContext *,nsIFrame *,unsigned int) 	layout/base/nsDisplayList.cpp
9 	xul.dll 	nsLayoutUtils::PaintFrame(nsRenderingContext *,nsIFrame *,nsRegion const &,unsigned int,unsigned int) 	layout/base/nsLayoutUtils.cpp
10 	xul.dll 	PresShell::Paint(nsView *,nsRegion const &,unsigned int) 	layout/base/nsPresShell.cpp
11 	xul.dll 	nsViewManager::ProcessPendingUpdatesPaint(nsIWidget *) 	view/src/nsViewManager.cpp
12 	xul.dll 	nsViewManager::ProcessPendingUpdatesForView(nsView *,bool) 	view/src/nsViewManager.cpp
13 	xul.dll 	nsViewManager::ProcessPendingUpdates() 	view/src/nsViewManager.cpp
14 	xul.dll 	nsRefreshDriver::Tick(__int64,mozilla::TimeStamp) 	layout/base/nsRefreshDriver.cpp
15 	xul.dll 	mozilla::RefreshDriverTimer::Tick() 	layout/base/nsRefreshDriver.cpp
16 	xul.dll 	nsTimerImpl::Fire() 	xpcom/threads/nsTimerImpl.cpp
17 	xul.dll 	nsTimerEvent::Run() 	xpcom/threads/nsTimerImpl.cpp
18 	xul.dll 	nsThread::ProcessNextEvent(bool,bool *) 	xpcom/threads/nsThread.cpp
19 	xul.dll 	NS_ProcessPendingEvents(nsIThread *,unsigned int) 	xpcom/glue/nsThreadUtils.cpp
20 	xul.dll 	nsBaseAppShell::NativeEventCallback() 	widget/xpwidgets/nsBaseAppShell.cpp
21 	xul.dll 	nsAppShell::EventWindowProc(HWND__ *,unsigned int,unsigned int,long) 	widget/windows/nsAppShell.cpp
22 	user32.dll 	InternalCallWinProc 	
23 	user32.dll 	UserCallWinProcCheckWow 	
24 	user32.dll 	DispatchClientMessage 	
25 	user32.dll 	__fnDWORD 	
26 	ntdll.dll 	KiUserCallbackDispatcher 	
27 	xul.dll 	nsBaseAppShell::NativeEventCallback() 	widget/xpwidgets/nsBaseAppShell.cpp
28 	user32.dll 	RealDefWindowProcW 	
29 	user32.dll 	DefWindowProcW 	
30 	xul.dll 	`anonymous namespace'::ProcessOrDeferMessage(HWND__ *,unsigned int,unsigned int,long) 	ipc/glue/WindowsMessageLoop.cpp
31 	xul.dll 	NeuteredWindowProc(HWND__ *,unsigned int,unsigned int,long) 	ipc/glue/WindowsMessageLoop.cpp
32 	user32.dll 	InternalCallWinProc 	
33 	user32.dll 	UserCallWinProcCheckWow 	
34 	user32.dll 	DispatchClientMessage 	
35 	user32.dll 	__fnDWORD 	
36 	ntdll.dll 	KiUserCallbackDispatcher 	
37 	xul.dll 	`anonymous namespace'::ProcessOrDeferMessage(HWND__ *,unsigned int,unsigned int,long) 	ipc/glue/WindowsMessageLoop.cpp
38 	user32.dll 	NtUserPeekMessage 	
39 	xul.dll 	mozilla::ipc::MessageChannel::WaitForSyncNotify() 	ipc/glue/WindowsMessageLoop.cpp
40 	xul.dll 	mozilla::ipc::MessageChannel::SendAndWait(IPC::Message *,IPC::Message *) 	ipc/glue/MessageChannel.cpp
41 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message *,IPC::Message *) 	ipc/glue/MessageChannel.cpp
42 	xul.dll 	mozilla::layers::PCompositorChild::SendFlushRendering() 	obj-firefox/ipc/ipdl/PCompositorChild.cpp
43 	xul.dll 	nsViewManager::Refresh(nsView *,nsIntRegion const &) 	view/src/nsViewManager.cpp
44 	xul.dll 	nsViewManager::PaintWindow(nsIWidget *,nsIntRegion) 	view/src/nsViewManager.cpp
45 	xul.dll 	nsView::PaintWindow(nsIWidget *,nsIntRegion) 	view/src/nsView.cpp
46 	xul.dll 	nsWindow::OnPaint(HDC__ *,unsigned int) 	widget/windows/nsWindowGfx.cpp
47 	xul.dll 	nsWindow::ProcessMessage(unsigned int,unsigned int &,long &,long *) 	widget/windows/nsWindow.cpp
48 	xul.dll 	nsWindow::WindowProcInternal(HWND__ *,unsigned int,unsigned int,long) 	widget/windows/nsWindow.cpp
49 	xul.dll 	CallWindowProcCrashProtected 	xpcom/base/nsCrashOnException.cpp
50 	xul.dll 	nsWindow::WindowProc(HWND__ *,unsigned int,unsigned int,long) 	widget/windows/nsWindow.cpp
51 	user32.dll 	InternalCallWinProc 	
52 	user32.dll 	UserCallWinProcCheckWow 	
53 	user32.dll 	DispatchClientMessage 	
54 	user32.dll 	__fnDWORD 	
55 	ntdll.dll 	KiUserCallbackDispatcher 	
56 	xul.dll 	nsIFrame::IsVisibleForPainting(nsDisplayListBuilder *) 	layout/generic/nsFrame.cpp
57 	user32.dll 	DispatchMessageW 	
58 	xul.dll 	nsAppShell::ProcessNextNativeEvent(bool) 	widget/windows/nsAppShell.cpp
59 	xul.dll 	nsBaseAppShell::OnProcessNextEvent(nsIThreadInternal *,bool,unsigned int) 	widget/xpwidgets/nsBaseAppShell.cpp
60 	xul.dll 	nsThread::ProcessNextEvent(bool,bool *) 	xpcom/threads/nsThread.cpp
61 	xul.dll 	NS_ProcessNextEvent(nsIThread *,bool) 	xpcom/glue/nsThreadUtils.cpp
62 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate *) 	ipc/glue/MessagePump.cpp
63 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/
64 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/
65 	xul.dll 	nsBaseAppShell::Run() 	widget/xpwidgets/nsBaseAppShell.cpp
66 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp
67 	xul.dll 	nsAppStartup::Run() 	toolkit/components/startup/nsAppStartup.cpp
68 	xul.dll 	XREMain::XRE_mainRun() 	toolkit/xre/nsAppRunner.cpp
69 	xul.dll 	XREMain::XRE_main(int,char * * const,nsXREAppData const *) 	toolkit/xre/nsAppRunner.cpp
70 	xul.dll 	XRE_main 	toolkit/xre/nsAppRunner.cpp
71 	firefox.exe 	do_main 	browser/app/nsBrowserApp.cpp
72 	firefox.exe 	NS_internal_main(int,char * *) 	browser/app/nsBrowserApp.cpp
73 	firefox.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp
74 	firefox.exe 	__tmainCRTStartup 	f:/dd/vctools/crt_bld/self_x86/crt/src/crtexe.c:552
75 	kernel32.dll 	BaseProcessStart
###!!! ABORT: nested sync messages are not supported: file c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\ipc\glue\MessageChannel.cpp, line 1732
Summary: crash in mozalloc_abort(char const* const) | NS_DebugBreak | mozilla::ipc::MessageChannel::DebugAbort(char const*, int, char const*, char const*, bool) | mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*) → abort/crash in mozilla::ipc::MessageChannel::Send with "ABORT: nested sync messages are not supported"
bsmedberg says this is most likely an issue of OMTC on Windows. As the signature summary says the vast majority of those are WinXP, setting dependency for bug 899787.
Blocks: 899787
Keywords: topcrashtopcrash-win
#1 on Nightly, Fx 33
If I'm reading Socorro correctly, it looks like this crash still exists on 32, although to a lesser extent, even after OMTC was disabled.

Kairo - Can you please confirm?
Flags: needinfo?(kairo)
(In reply to Lawrence Mandel [:lmandel] from comment #4)
> If I'm reading Socorro correctly, it looks like this crash still exists on
> 32, although to a lesser extent, even after OMTC was disabled.
> Kairo - Can you please confirm?

I actually can't confirm, I can verify that it has been fixed by the disabling.

Look at*+const%29+|+NS_DebugBreak+|+mozilla%3A%3Aipc%3A%3AMessageChannel%3A%3ADebugAbort%28char+const*%2C+int%2C+char+const*%2C+char+const*%2C+bool%29+|+mozilla%3A%3Aipc%3A%3AMessageChannel%3A%3ASend%28IPC%3A%3AMessage*%2C+IPC%3A%3AMessage*%29&version=32.0a2&date=%3E%3D2014-06-01&_facets=build_id&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform and the "build ID facet". If you sort by build ID, it's easily visible that the newest build ID this is seen with is 20140626004001.
Flags: needinfo?(kairo)
I'm dropping the tracking flag from 32 and marking as unaffected. We'll need to address this in 33.
#1 crash also for nightly Thunderbird.
Summary: abort/crash in mozilla::ipc::MessageChannel::Send with "ABORT: nested sync messages are not supported" → abort/crash in mozilla::ipc::MessageChannel::Send with "ABORT: nested sync messages are not supported" with OMTC enabled
Here is a way to reproduce the crash, using Suspend Tab add-on:

1. Open a private window
2. Open a new tab
3. Right-click the other tab and suspend it
4. Try to close the private window

Then, the private window won't close at first. After some attempts, Firefox crashes.
See Also: → 1041457
Here's an odd stack -

The whole point of WindowsMessageLoop is to prevent this from happening, although I'm not sure how. We dump an event to DefWindowProcW, and somehow that ends up calling nsAppShell::EventWindowProc. This isn't supposed to happen.

27 	xul.dll 	NS_ProcessPendingEvents(nsIThread*, unsigned int) 	xpcom/glue/nsThreadUtils.cpp
28 	xul.dll 	nsBaseAppShell::NativeEventCallback() 	widget/xpwidgets/nsBaseAppShell.cpp
29 	xul.dll 	nsAppShell::EventWindowProc(HWND__*, unsigned int, unsigned int, long) 	widget/windows/nsAppShell.cpp
30 	user32.dll 	InternalCallWinProc 	
31 	user32.dll 	UserCallWinProcCheckWow 	
32 	user32.dll 	DispatchClientMessage 	
33 	user32.dll 	__fnDWORD 	
34 	ntdll.dll 	KiUserCallbackDispatcher 	
35 	xul.dll 	mozilla::css::SetStyleSheetReference 	layout/style/nsCSSRules.cpp
36 	user32.dll 	RealDefWindowProcW 	
37 	user32.dll 	DefWindowProcW 	
38 	xul.dll 	`anonymous namespace'::ProcessOrDeferMessage(HWND__*, unsigned int, unsigned int, long) 	ipc/glue/WindowsMessageLoop.cpp
39 	xul.dll 	NeuteredWindowProc(HWND__*, unsigned int, unsigned int, long) 	ipc/glue/WindowsMessageLoop.cpp
40 	user32.dll 	InternalCallWinProc 	
41 	user32.dll 	UserCallWinProcCheckWow 	
42 	user32.dll 	DispatchClientMessage 	
43 	user32.dll 	__fnDWORD 	
44 	ntdll.dll 	KiUserCallbackDispatcher 	
45 	xul.dll 	`anonymous namespace'::ProcessOrDeferMessage(HWND__*, unsigned int, unsigned int, long) 	ipc/glue/WindowsMessageLoop.cpp
46 	user32.dll 	NtUserPeekMessage 	
47 	xul.dll 	mozilla::ipc::MessageChannel::WaitForSyncNotify()
(In reply to Jim Mathies [:jimm] from comment #9)
> The whole point of WindowsMessageLoop is to prevent this from happening,
> although I'm not sure how. We dump an event to DefWindowProcW, and somehow

ehm, I understand how WindowsMessageLoop works.. :) What I don't understand is how this event pops up from DefWindowProcW.
#3 topcrasher on Firefox 31.0a1 with 841/14787 crashes in the last 7 days.
Just got this on my home computer:

STR is just right click on the browser in the taskbar with several tabs open and it immediately crashes.
2 more:

refinement of STR: Replicated with just a single tab loaded to one of the above crash reports and then right clicking in the taskbar.
This is from what I can see the largest blocker for OMTC on Windows stability-wise. jimm, are you actively investigating and/or working on this? If not, who can we get on it?
Flags: needinfo?(jmathies)
Depends on: 874437
Also adding Milan & Nical to the loop. Could you help us here?
As Kairo said, this is critical.
Severity: critical → blocker
Flags: needinfo?(nical.bugzilla)
Flags: needinfo?(milan)
Please also see the discussion in bug 874437.
bug 874437 should fix 99.9% of this spew, although we'll still see them in plugin tests occasionally. Most of these can be fixed when they show up (like we did in bug 1047842) except system and "registered window message" events which are questionable to fix, since we don't know what those events do or carry with them.
Flags: needinfo?(nical.bugzilla)
Flags: needinfo?(milan)
Flags: needinfo?(jmathies)
Assignee: nobody → jmathies
(In reply to Jim Mathies [:jimm] from comment #18)
> omtc no longer uses the deferred message stuff, so this should be fixed.
> ni'ing myself to check crash stats in a week or so.

Given that OMTC is active on Aurora 33, we'll want whatever fixes this (if the fix can be verified) to be uplifted there or disable OMTC again.
Target Milestone: --- → mozilla34
Preliminary crash data is looking good. We have a number of crashes on the 17th, zero on the 18th, and crash stats has 18th crashes for other signatures processed.
Jim, can we get an uplift request for aurora? Thanks
(In reply to Sylvestre Ledru [:sylvestre] from comment #21)
> Jim, can we get an uplift request for aurora? Thanks

already in the works, see bug 874437.
Flags: needinfo?(jmathies)
So far we only have one crash with this signature on mc since the work in bug 874437. That particular crash is CPOW related and doesn't have anything to do with OMTC. I'll file a new e10s bug for that.
Flags: qe-verify+
Remaining crashes on mc are e10s related, no crashes on aurora currently after three days. We have individual bugs filed on e10s related work. billm also has a rework of ipc prioritization that should facilitate addressing these in our e10s work.

marking verified.
Blocks: 1049879
You need to log in before you can comment on or make changes to this bug.