Crash [@ mozilla::ipc::MessageChannel::ShouldDeferMessage]

RESOLVED WORKSFORME

Status

()

Core
IPC
--
critical
RESOLVED WORKSFORME
3 years ago
4 months ago

People

(Reporter: alex_mayorga, Unassigned)

Tracking

({crash, leave-open})

41 Branch
x86_64
Windows
crash, leave-open
Points:
---

Firefox Tracking Flags

(firefox41 affected, firefox42 ?, firefox43 ?, firefox44 affected, firefox48 affected, firefox49 affected, firefox50 affected, firefox51 affected, firefox52 wontfix)

Details

(Whiteboard: [bugday-20160912], crash signature)

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

3 years ago
This bug was filed from the Socorro interface and is 
report bp-ae8b99f0-bcee-4657-bb2a-a3f912150512.
=============================================================

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	mozilla::ipc::MessageChannel::ShouldDeferMessage(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
1 		@0xd000000000074fed

Shutting down the laptop without prior closing Nightly causes all tabs to crash.

Updated

2 years ago
Crash Signature: [@ mozilla::ipc::MessageChannel::ShouldDeferMessage(IPC::Message const&)] → [@ mozilla::ipc::MessageChannel::ShouldDeferMessage(IPC::Message const&)] [@ mozilla::ipc::MessageChannel::ShouldDeferMessage]
status-firefox42: --- → ?
status-firefox43: --- → ?
status-firefox44: --- → affected
Component: Untriaged → IPC
OS: Windows NT → Windows
Product: Firefox → Core
Hardware: Unspecified → x86_64
Summary: crash in mozilla::ipc::MessageChannel::ShouldDeferMessage(IPC::Message const&) → Crash [@ mozilla::ipc::MessageChannel::ShouldDeferMessage(IPC::Message const&) ]
Version: Trunk → 41 Branch
FF44.0a1, Win7, 64bit:

https://crash-stats.mozilla.com/report/index/32bdb5c3-2226-4fb4-8f86-2a8932151015

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	mozilla::ipc::MessageChannel::ShouldDeferMessage(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
1 	xul.dll 	mozilla::ipc::MessageChannel::ProcessPendingRequests() 	ipc/glue/MessageChannel.cpp
2 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*) 	ipc/glue/MessageChannel.cpp
3 	xul.dll 	mozilla::dom::PBrowserChild::SendRpcMessage(nsString const&, mozilla::dom::ClonedMessageData const&, nsTArray<mozilla::jsipc::CpowEntry> const&, IPC::Principal const&, nsTArray<mozilla::dom::ipc::StructuredCloneData>*) 	obj-firefox/ipc/ipdl/PBrowserChild.cpp
4 	xul.dll 	mozilla::dom::TabChild::DoSendBlockingMessage(JSContext*, nsAString_internal const&, mozilla::dom::ipc::StructuredCloneData&, JS::Handle<JSObject*>, nsIPrincipal*, nsTArray<mozilla::dom::ipc::StructuredCloneData>*, bool) 	dom/ipc/TabChild.cpp
5 	xul.dll 	nsFrameMessageManager::SendMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>, bool) 	dom/base/nsFrameMessageManager.cpp
6 	xul.dll 	nsFrameMessageManager::SendRpcMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>) 	dom/base/nsFrameMessageManager.cpp
7 	xul.dll 	mozilla::dom::TabChildGlobal::SendRpcMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>) 	dom/ipc/TabChild.h
8 	xul.dll 	XPTC__InvokebyIndex 	xpcom/reflect/xptcall/md/win32/xptcinvoke_asm_x86_64.asm
9 		@0x1d32007 	
10 	xul.dll 	XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode) 	js/xpconnect/src/XPCWrappedNative.cpp
11 	xul.dll 	XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) 	js/xpconnect/src/XPCWrappedNativeJSOps.cpp
12 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
13 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
14 	xul.dll 	js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/src/proxy/CrossCompartmentWrapper.cpp
15 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
16 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
17 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
18 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
19 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
20 	xul.dll 	js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/src/proxy/CrossCompartmentWrapper.cpp
21 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
22 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
23 	xul.dll 	JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) 	js/src/jsapi.cpp
24 	xul.dll 	nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS*, unsigned short, XPTMethodDescriptor const*, nsXPTCMiniVariant*) 	js/xpconnect/src/XPCWrappedJSClass.cpp
25 	xul.dll 	nsXPCWrappedJS::CallMethod(unsigned short, XPTMethodDescriptor const*, nsXPTCMiniVariant*) 	js/xpconnect/src/XPCWrappedJS.cpp
26 	xul.dll 	PrepareAndDispatch 	xpcom/reflect/xptcall/md/win32/xptcstubs_x86_64.cpp
27 	xul.dll 	SharedStub 	xpcom/reflect/xptcall/md/win32/xptcstubs_asm_x86_64.asm
28 	xul.dll 	mozilla::EventListenerManager::HandleEventInternal(nsPresContext*, mozilla::WidgetEvent*, nsIDOMEvent**, mozilla::dom::EventTarget*, nsEventStatus*) 	dom/events/EventListenerManager.cpp
29 	xul.dll 	mozilla::EventTargetChainItem::HandleEventTargetChain(nsTArray<mozilla::EventTargetChainItem>&, mozilla::EventChainPostVisitor&, mozilla::EventDispatchingCallback*, mozilla::ELMCreationDetector&) 	dom/events/EventDispatcher.cpp
30 	xul.dll 	mozilla::EventDispatcher::Dispatch(nsISupports*, nsPresContext*, mozilla::WidgetEvent*, nsIDOMEvent*, nsEventStatus*, mozilla::EventDispatchingCallback*, nsTArray<mozilla::dom::EventTarget*>*) 	dom/events/EventDispatcher.cpp
31 	xul.dll 	mozilla::EventDispatcher::DispatchDOMEvent(nsISupports*, mozilla::WidgetEvent*, nsIDOMEvent*, nsPresContext*, nsEventStatus*) 	dom/events/EventDispatcher.cpp
32 	xul.dll 	mozilla::DOMEventTargetHelper::DispatchEvent(nsIDOMEvent*, bool*) 	dom/events/DOMEventTargetHelper.cpp
33 	xul.dll 	mozilla::DOMEventTargetHelper::DispatchTrustedEvent(nsIDOMEvent*) 	dom/events/DOMEventTargetHelper.cpp
34 	xul.dll 	mozilla::DOMEventTargetHelper::DispatchTrustedEvent(nsAString_internal const&) 	dom/events/DOMEventTargetHelper.cpp
35 	xul.dll 	mozilla::dom::TabChild::RecvDestroy() 	dom/ipc/TabChild.cpp
36 	xul.dll 	mozilla::dom::PBrowserChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PBrowserChild.cpp
37 	xul.dll 	mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PContentChild.cpp
38 	xul.dll 	mozilla::ipc::MessageChannel::DispatchMessageW(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
39 	xul.dll 	mozilla::ipc::MessageChannel::OnMaybeDequeueOne() 	ipc/glue/MessageChannel.cpp
40 	xul.dll 	RunnableMethod<mozilla::ipc::MessageChannel, bool ( mozilla::ipc::MessageChannel::*)(void), Tuple0>::Run() 	ipc/chromium/src/base/task.h
41 	xul.dll 	MessageLoop::DoWork() 	ipc/chromium/src/base/message_loop.cc
42 	xul.dll 	mozilla::ipc::DoWorkRunnable::Run() 	ipc/glue/MessagePump.cpp
43 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
44 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
45 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
46 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
47 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
48 	xul.dll 	nsBaseAppShell::Run() 	widget/nsBaseAppShell.cpp
49 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp
50 	xul.dll 	XRE_RunAppShell 	toolkit/xre/nsEmbedFunctions.cpp
51 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
52 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
53 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
54 	xul.dll 	XRE_InitChildProcess 	toolkit/xre/nsEmbedFunctions.cpp
55 	plugin-container.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp
56 	plugin-container.exe 	__tmainCRTStartup 	f:/dd/vctools/crt/crtw32/startup/crt0.c:255
Ø 57 	kernel32.dll 	kernel32.dll@0x159dc 	
Ø 58 	ntdll.dll 	ntdll.dll@0x2a630
(Reporter)

Comment 2

a year ago
1041 reports in the past week for [@ mozilla::ipc::MessageChannel::ShouldDeferMessage]
Crash Signature: [@ mozilla::ipc::MessageChannel::ShouldDeferMessage(IPC::Message const&)] [@ mozilla::ipc::MessageChannel::ShouldDeferMessage] → [@ mozilla::ipc::MessageChannel::ShouldDeferMessage]
status-firefox48: --- → affected
status-firefox49: --- → affected
Summary: Crash [@ mozilla::ipc::MessageChannel::ShouldDeferMessage(IPC::Message const&) ] → Crash [@ mozilla::ipc::MessageChannel::ShouldDeferMessage]
Whiteboard: [bugday-20160912]
(97.86% in signature vs 00.12% overall) moz_crash_reason = MOZ_RELEASE_ASSERT(aMsg.priority() == IPC::Message::PRIORITY_NORMAL)

https://dxr.mozilla.org/mozilla-central/rev/17242152a8fbe65f20b864bd5a577b0dff624857/ipc/glue/MessageChannel.cpp#844

We're not deferring the message if the priority is PRIORITY_URGENT and then, if
the message is async, we're asserting that the priority must be PRIORITY_NORMAL.
Are we sure it can't be PRIORITY_HIGH?

If it can't, then probably one of the consumers is doing something wrong. Most
of the times 'mozilla::layers::PLayerTransactionChild::SendUpdate' is on the
stack, but it looks like this function is sending a message with a
PRIORITY_NORMAL priority. The crash might be caused by some other message
in the queue.
Created attachment 8792304 [details] [diff] [review]
Patch

I don't know anything about this code but, given the comments, this might be the right fix.
Flags: needinfo?(wmccloskey)
We're never supposed to have async high priority messages. The IPDL compiler should enforce that. So please don't change this assertion.

However, we could try asserting that we never send an async high priority message. That could happen somewhere here:
http://searchfox.org/mozilla-central/rev/f6c298b36db67a7109079c0dd7755f329c1d58e2/ipc/glue/MessageChannel.cpp#781

If this fails, we'll know exactly which message is the "bad" one.
Flags: needinfo?(wmccloskey)
I've tried to add the assertion, let's see if something blows up: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a6c2a3a6000d.
I can't see any crashes in the try run.
Created attachment 8792866 [details] [diff] [review]
Assert we don't send async HIGH priority messages
Attachment #8792304 - Attachment is obsolete: true
Attachment #8792866 - Attachment description: Assert we don't send sync HIGH priority messages → Assert we don't send async HIGH priority messages
Comment on attachment 8792866 [details] [diff] [review]
Assert we don't send async HIGH priority messages

It doesn't find the culprit in a try run, but might help getting better stack traces when we hit the crash.
Attachment #8792866 - Flags: review?(wmccloskey)
Comment on attachment 8792866 [details] [diff] [review]
Assert we don't send async HIGH priority messages

Review of attachment 8792866 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks Marco!

::: ipc/glue/MessageChannel.cpp
@@ +769,5 @@
>                                nsDependentCString(aMsg->name()), aMsg->size());
>      }
>  
> +    // We never send an async high priority message.
> +    MOZ_RELEASE_ASSERT(aMsg->is_sync() || aMsg->priority() != IPC::Message::PRIORITY_HIGH);

This variant of MessageChannel::Send is only ever used for async messages, so the first part isn't really useful. You could take it out or change this to two separate assertions:

MOZ_RELEASE_ASSERT(!aMsg->is_sync());
MOZ_RELEASE_ASSERT(aMsg->priority() != IPC::Message::PRIORITY_HIGH);
Attachment #8792866 - Flags: review?(wmccloskey) → review+
As expected, still green after the suggested change: https://treeherder.mozilla.org/#/jobs?repo=try&revision=11c05fd12a04.

Comment 13

a year ago
Pushed by mcastelluccio@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/bd880243ff4c
Assert that we never send an async high priority message. r=billm
status-firefox50: --- → affected
status-firefox51: --- → affected
Keywords: leave-open
We're mainly seeing crash reports with release builds (only 1 with
51 nightly in the last week and a few with different betas).

Alex, are you still able to reproduce this crash? If you are, could
you try the same steps to reproduce in the next Nightly build that
contains this change? The reports should be easier to analyze now.
Flags: needinfo?(alex_mayorga)
Note to future self: the assertion was changed in https://hg.mozilla.org/mozilla-central/rev/fb124a19fb68#l12.327 to MOZ_RELEASE_ASSERT(aMsg->nested_level() != IPC::Message::NESTED_INSIDE_SYNC).
(Reporter)

Comment 17

a year ago
Ciao Marco!

I no longer ge this crash on Nightly.

The most recent version with a crash is 52.0a2 per https://crash-stats.mozilla.com/signature/?signature=mozilla%3A%3Aipc%3A%3AMessageChannel%3A%3AShouldDeferMessage

Shall I mark this RESOLVED?

Grazie!
Alex
status-firefox52: --- → affected
Flags: needinfo?(alex_mayorga) → needinfo?(mcastelluccio)
With the patch, the signature might have changed. Let's wait a while before closing.
Flags: needinfo?(mcastelluccio)
Too late for firefox 52, mass-wontfix.
status-firefox52: affected → wontfix

Comment 20

4 months ago
Marco, can we close this out?
Flags: needinfo?(mcastelluccio)
It looks like the assertion was hit a few times (but extremely rarely, only 7 times in the past 6 months, all in the same day and from the same user).
I guess we can close it.
Status: NEW → RESOLVED
Crash Signature: [@ mozilla::ipc::MessageChannel::ShouldDeferMessage] → [@ mozilla::ipc::MessageChannel::ShouldDeferMessage] [@ mozilla::ipc::MessageChannel::Send | mozilla::ipc::PBackgroundChild::SendPServiceWorkerManagerConstructor] [@ mozilla::ipc::MessageChannel::Send | mozilla::layers::PCompositorBridgeChild::Se&hellip;
Last Resolved: 4 months ago
Flags: needinfo?(mcastelluccio)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.