937216 - [e10s] "ABORT: rpc calls cannot be issued within interrupts" when opening link from zimbra webmail

Status: RESOLVED FIXED

(Core :: IPC, defect)

Description

[Tim Abraldes [:TimAbraldes] [:tabraldes]]

When I click a link in the zimbra webmail client, plugin-container.exe crashes. I get the following stack trace.

mozalloc!mozalloc_abort(char * msg = 0x0059a964 "[Child 428] ###!!! ABORT: rpc calls cannot be issued within interrupts: file c:/src/mc2/ipc/glue/MessageChannel.cpp, line 1540")+0x32
xul!Abort(char * aMsg = 0x0059a964 "[Child 428] ###!!! ABORT: rpc calls cannot be issued within interrupts: file c:/src/mc2/ipc/glue/MessageChannel.cpp, line 1540")+0xd
xul!NS_DebugBreak(unsigned int aSeverity = 3, char * aStr = 0x6813cb90 "rpc calls cannot be issued within interrupts", char * aExpr = 0x00000000 "", char * aFile = 0x6813dbd0 "c:/src/mc2/ipc/glue/MessageChannel.cpp", int aLine = 0n1540)+0x2a3
xul!mozilla::ipc::MessageChannel::DebugAbort(char * file = 0x6813cbdc "c:/src/mc2/ipc/glue/MessageChannel.cpp", int line = 0n458, char * cond = 0x6813cbc0 "!AwaitingInterruptReply()", char * why = 0x6813cb90 "rpc calls cannot be issued within interrupts", bool reply = false)+0x1a0
xul!mozilla::ipc::MessageChannel::RPCCall(class IPC::Message * aMsg = 0x0f515bf8, class IPC::Message * aReply = 0x0059ae44)+0xbb
xul!mozilla::ipc::MessageChannel::Call(class IPC::Message * aMsg = 0x0f515bf8, class IPC::Message * aReply = 0x0059ae44)+0x47
xul!mozilla::dom::PContentChild::CallRpcMessage(class nsString * aMessage = 0x0059aec8, class mozilla::dom::ClonedMessageData * aData = 0x0059aef0, class nsTArray<mozilla::jsipc::CpowEntry> * aCpows = 0x0059af00, class IPC::Principal * aPrincipal = 0x0059aed4, class nsTArray<nsString> * retval = 0x0059af90)+0x136
xul!ChildProcessMessageManagerCallback::DoSendBlockingMessage(struct JSContext * aCx = 0x0f46a6a0, class nsAString_internal * aMessage = 0x020469c8, struct mozilla::dom::StructuredCloneData * aData = 0x0059af7c, class JS::Handle<JSObject *> aCpows = class JS::Handle<JSObject *>, class nsIPrincipal * aPrincipal = 0x00000000, class nsTArray<nsString> * aJSONRetVal = 0x0059af90, bool aIsSync = false)+0x130
xul!nsFrameMessageManager::SendMessage(class nsAString_internal * aMessageName = 0x020469c8, class JS::Value * aJSON = 0x0059b080, class JS::Value * aObjects = 0x0059b090, class nsIPrincipal * aPrincipal = 0x00000000, struct JSContext * aCx = 0x0f46a6a0, unsigned char aArgc = 0x03 '', class JS::Value * aRetval = 0x0059b0d0, bool aIsSync = false)+0x15b
xul!nsFrameMessageManager::SendRpcMessage(class nsAString_internal * aMessageName = 0x020469c8, class JS::Value * aJSON = 0x0059b080, class JS::Value * aObjects = 0x0059b090, class nsIPrincipal * aPrincipal = 0x00000000, struct JSContext * aCx = 0x0f46a6a0, unsigned char aArgc = 0x03 '', class JS::Value * aRetval = 0x0059b0d0)+0x2a
xul!NS_InvokeByIndex(class nsISupports * that = 0x04118da8, unsigned int methodIndex = 0xa, unsigned int paramCount = 7, struct nsXPTCVariant * params = 0x0059b070)+0x27
xul!CallMethodHelper::Invoke(void)+0x41
xul!CallMethodHelper::Call(void)+0xd5
xul!XPCWrappedNative::CallMethod(class XPCCallContext * ccx = 0x0059b148, XPCWrappedNative::CallMode mode = CALL_METHOD (0n0))+0x172
xul!XPC_WN_CallMethod(struct JSContext * cx = 0x0f46a6a0, unsigned int argc = 3, class JS::Value * vp = 0x0059b20c)+0x208
0x1c48e5ec
ntdll!RtlAllocateHeap+0x14c
mozjs!EnterBaseline(struct JSContext * cx = 0x0f46a6a0, struct js::jit::EnterJitData * data = 0x0059b3b0)+0x16b
mozjs!js::jit::EnterBaselineMethod(struct JSContext * cx = 0x0f46a6a0, class js::RunState * state = 0x0059b5f8)+0xa1
mozjs!js::RunScript(struct JSContext * cx = 0x0f46a6a0, class js::RunState * state = 0x0059b5f8)+0x116
mozjs!js::Invoke(struct JSContext * cx = 0x0f46a6a0, class JS::CallArgs args = class JS::CallArgs, js::MaybeConstruct construct = NO_CONSTRUCT (0n0))+0x217
mozjs!js::Invoke(struct JSContext * cx = 0x0f46a6a0, class JS::Value * thisv = 0x0059b714, class JS::Value * fval = 0x0059b74c, unsigned int argc = 7, class JS::Value * argv = 0x0059ba80, class JS::MutableHandle<JS::Value> rval = class JS::MutableHandle<JS::Value>)+0x163
mozjs!JS_CallFunctionValue(struct JSContext * cx = 0x0f46a6a0, class JSObject * objArg = 0x05f09f40, class JS::Value fval = class JS::Value, unsigned int argc = 7, class JS::Value * argv = 0x0059ba80, class JS::Value * rval = 0x0059ba48)+0xa1
xul!nsXPCWrappedJSClass::CallMethod(class nsXPCWrappedJS * wrapper = 0x0f4c7c68, unsigned short methodIndex = 3, struct XPTMethodDescriptor * info_ = 0x0366f508, struct nsXPTCMiniVariant * nativeParams = 0x0059bb4c)+0xf75
xul!nsXPCWrappedJS::CallMethod(unsigned short methodIndex = 3, struct XPTMethodDescriptor * info = 0x0366f508, struct nsXPTCMiniVariant * params = 0x0059bb4c)+0x141
xul!PrepareAndDispatch(class nsXPTCStubBase * self = 0x0f4c7cc0, unsigned int methodIndex = 3, unsigned int * args = 0x0059bc14, unsigned int * stackBytesToPop = 0x0059bc04)+0x2aa
xul!SharedStub(void)+0x16
xul!nsContentPolicy::CheckPolicy(<function> * policyMethod = 0x673273f0, unsigned int contentType = 6, class nsIURI * contentLocation = 0x0f4c7698, class nsIURI * requestingLocation = 0x0f4c7730, class nsISupports * requestingContext = 0x0f4ba200, class nsACString_internal * mimeType = 0x6852e8b4, class nsISupports * extra = 0x00000000, class nsIPrincipal * requestPrincipal = 0x0f4c7708, short * decision = 0x0059c268)+0x11f
xul!nsContentPolicy::ShouldLoad(unsigned int contentType = 6, class nsIURI * contentLocation = 0x0f4c7698, class nsIURI * requestingLocation = 0x0f4c7730, class nsISupports * requestingContext = 0x0f4ba200, class nsACString_internal * mimeType = 0x6852e8b4, class nsISupports * extra = 0x00000000, class nsIPrincipal * requestPrincipal = 0x0f4c7708, short * decision = 0x0059c268)+0x35
xul!NS_CheckContentLoadPolicy(unsigned int contentType = 6, class nsIURI * contentLocation = 0x0f4c7698, class nsIPrincipal * originPrincipal = 0x0f4c7708, class nsISupports * context = 0x0f4ba200, class nsACString_internal * mimeType = 0x6852e8b4, class nsISupports * extra = 0x00000000, short * decision = 0x0059c268, class nsIContentPolicy * policyService = 0x00000000, class nsIScriptSecurityManager * aSecMan = 0x00000000)+0x38b
xul!nsDocShell::InternalLoad(class nsIURI * aURI = 0x0f4c7698, class nsIURI * aReferrer = 0x00000000, class nsISupports * aOwner = 0x0f4c7708, unsigned int aFlags = 0, wchar_t * aWindowTarget = 0x0f4b9088 "", char * aTypeHint = 0x00000000 "", class nsAString_internal * aFileName = 0x6852e8c4, class nsIInputStream * aPostData = 0x00000000, class nsIInputStream * aHeadersData = 0x00000000, unsigned int aLoadType = 1, class nsISHEntry * aSHEntry = 0x00000000, bool aFirstParty = true, class nsAString_internal * aSrcdoc = 0x0059c420, class nsIDocShell ** aDocShell = 0x00000000, class nsIRequest ** aRequest = 0x00000000)+0x599
xul!nsDocShell::LoadURI(class nsIURI * aURI = 0x0f4c7698, class nsIDocShellLoadInfo * aLoadInfo = 0x0f4be8a0, unsigned int aLoadFlags = 0x40000, bool aFirstParty = true)+0xb94
xul!nsDocShell::LoadURI(wchar_t * aURI = 0x0059c638 "about:blank", unsigned int aLoadFlags = 0, class nsIURI * aReferringURI = 0x00000000, class nsIInputStream * aPostStream = 0x00000000, class nsIInputStream * aHeaderStream = 0x00000000)+0x40d
xul!nsWebBrowser::LoadURI(wchar_t * aURI = 0x0059c638 "about:blank", unsigned int aLoadFlags = 0x140000, class nsIURI * aReferringURI = 0x00000000, class nsIInputStream * aPostDataStream = 0x00000000, class nsIInputStream * aExtraHeaderStream = 0x00000000)+0x52
xul!mozilla::dom::TabChild::RecvLoadURL(class nsCString * uri = 0x0059cf14)+0x71
xul!mozilla::dom::PBrowserChild::OnMessageReceived(class IPC::Message * __msg = 0x0059ded4)+0x6ec
xul!mozilla::dom::PContentChild::OnMessageReceived(class IPC::Message * __msg = 0x0059ded4)+0x81
xul!mozilla::ipc::MessageChannel::DispatchAsyncMessage(class IPC::Message * aMsg = 0x0059ded4)+0x63
xul!mozilla::ipc::MessageChannel::DispatchMessageW(class IPC::Message * aMsg = 0x0059ded4)+0x89
xul!mozilla::ipc::MessageChannel::InterruptCall(class IPC::Message * aMsg = 0x0f4b9d18, class IPC::Message * aReply = 0x0059df64)+0x58b
xul!mozilla::ipc::MessageChannel::Call(class IPC::Message * aMsg = 0x0f4b9d18, class IPC::Message * aReply = 0x0059df64)+0x59
xul!mozilla::dom::PBrowserChild::CallCreateWindow(class mozilla::dom::PBrowserChild ** window = 0x0059dff8)+0xf2
xul!mozilla::dom::TabChild::ProvideWindow(class nsIDOMWindow * aParent = 0x07c39658, unsigned int aChromeFlags = 0xffe, bool aCalledFromJS = false, bool aPositionSpecified = false, bool aSizeSpecified = false, class nsIURI * aURI = 0x0f4b0400, class nsAString_internal * aName = 0x0059e380, class nsACString_internal * aFeatures = 0x0059e2d0, bool * aWindowIsNew = 0x0059e42f, class nsIDOMWindow ** aReturn = 0x0059e2b0)+0x99
xul!nsWindowWatcher::OpenWindowInternal(class nsIDOMWindow * aParent = 0x07c39658, char * aUrl = 0x0f4b2cc8 "https://bugzilla.mozilla.org/show_bug.cgi?id=935784", char * aName = 0x0059e5e0 "_blank", char * aFeatures = 0x00000000 "", bool aCalledFromJS = false, bool aDialog = false, bool aNavigate = false, class nsIArray * argv = 0x00000000, class nsIDOMWindow ** _retval = 0x0059e690)+0x7c7
xul!nsWindowWatcher::OpenWindow2(class nsIDOMWindow * aParent = 0x07c39658, char * aUrl = 0x0f4b2cc8 "https://bugzilla.mozilla.org/show_bug.cgi?id=935784", char * aName = 0x0059e5e0 "_blank", char * aFeatures = 0x00000000 "", bool aCalledFromScript = false, bool aDialog = false, bool aNavigate = false, class nsISupports * aArguments = 0x00000000, class nsIDOMWindow ** _retval = 0x0059e690)+0xa0
xul!nsGlobalWindow::OpenInternal(class nsAString_internal * aUrl = 0x0059ec10, class nsAString_internal * aName = 0x0059eae8, class nsAString_internal * aOptions = 0x6852e8a4, bool aDialog = false, bool aContentModal = false, bool aCalledNoScript = true, bool aDoJSFixups = false, bool aNavigate = false, class nsIArray * argv = 0x00000000, class nsISupports * aExtraArgument = 0x00000000, class nsIPrincipal * aCalleePrincipal = 0x0405bd70, struct JSContext * aJSCallerContext = 0x00000000, class nsIDOMWindow ** aReturn = 0x0059eaf8)+0x67e
xul!nsGlobalWindow::OpenNoNavigate(class nsAString_internal * aUrl = 0x0059ec10, class nsAString_internal * aName = 0x0059eae8, class nsAString_internal * aOptions = 0x6852e8a4, class nsIDOMWindow ** _retval = 0x0059eaf8)+0x44
xul!nsDocShell::InternalLoad(class nsIURI * aURI = 0x0f4b1a28, class nsIURI * aReferrer = 0x0d8f9670, class nsISupports * aOwner = 0x0405bd70, unsigned int aFlags = 0, wchar_t * aWindowTarget = 0x0059eeb0 "_blank", char * aTypeHint = 0x6852e8e4 "", class nsAString_internal * aFileName = 0x0f4add00, class nsIInputStream * aPostData = 0x00000000, class nsIInputStream * aHeadersData = 0x00000000, unsigned int aLoadType = 0x200001, class nsISHEntry * aSHEntry = 0x00000000, bool aFirstParty = true, class nsAString_internal * aSrcdoc = 0x6852e8c4, class nsIDocShell ** aDocShell = 0x00000000, class nsIRequest ** aRequest = 0x00000000)+0xb8c
xul!nsDocShell::OnLinkClickSync(class nsIContent * aContent = 0x0db81c38, class nsIURI * aURI = 0x0f4b1af0, wchar_t * aTargetSpec = 0x0f4ee7b8 "_blank", class nsAString_internal * aFileName = 0x0f4add00, class nsIInputStream * aPostDataStream = 0x00000000, class nsIInputStream * aHeadersDataStream = 0x00000000, class nsIDocShell ** aDocShell = 0x00000000, class nsIRequest ** aRequest = 0x00000000)+0x4ca
xul!OnLinkClickEvent::Run(void)+0xb7
xul!nsThread::ProcessNextEvent(bool mayWait = false, bool * result = 0x0059f163)+0x373
xul!NS_ProcessNextEvent(class nsIThread * thread = 0x020041b0, bool mayWait = false)+0x3d
xul!mozilla::ipc::MessagePump::Run(class base::MessagePump::Delegate * aDelegate = 0x0059f354)+0x62
xul!mozilla::ipc::MessagePumpForChildProcess::Run(class base::MessagePump::Delegate * aDelegate = 0x0059f354)+0x87
xul!MessageLoop::RunInternal(void)+0x33
xul!MessageLoop::RunHandler(void)+0x82
xul!MessageLoop::Run(void)+0x1d
xul!nsBaseAppShell::Run(void)+0x39
xul!nsAppShell::Run(void)+0x12
xul!XRE_RunAppShell(void)+0x63
xul!mozilla::ipc::MessagePumpForChildProcess::Run(class base::MessagePump::Delegate * aDelegate = 0x0059f354)+0x20
xul!MessageLoop::RunInternal(void)+0x33
xul!MessageLoop::RunHandler(void)+0x82
xul!MessageLoop::Run(void)+0x1d
xul!XRE_InitChildProcess(int aArgc = 0n4, char ** aArgv = 0x0213e700, GeckoProcessType aProcess = GeckoProcessType_Content (0n2))+0x6e5
plugin_container!NS_internal_main(int argc = 0n7, char ** argv = 0x0213e700)+0xcd
plugin_container!wmain(int argc = 0n8, wchar_t ** argv = 0x0213bd58)+0x119
plugin_container!__tmainCRTStartup(void)+0x122
KERNEL32!BaseThreadInitThunk+0xe
ntdll!__RtlUserThreadStart+0x20
ntdll!_RtlUserThreadStart+0x1b

Comment 1

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

David, it looks like we're hitting an IPC assertion.

Comment 2

[Tom S]

Could you do "call DumpJSStack()" in gdb? That would be super helpful. Thanks.

Comment 3

[Tim Abraldes [:TimAbraldes] [:tabraldes]]

As usual, I haven't been able to repro since getting instructions on how to provide useful debugging info. I'll keep running in the debugger to hopefully get that js call stack

Comment 4

[David Anderson [:dvander] - inactive, e-mail if emergency]

Interesting. Do you know if you had Flash or any sort of NPAPI plugin running at the time?

Comment 5

[Tim Abraldes [:TimAbraldes] [:tabraldes]]

(In reply to David Anderson [:dvander] from comment #4)
> Interesting. Do you know if you had Flash or any sort of NPAPI plugin
> running at the time?

Seems unlikely. I've just been using zimbra, gmail, bugzilla, and etherpad mostly

Comment 6

[Tim Abraldes [:TimAbraldes] [:tabraldes]]

So it turns out you can repro this behavior if you do the following:
 1. Disable AdBlockPlus
 2. Re-enable AdBlockPlus
 3. Click a link from an email in the Zimbra web client

This is the output  I got when I called DumpJSStack():
  0 anonymous(contentType = 6, contentLocation = [xpconnect wrapped nsIURI], requestOrigin = [xpconnect wrapped nsIURI], node = [object XrayWrapper [object Window]], mimeTypeGuess = "", extra = null, cpmm = [xpconnect wrapped (nsISupports, nsIPrincipal, nsISerializable)]) ["resource://gre/modules/RemoteAddonsChild.jsm":78].    this = [object Object]..

Comment 7

[David Anderson [:dvander] - inactive, e-mail if emergency]

Thanks, I can reproduce this on Windows. I can't seem to get it on Linux.

Comment 8

[David Anderson [:dvander] - inactive, e-mail if emergency]

Attachment: ipdl test case

What's happening: the child process asks the parent to open a window, which unfortunately is an interrupt message. On the parent process, this triggers some Adblock code which sends a CPOW. Back on the child side, we abort since we don't expect to see CPOWs on top of an interrupt message.

There isn't really a legitimate reason for not supporting this. I punted on it originally just to get the refactoring landed. It shouldn't be too hard to fix though.

Comment 9

[David Anderson [:dvander] - inactive, e-mail if emergency]

Attachment: rpc-test.patch

I audited the IPC code to see what was needed to remove this assert... and the answer looks like nothing. Even in the case of deferring RPC calls within an interrupt, it's already set up to handle nested event loops doing weird things to message queues. Just removing the assert, the test passes and I can't repro the bug using the STR anymore.

Comment 10

[David Anderson [:dvander] - inactive, e-mail if emergency]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c7e90ee57d85

Comment 11

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/c7e90ee57d85