1075691 - Content process crashes with MsgProcessingError when TabParent::Show sends Show message with browser.tabs.remote.autostart set to false

Status: RESOLVED FIXED

(Core :: DOM: Content Processes, defect)

Description

[Mike Conley (:mconley) (:⚙️)]

STR:

1) Start up Nightly in a new profile, and make sure browser.tabs.remote.autostart is set to false (this is the default, so this should be the case)
2) Go to File > New e10s window

ER:

A new browser window should open with a working remote tab.

AR:

A new browser window opens, but the initial tab in it immediately hits the crashed state.

I attached lldb, and got the following stack:


* thread #1: tid = 0x887f07, 0x000000010006b31d libmozalloc.dylib`mozalloc_abort(msg=0x00007fff5fbfd418) + 93 at mozalloc_abort.cpp:37, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x000000010006b31d libmozalloc.dylib`mozalloc_abort(msg=0x00007fff5fbfd418) + 93 at mozalloc_abort.cpp:37
    frame #1: 0x000000010066cc25 XUL`Abort(aMsg=0x00007fff5fbfd418) + 21 at nsDebugImpl.cpp:469
    frame #2: 0x000000010066c680 XUL`NS_DebugBreak(aSeverity=3, aStr=0x000000010710dc1e, aExpr=0x0000000000000000, aFile=0x000000010710d615, aLine=1634) + 1232 at nsDebugImpl.cpp:426
    frame #3: 0x0000000102e7c248 XUL`mozilla::dom::ContentChild::ProcessingError(this=0x000000011122a830, what=MsgProcessingError) + 232 at ContentChild.cpp:1634
    frame #4: 0x0000000100fa6452 XUL`mozilla::dom::PContentChild::OnProcessingError(this=0x000000011122a830, aCode=MsgProcessingError) + 34 at PContentChild.cpp:5607
    frame #5: 0x0000000100dbb5f5 XUL`mozilla::ipc::MessageChannel::MaybeHandleError(this=0x000000011122a890, code=MsgProcessingError, aMsg=0x00007fff5fbfdbf0, channelName=0x0000000106f7c13d) + 421 at MessageChannel.cpp:1599
    frame #6: 0x0000000100dbb442 XUL`mozilla::ipc::MessageChannel::DispatchAsyncMessage(this=0x000000011122a890, aMsg=0x00007fff5fbfdbf0) + 290 at MessageChannel.cpp:1233
    frame #7: 0x0000000100dba6ef XUL`mozilla::ipc::MessageChannel::DispatchMessage(this=0x000000011122a890, aMsg=0x00007fff5fbfdbf0) + 207 at MessageChannel.cpp:1115
    frame #8: 0x0000000100db6799 XUL`mozilla::ipc::MessageChannel::OnMaybeDequeueOne(this=0x000000011122a890) + 505 at MessageChannel.cpp:1098
    frame #9: 0x0000000100dd3463 XUL`void DispatchToMethod<mozilla::ipc::MessageChannel, bool (obj=0x000000011122a890, method=a0 65 db 00 01 00 00 00, arg=0x000000011121bb70)()>(mozilla::ipc::MessageChannel*, bool (mozilla::ipc::MessageChannel::*)(), Tuple0 const&) + 131 at tuple.h:383
    frame #10: 0x0000000100dd335e XUL`RunnableMethod<mozilla::ipc::MessageChannel, bool (this=0x000000011121bb40)(), Tuple0>::Run() + 78 at task.h:307
    frame #11: 0x0000000100dd5ee8 XUL`mozilla::ipc::MessageChannel::RefCountedTask::Run(this=0x0000000111226190) + 40 at MessageChannel.h:411
    frame #12: 0x0000000100dd5eb4 XUL`mozilla::ipc::MessageChannel::DequeueTask::Run(this=0x0000000111206d20) + 36 at MessageChannel.h:428
    frame #13: 0x0000000100d529c0 XUL`MessageLoop::RunTask(this=0x00007fff5fbfe180, task=0x0000000111206d20) + 96 at message_loop.cc:358
    frame #14: 0x0000000100d52f3f XUL`MessageLoop::DeferOrRunPendingTask(this=0x00007fff5fbfe180, pending_task=0x00007fff5fbfddc8) + 79 at message_loop.cc:366
    frame #15: 0x0000000100d53174 XUL`MessageLoop::DoWork(this=0x00007fff5fbfe180) + 292 at message_loop.cc:444
    frame #16: 0x0000000100dbe7e3 XUL`mozilla::ipc::MessagePumpForChildProcess::Run(this=0x00000001112211f0, aDelegate=0x00007fff5fbfe180) + 627 at MessagePump.cpp:297
    frame #17: 0x0000000100d528a5 XUL`MessageLoop::RunInternal(this=0x00007fff5fbfe180) + 117 at message_loop.cc:230
    frame #18: 0x0000000100d527b5 XUL`MessageLoop::RunHandler(this=0x00007fff5fbfe180) + 21 at message_loop.cc:223
    frame #19: 0x0000000100d5275d XUL`MessageLoop::Run(this=0x00007fff5fbfe180) + 45 at message_loop.cc:197
    frame #20: 0x00000001047c6a52 XUL`XRE_RunAppShell + 306 at nsEmbedFunctions.cpp:708
    frame #21: 0x0000000100dbe634 XUL`mozilla::ipc::MessagePumpForChildProcess::Run(this=0x00000001112211f0, aDelegate=0x00007fff5fbfe180) + 196 at MessagePump.cpp:272
    frame #22: 0x0000000100d528a5 XUL`MessageLoop::RunInternal(this=0x00007fff5fbfe180) + 117 at message_loop.cc:230
    frame #23: 0x0000000100d527b5 XUL`MessageLoop::RunHandler(this=0x00007fff5fbfe180) + 21 at message_loop.cc:223
    frame #24: 0x0000000100d5275d XUL`MessageLoop::Run(this=0x00007fff5fbfe180) + 45 at message_loop.cc:197
    frame #25: 0x00000001047c62b6 XUL`XRE_InitChildProcess(aArgc=3, aArgv=0x00007fff5fbff730) + 2678 at nsEmbedFunctions.cpp:550
    frame #26: 0x0000000100000d19 plugin-container`content_process_main(argc=6, argv=0x00007fff5fbff730) + 185 at plugin-container.cpp:158
    frame #27: 0x0000000100000de2 plugin-container`main(argc=7, argv=0x00007fff5fbff730) + 34 at MozillaRuntimeMain.cpp:11

Comment 1

[Mike Conley (:mconley) (:⚙️)]

It looks like the message we're attempting to process at the time of the error is one with name "PBrowser::Msg_Show".

Comment 2

[Mike Conley (:mconley) (:⚙️)]

Digging in, it looks like we're failing out in InitTabChildGlobal - specifically, when calling InitTabChildGlobalInternal, at this point:

http://hg.mozilla.org/mozilla-central/file/14665b1de5ee/content/base/src/nsFrameMessageManager.cpp#l1598

Comment 3

[Mike Conley (:mconley) (:⚙️)]

Here's what's failing:

http://hg.mozilla.org/mozilla-central/file/14665b1de5ee/js/xpconnect/src/XPCWrappedNative.cpp#l191

XPCWrappedNativeProto *proto =
  XPCWrappedNativeProto::GetNewOrUsed(scope,
                                      nativeHelper.GetClassInfo(), &sciProto,
                                      /* callPostCreatePrototype = */ false);
if (!proto)
   return NS_ERROR_FAILURE;  // <-- returning this

Comment 4

[Mike Conley (:mconley) (:⚙️)]

So I'm really down in the guts of XPConnect here, and I really don't know what's happening, but this is what I see:

We're entering XPCNativeSet::GetNewOrUsed, passing some nsIClassInfo about ContentFrameMessageManager.

We reach this point of XPCNativeSet::GetNewOrUsed, which appears to be some kind of fallback... we "give up", and get the set for nsISupports:

http://hg.mozilla.org/mozilla-central/file/14665b1de5ee/js/xpconnect/src/XPCWrappedNativeInfo.cpp#l519

Which causes us to re-enter XPCNativeSet::GetNewOrUsed.  I think what happens then is we hit this code:

    ClassInfo2NativeSetMap* map = rt->GetClassInfo2NativeSetMap();
    if (!map)
        return nullptr;

And we don't find a map, so we return nullptr.

Comment 5

[Mike Conley (:mconley) (:⚙️)]

Felipe says he's starting a bisect.

Comment 6

[Tim Taubert [:ttaubert] (inactive)]

This breaks background thumbnailing :/

Comment 7

[Tim Taubert [:ttaubert] (inactive)]

FTR, seems to affect custom builds only. Bad enough for development but nothing that breaks official builds.

Comment 8

[:Felipe Gomes (needinfo for replies!)]

This bisecting has been strange, but I think I got it down to:

$ hg bisect --bad
Due to skipped revisions, the first bad revision could be any of:
changeset:   207867:3ff20d4c7f39
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:50:52 2014 -0700
summary:     Mac v2 signing - Bug 1047584 - Enable DMG v2 signing. r=bhearsum

changeset:   207868:fbc1ceea4d6f
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:50:56 2014 -0700
summary:     Mac v2 signing - Bug 1047584 - Modify the .app file structure to allow for OSX v2 signing. r=bsmedberg

changeset:   207869:f877c9b66cec
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:00 2014 -0700
summary:     Mac v2 signing - Bug 1048687 - Read dependentlibs.list from Contents/Resources due to the new .app bundle structure. r=bsmedberg

changeset:   207870:11d48c719ec9
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:04 2014 -0700
summary:     Mac v2 signing - Bug 1050944 - Get Firefox to launch and run on OSX with the new .app bundle structure, made necessary by Apple's v2 signatures. r=smichaud, r=ted, sr=bsmedberg

changeset:   207871:0ce9e74b0abe
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:08 2014 -0700
summary:     Mac v2 signing - Bug 1046924 - Move updates directory out of the .app bundle. r=rstrong

changeset:   207872:b503560736fa
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:13 2014 -0700
summary:     Mac v2 signing - Bug 1055774 - Update lookup for crashreporter.ini for the new v2 bundle structure on OSX. r=bsmedberg

changeset:   207873:cbd3f8e4bf49
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:17 2014 -0700
summary:     Mac v2 signing - Bug 1059504 - Avoid plugin-container from crashing due to the new v2 bundle structure on OSX. r=bsmedberg

changeset:   207874:24e1e95d7af2
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:21 2014 -0700
summary:     Mac v2 signing - Bug 1053838 - Fix GTests to run with the new v2 bundle structure. r=bsmedberg

changeset:   207875:427a562e16d4
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:25 2014 -0700
summary:     Mac v2 signing - Bug 1060652 - Make mochitest harness work with the new bundle structure for v2 signatures on OSX. r=jmaher

changeset:   207876:33000c22f91f
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:29 2014 -0700
summary:     Mac v2 signing - Bug 1060562 - Update xpcshell-tests for the new v2 bundle structure on OSX. r=jmaher

changeset:   207877:5af85a6db219
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:39 2014 -0700
summary:     Mac v2 signing - Bug 1064952 - Update Cpp unit test harness for the new v2 bundle structure on OSX. r=jmaher

changeset:   207878:09e4c6245f6d
user:        Stephen Pohl <[e-mail]>
date:        Mon Sep 29 11:51:43 2014 -0700
summary:     Mac v2 signing - Bug 1064910 - Update Webapp Runtime to work with the new v2 bundle structure of Firefox on OSX. r=myk

Comment 9

[:Felipe Gomes (needinfo for replies!)]

To double check, we should see if 20c7e70e1b1aworks and c5ba012a6944 presents the bug

Comment 10

[:Felipe Gomes (needinfo for replies!)]

(In reply to :Felipe Gomes from comment #9)
> To double check, we should see if 20c7e70e1b1a works and c5ba012a6944
> presents the bug

mccr8 confirmed this, so the bug started happening with these csets. I'm not sure if it's worth bisecting inside them because they are all part of the same work, but if someone thinks it will be useful, I can do it.  I had tried but some csets didn't compile or presented other problems due to being incomplete, so I then skipped them during bisect.

> c5ba012a6944	Robert Strong — Mac v2 signing - Bug 1074044 - Force add instead of patch the removed-files file. r=bhearsum
> c5bb9c221c3e	Robert Strong — Mac v2 signing - Bug 1074000 - Update xpcshell-tests for new OSX v2 bundle structure to run on comm-central. r=jmaher
> be53b555dda6	Robert Strong — Mac v2 signing - Bug 1072722 - With older clients the new maintenance service checks the updated directory's updater.exe when verifying the updater.exe for replace requests. r=spohl
> 28ae112eba8d	Robert Strong — Mac v2 signing - Bug 1072716 - Update B2G removed-files.in for mac v2 signing. r=mwu
> 4f0a10f05200	Robert Strong — Mac v2 signing - Bug 1072554 - Remove update directories when running xpcshell tests. r=spohl
> 4932dbbc1cc8	Robert Strong — Mac v2 signing - Bug 1071465 - old precomplete file is not removed on update. r=spohl
> 818b73513ce2	Robert Strong — Mac v2 signing - Bug 1071134 - Fix command line help for updater. r=spohl
> 2cc605ebaad8	Robert Strong — Mac v2 signing - Bug 1070863 - The preprocessed channel-prefs.js file needs to be the same for each build. r=ted
> 5d6da7c4a822	Robert Strong — Mac v2 signing - Bug 1070149 - Add explicit file removals for channel-prefs.js, update-settings.ini, and precomplete files. r=spohl
> 972e4d9da1b5	Robert Strong — Mac v2 signing - Temporary workaround for Bug 1070148 - Copy maintenance service binary into its install directory when the installed binary is different. r=bbondy
> 1599a302abfb	Robert Strong — Mac v2 signing - Bug 1070661 - Error setting access/modification time on application bundle. r=spohl
> 67261da9cb7d	Robert Strong — Mac v2 signing - Bug 1068439 - Move the distribution directory from Content/MacOS to Contents/Resources on app update due to v2 signing requirements. r=spohl
> b1d8f769952a	Robert Strong — Mac v2 signing - Bug 1058182 - Fix app update xpcshell tests due to the Mac v2 bundle structure. r=bbondy
> 71611abb9569	Robert Strong — Mac v2 signing - Bug 1064523 - Create staging directory outside of the Mac bundle. r=bbondy
> 432ef1ab72b0	Robert Strong — Mac v2 signing - Bug 1059567 - Packaging changes for the move of removed-files file from Contents/MacOS to Contents/Resources. r=bbondy, r=nthomas
> d3025e55dfc3	Robert Strong — Mac v2 signing - Bug 1059467 - Move precomplete file from the root of the Mac bundle to Contents/Resources. r=bbondy, r=nthomas
> c6e905b1e209	Steven Michaud — Mac v2 signing - Bug 1047738 - Make distribution code look for the distribution directory under Contents/Resources due to v2 signing requirements. r=bsmedberg
> 47c2f60b1174	Stephen Pohl — Mac v2 signing - Bug 1066123 - Ensure b2g desktop OSX builds still work after the v2 signature changes to Firefox.app bundles. r=ted
> 09e4c6245f6d	Stephen Pohl — Mac v2 signing - Bug 1064910 - Update Webapp Runtime to work with the new v2 bundle structure of Firefox on OSX. r=myk
> 5af85a6db219	Stephen Pohl — Mac v2 signing - Bug 1064952 - Update Cpp unit test harness for the new v2 bundle structure on OSX. r=jmaher
> 33000c22f91f	Stephen Pohl — Mac v2 signing - Bug 1060562 - Update xpcshell-tests for the new v2 bundle structure on OSX. r=jmaher
> 427a562e16d4	Stephen Pohl — Mac v2 signing - Bug 1060652 - Make mochitest harness work with the new bundle structure for v2 signatures on OSX. r=jmaher
> 24e1e95d7af2	Stephen Pohl — Mac v2 signing - Bug 1053838 - Fix GTests to run with the new v2 bundle structure. r=bsmedberg
> cbd3f8e4bf49	Stephen Pohl — Mac v2 signing - Bug 1059504 - Avoid plugin-container from crashing due to the new v2 bundle structure on OSX. r=bsmedberg
> b503560736fa	Stephen Pohl — Mac v2 signing - Bug 1055774 - Update lookup for crashreporter.ini for the new v2 bundle structure on OSX. r=bsmedberg
> 0ce9e74b0abe	Stephen Pohl — Mac v2 signing - Bug 1046924 - Move updates directory out of the .app bundle. r=rstrong
> 11d48c719ec9	Stephen Pohl — Mac v2 signing - Bug 1050944 - Get Firefox to launch and run on OSX with the new .app bundle structure, made necessary by Apple's v2 signatures. r=smichaud, r=ted, sr=bsmedberg
> f877c9b66cec	Stephen Pohl — Mac v2 signing - Bug 1048687 - Read dependentlibs.list from Contents/Resources due to the new .app bundle structure. r=bsmedberg
> fbc1ceea4d6f	Stephen Pohl — Mac v2 signing - Bug 1047584 - Modify the .app file structure to allow for OSX v2 signing. r=bsmedberg
> 3ff20d4c7f39	Stephen Pohl — Mac v2 signing - Bug 1047584 - Enable DMG v2 signing. r=bhearsum

Comment 11

[Stephen A Pohl [:spohl]]

Looking into it.

Comment 12

[Stephen A Pohl [:spohl]]

I'd appreciate it if someone could confirm what I'm seeing, namely that locally packaged builds don't exhibit this issue (created via 'make package' in the obj-dir, then extracted from the .dmg). It looks to me like only builds that were built via ./mach build run into this issue. Just want to make sure that I'm not missing anything.

Comment 13

[:Felipe Gomes (needinfo for replies!)]

I won't have a chance to confirm this still today, but I believe it's highly likely that that's right, because that would explain well why we run into this bug when building locally, but not on the regular Nightly builds

Comment 14

[Stephen A Pohl [:spohl]]

Okay, once the chrome.manifest file and the components directory is copied from Contents/Resources to Contents/MacOS we don't crash anymore. The packaged build doesn't crash because the chrome.manifest and the components directory appear to be compiled into the omni.ja file and we seem to be reading this in successfully from Contents/Resources. I'll figure out where we read in the chrome.manifest and components directory tomorrow. It should now look in Contents/Resources instead of Contents/MacOS.

Comment 15

[Stephen A Pohl [:spohl]]

Found the issue. The GreD for the plugin-container is set incorrectly. Working on a fix.

Comment 16

[Stephen A Pohl [:spohl]]

Attachment: Patch

Comment 17

[Stephen A Pohl [:spohl]]

Attachment: Patch

Forgot to add curly brackets after the |if (NS_FAILED(rv))|.

Comment 18

[Benjamin Smedberg]

r=me in either case

Comment 19

[Mike Conley (:mconley) (:⚙️)]

With this patch applied, I no longer see the crash. Thanks spohl!!

Comment 20

[Stephen A Pohl [:spohl]]

Thanks, bsmedberg and mconley! Landed on inbound:

https://hg.mozilla.org/integration/mozilla-inbound/rev/cabd17a5837a

Comment 21

[Stephen A Pohl [:spohl]]

Also landed on oak:
https://hg.mozilla.org/projects/oak/rev/f1f1bdaccadf

Comment 22

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/cabd17a5837a

Comment 23

[Robert Strong (they/them - no direct email)]

Landed on aurora in the Mac V2 signing combined patch in bug 1047584