Closed Bug 1116884 (killhard-win) Opened 9 years ago Closed 6 years ago

[meta] e10s KillHard aborts on Windows

Categories

(Core :: DOM: Content Processes, defect, P2)

All
Windows
defect

Tracking

()

RESOLVED WONTFIX
Tracking Status
e10s - ---
firefox40 - affected
firefox41 --- affected
firefox42 --- affected
firefox43 --- affected
firefox44 --- affected
firefox47 --- affected

People

(Reporter: marcia, Unassigned)

References

(Depends on 3 open bugs, Blocks 1 open bug)

Details

(4 keywords, Whiteboard: [KillHard Windows signature][do not file ShutdownKill signatures here])

Crash Data

This bug was filed from the Socorro interface and is 
report bp-e7c7f088-fdaf-4e68-b248-10dd32141231.
=============================================================

Seen in crash stats, but stephend also crashed in this stack today. Reports in the last 7 days: https://crash-stats.mozilla.com/report/list?productFirefox&signature=WaitForSingleObjectEx%20|%20WaitForSingleObject%20|%20PR_WaitCondVar%20|%20mozilla::CondVar::Wait%28unsigned%20int%29

Many of the URLS seem to be either about:blank or https://get3.adobe.com/flashplayer/ links

Frame 	Module 	Signature 	Source
0 	ntdll.dll 	ZwWaitForSingleObject 	
1 	kernelbase.dll 	WaitForSingleObjectEx 	
2 	nss3.dll 	PR_WaitCondVar 	nsprpub/pr/src/threads/combined/prucv.c
3 		@0x137 	
4 	xul.dll 	mozilla::CondVar::Wait(unsigned int) 	xpcom/glue/CondVar.h
5 	xul.dll 	mozilla::ipc::MessageChannel::WaitForSyncNotify() 	ipc/glue/WindowsMessageLoop.cpp
6 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*) 	ipc/glue/MessageChannel.cpp
7 	xul.dll 	mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned __int64*) 	obj-firefox/ipc/ipdl/PPluginWidgetChild.cpp
8 	xul.dll 	mozilla::widget::PluginWidgetProxy::GetNativeData(unsigned int) 	widget/PluginWidgetProxy.cpp
9 	xul.dll 	nsPluginInstanceOwner::GetPluginPort() 	dom/plugins/base/nsPluginInstanceOwner.cpp
10 	xul.dll 	nsPluginInstanceOwner::CreateWidget() 	dom/plugins/base/nsPluginInstanceOwner.cpp
11 	xul.dll 	nsPluginHost::CreateWidget(nsPluginInstanceOwner*) 	dom/plugins/base/nsPluginHost.cpp
12 	xul.dll 	nsPluginHost::InstantiatePluginInstance(char const*, nsIURI*, nsObjectLoadingContent*, nsPluginInstanceOwner**) 	dom/plugins/base/nsPluginHost.cpp
13 	xul.dll 	nsObjectLoadingContent::InstantiatePluginInstance(bool) 	dom/base/nsObjectLoadingContent.cpp
14 	xul.dll 	nsObjectLoadingContent::LoadObject(bool, bool, nsIRequest*) 	dom/base/nsObjectLoadingContent.cpp
15 	xul.dll 	nsObjectLoadingContent::OnStartRequest(nsIRequest*, nsISupports*) 	dom/base/nsObjectLoadingContent.cpp
16 	xul.dll 	mozilla::net::HttpChannelChild::DoOnStartRequest(nsIRequest*, nsISupports*) 	netwerk/protocol/http/HttpChannelChild.cpp
17 	xul.dll 	mozilla::net::HttpChannelChild::OnStartRequest(tag_nsresult const&, mozilla::net::nsHttpResponseHead const&, bool const&, mozilla::net::nsHttpHeaderArray const&, bool const&, bool const&, unsigned int const&, nsCString const&, nsCString const&, mozilla::net::NetAddr const&, mozilla::net::NetAddr const&) 	netwerk/protocol/http/HttpChannelChild.cpp
18 	xul.dll 	mozilla::net::HttpChannelChild::RecvOnStartRequest(tag_nsresult const&, mozilla::net::nsHttpResponseHead const&, bool const&, mozilla::net::nsHttpHeaderArray const&, bool const&, bool const&, unsigned int const&, nsCString const&, nsCString const&, mozilla::net::NetAddr const&, mozilla::net::NetAddr const&, short const&) 	netwerk/protocol/http/HttpChannelChild.cpp
19 	xul.dll 	mozilla::net::PHttpChannelChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PHttpChannelChild.cpp
20 	xul.dll 	mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PContentChild.cpp
21 	xul.dll 	mozilla::ipc::MessageChannel::OnMaybeDequeueOne() 	ipc/glue/MessageChannel.cpp
22 	xul.dll 	RunnableMethod<mozilla::ipc::MessageChannel, void ( mozilla::ipc::MessageChannel::*)(void), Tuple0>::Run() 	ipc/chromium/src/base/task.h
23 	xul.dll 	MessageLoop::DoWork() 	ipc/chromium/src/base/message_loop.cc
24 	xul.dll 	mozilla::ipc::DoWorkRunnable::Run() 	ipc/glue/MessagePump.cpp
25 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
26 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/glue/nsThreadUtils.cpp
27 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
28 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
29 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
30 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
31 	xul.dll 	nsBaseAppShell::Run() 	widget/nsBaseAppShell.cpp
32 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp
33 	xul.dll 	XRE_RunAppShell 	toolkit/xre/nsEmbedFunctions.cpp
34 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
35 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
36 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
37 	xul.dll 	XRE_InitChildProcess 	toolkit/xre/nsEmbedFunctions.cpp
38 	plugin-container.exe 	content_process_main(int, char** const) 	ipc/contentproc/plugin-container.cpp
39 	plugin-container.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp
40 	plugin-container.exe 	__tmainCRTStartup 	f:/dd/vctools/crt/crtw32/startup/crt0.c:255
41 	kernel32.dll 	BaseThreadInitThunk 	
42 	ntdll.dll 	RtlUserThreadStart
There are two common signatures here - 

mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const&)

and

mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned __int64*)

from within nsObjectLoadingContent::InstantiatePluginInstance.

These two represent #1 and #2 top content crashers in Nightly. Definitely e10s related.

https://crash-stats.mozilla.com/report/list?product=Firefox&range_value=7&range_unit=days&date=2015-01-05&signature=WaitForSingleObjectEx+|+WaitForSingleObject+|+PR_WaitCondVar+|+mozilla%3A%3ACondVar%3A%3AWait%28unsigned+int%29&version=Firefox%3A37.0a1#tab-sigsummary

https://crash-stats.mozilla.com/report/list?product=Firefox&range_value=7&range_unit=days&date=2015-01-05&signature=WaitForSingleObjectEx+|+PR_WaitCondVar+|+mozilla%3A%3ACondVar%3A%3AWait%28unsigned+int%29&version=Firefox%3A37.0a1#tab-sigsummary
Crash Signature: [@ WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] → [@ WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [@ WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)]
tracking-e10s: --- → ?
Keywords: topcrash
Assignee: nobody → jmathies
I'd bet bug 1118517 is this bug on OS X.
See Also: → 1118517
Like bug 1118517, these also appear to start with the 2014-12-23 mozilla-central nightly.
(In reply to Steven Michaud from comment #3)
> Like bug 1118517, these also appear to start with the 2014-12-23
> mozilla-central nightly.

Your regression range doesn't have much. Maybe bug 1068349.

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=b915a50bc6be&tochange=0532f2509f3f
[Tracking Requested - why for this release]:
This has been the top crash in the last days for the last 37 Nightly builds and also is significant on 38 - but it's on content processes only, so not affecting Developer Edition.
Report ID 	Date Submitted
bp-13c8f74c-e1c6-469b-9a5c-dbfd42150120	20/01/2015	12:46 p.m.

Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:38.0) Gecko/20100101 Firefox/38.0 ID:20150120030203 CSet: 93f526d85b47
My Release-channel Firefox 35.0 just crashed several times on startup with this signature. I was using a a brand-new profile.

The first crash was while sitting on about:home. Then when I tried to restart, I kept getting startup crashes until I launch Firefox in safe-mode!
It looks like the startup crashes are related to my gfx driver:

https://crash-stats.mozilla.com/report/index/ed2770f4-7812-4c2c-bed4-77b152150120
https://crash-stats.mozilla.com/report/index/796e4410-2c99-4357-953d-c45fe2150120

I restarted Windows and the startup crashes disappeared. It's possible the first crash was caused by driver issues as well
A common entry point for this crash is ScriptRequestPluginInstance. Also, this blew up around the 18th -

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=369a8f14ccf8&tochange=0c454540fc2b

I'd bet the plugin hang dialog patches did that. (bug 1118618)

I'll try to find a good test case.
Hmm, actually crashes seem to be an even mixture of that script call and nsAsyncInstantiateEvent. So I guess the script call isn't unique to this crash.

http://mxr.mozilla.org/mozilla-central/source/dom/base/nsObjectLoadingContent.cpp#129
David, any insight here? I'm having a hard time understanding what actually causes this crash. Best guess is that MessageChannel's mMonitor gets torn down while we're sitting in MessageChannel::WaitForSyncNotify().
Flags: needinfo?(dmajor)
I guess it could be anywhere in here as well - 
http://mxr.mozilla.org/mozilla-central/source/nsprpub/pr/src/threads/combined/prucv.c#154

it sucks that socorro isn't extracting information from those nspr routines.
Digging down a bit further, looks like it may be here - 

http://mxr.mozilla.org/mozilla-central/source/nsprpub/pr/src/md/windows/w95cv.c#248

although it's odd that the signature has WaitForSingleObjectEx and this routine uses WaitForSingleObject.
Maybe this isn't a crash. Looking at the patches that landed in bug 1068349, which is when this regressed, we grab crash stacks in ContentParent::KillHard() when we're about to abort the child process. That would explain the weird content stack. There's no matching parent stack though, not sure where that might be if this is indeed the case.
I guess we don't have a way to get at the browser stack from socorro, but maybe the "raw dumps" have that data. This according to bug 1068349, comment 14.
Mike, these two signatures (top content crashers #2, and #3) appear to be the stacks generated by your work in bug 1068349. Any ideas here on how to get the parent stacks?
Flags: needinfo?(mconley)
One thing we can do here is guess that the most common parent side failure is related to SendLoadPlugin, which is a very common content process call in the list. In which case we might be failing in setting up the plugin bridge:

http://mxr.mozilla.org/mozilla-central/source/dom/ipc/ContentParent.cpp#979
http://mxr.mozilla.org/mozilla-central/source/dom/plugins/ipc/PluginModuleParent.cpp#95
http://mxr.mozilla.org/mozilla-central/source/dom/plugins/base/nsPluginHost.cpp#1292
http://mxr.mozilla.org/mozilla-central/source/dom/plugins/base/nsPluginHost.cpp#1278

I'm wondering if we really want to abort in all of these load plugin failure cases.
(In reply to Jim Mathies [:jimm] from comment #17)
> Mike, these two signatures (top content crashers #2, and #3) appear to be
> the stacks generated by your work in bug 1068349. Any ideas here on how to
> get the parent stacks?

Unfortunately, at this stage, the only way to get the parent stack is to get the raw JSON dump of the crash report and parse it. :(

I filed a bug about getting the front-end to support multiple crash stacks, but it doesn't appear to be getting any traction - bug 1110498.
Flags: needinfo?(mconley)
Depends on: 1110498
Flags: needinfo?(dmajor)
Do bug 1121751 / bug 1123551 help? They may be related, though they spiked well after this one was filed.
Depends on: 1124933
Basically the same bug but with mac signatures.
Blocks: 1118517
See Also: 1118517
Most of the top ten crashes on linux probably fall in here as well, unfortunately they crash at random addresses so they don't group.

https://crash-stats.mozilla.com/topcrasher/products/Firefox/versions/38.0a1/date_range_type/build/crash_type/content/os_name/Linux/result_count/50?days=7
Well unfortunately the parent stacks here don't help much. We might have to rely on annotations to track all these down.

1 CrashReporter::CreatePairedMinidumps(void *,unsigned long,nsIFile * *)
2 mozilla::dom::CrashReporterParent::GeneratePairedMinidump<mozilla::dom::ContentParent>(mozilla::dom::ContentParent *)
3 mozilla::dom::ContentParent::KillHard()
4 mozilla::dom::ContentParent::ProcessingError(mozilla::ipc::HasResultCodes::Result)
5 mozilla::ipc::MessageChannel::DispatchSyncMessage(IPC::Message const &)
6 mozilla::ipc::MessageChannel::OnMaybeDequeueOne()
7 MessageLoop::DoWork()
8 mozilla::ipc::DoWorkRunnable::Run()
9 nsThread::ProcessNextEvent(bool,bool *)
10 mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate *)
The general pattern here is pretty simple: child makes a high priority call to the parent, in handling this the parent tries to send a syn or interrupt message -> KillHard on send error.

This is where our unexpected cpow aborts show up.
(In reply to Jim Mathies [:jimm] from comment #25)
> The general pattern here is pretty simple: child makes a high priority call
> to the parent, in handling this the parent tries to send a syn or interrupt
> message -> KillHard on send error.
> 
> This is where our unexpected cpow aborts show up.

Hmm, a lot of these are happening in DispatchSyncMessage in the parent, which looks like the MessageChannel call we make when delivering a sync or interrupt message to a handler. In which case a lot of these aborts are caused by failures in the handlers themselves (comment 18).
Another common parent stack involves DispatchInterruptMessage.
I'm seeing all tabs crash frequently on my laptop after having updated to today's nightly build from 20150121030203.
bp-83a93de6-d344-49cb-aaa3-4e7a12150127
(In reply to Wayne Mery (:wsmwk) from comment #28)
> I'm seeing all tabs crash frequently on my laptop after having updated to
> today's nightly build from 20150121030203.
> bp-83a93de6-d344-49cb-aaa3-4e7a12150127

That's one of the common signatures with plugins - 

mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const&)

Wayne, any tricks to reproducing? Do you sue click to play?
Flags: needinfo?(vseerror)
(In reply to Jim Mathies [:jimm] from comment #29)
> (In reply to Wayne Mery (:wsmwk) from comment #28)
> > I'm seeing all tabs crash frequently on my laptop after having updated to
> > today's nightly build from 20150121030203.
> > bp-83a93de6-d344-49cb-aaa3-4e7a12150127
> 
> That's one of the common signatures with plugins - 
> 
> mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const&)
> 
> Wayne, any tricks to reproducing? 

Not so far that I know. I currently have 158 tabs, 95% are bugzilla and crash-stats pages. I'm not yet sure if it's specific to a page. It feels like it's workload based. ANd I haven't crash once this morning so far.

I don't know that URL field can be trusted 100%, but it may be noteworthy that none of my 8 crashes from yesterday cite a bugzilla URL.

Are there specific steps others have found?


> Do you [use] click to play?

If you mean flash, I have flashblock currently enabled. And I think only one page with a flash image (which I haven't played) is http://crowdbooster.com/ 

All plugins except Openh264 and flash are "Ask to Activate"
A few addons enabled listed below. 

Extensions
----------

Name: About sessionstore
Version: 0.32
Enabled: true
ID: aboutsessionstore@dt

Name: about:addons-memory
Version: 10
Enabled: true
ID: about-addons-memory@tn123.org

Name: Cleanest Addon Manager
Version: 7.0
Enabled: true
ID: cam@sdrocking.com

Name: Context Search
Version: 0.6.3
Enabled: true
ID: {902D2C4A-457A-4EF9-AD43-7014562929FF}

Name: CopyAllUrls
Version: 0.9.2
Enabled: true
ID: {960BE052-4847-422b-9AD6-8631D3D0A607}

Name: DNS Flusher
Version: 3.1.1
Enabled: true
ID: {7d575baa-b543-11dc-8314-0800200c9a66}

Name: Flashblock
Version: 1.5.18
Enabled: true
ID: {3d7eb24f-2740-49df-8937-200b1cc08f8a}

Name: Google Shortcuts
Version: 2.1.8.2
Enabled: true
ID: {5C46D283-ABDE-4dce-B83C-08881401921C}

Name: Greasefire2
Version: 2.0.2
Enabled: true
ID: greasefire2@b0nk3rz.net

Name: Greasemonkey
Version: 2.3
Enabled: true
ID: {e4a8a97b-f2ed-450b-b12d-ee082ba24781}

Name: Nightly Tester Tools
Version: 3.7
Enabled: true
ID: {8620c15f-30dc-4dba-a131-7c5d20cf4a29}

Name: Restartless Restart
Version: 9
Enabled: true
ID: restartless.restart@erikvold.com

Name: Session Manager
Version: 0.8.1.7pre20141016b
Enabled: true
ID: {1280606b-2510-4fe0-97ef-9b5a22eafe30}

Name: Snap Links Plus
Version: 2.4.2
Enabled: true
ID: snaplinks@snaplinks.mozdev.org

Name: Tabhunter
Version: 1.0.5
Enabled: true
ID: tabhunter@ericpromislow.com

Name: TinyURL Generator
Version: 2.6.1
Enabled: true
ID: tinyurl.addon@fast-chat.co.uk

Name: Tree Style Tab
Version: 0.15.2014120200a034137
Enabled: true
ID: treestyletab@piro.sakura.ne.jp

Name: ViewAbout
Version: 2.0.1
Enabled: true
ID: viewabout@rumblingedge.com
Flags: needinfo?(vseerror) → needinfo?(jmathies)
Flags: needinfo?(jmathies)
I wonder if we should change the signature for RecvFoo functions so that you're required to return something more explicit. It would look something like:
  return Success;
or
  return Failure("this is why it failed");

Then we would annotate the crash dump with the failure message.

This would be a lot of grunt work, but it might be worth it given all these crashes.
Also, number of crashes approx doubled for this past one week, compared to the previous one week period, for the exact signature WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)
Blocks: 1124064
(In reply to Jim Mathies [:jimm] from comment #29)
> (In reply to Wayne Mery (:wsmwk) from comment #28)
> > I'm seeing all tabs crash frequently on my laptop after having updated to
> > today's nightly build from 20150121030203.
> > bp-83a93de6-d344-49cb-aaa3-4e7a12150127
> 
> That's one of the common signatures with plugins - 
> 
> mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const&)
> 
> Wayne, any tricks to reproducing? Do you sue click to play?

OK, best guess is I'm hitting Bug 1026093 - Firefox crashes when loading Flash in an e10s window with hardware acceleration disabled - because this laptop has HWA disabled. But afaict I hadn't played any flash image because flashblock is installed.  Of my 158 tabs, ~ 8 were not bugzilla links, and three definitely flash. #1 and #2 below were specifcally cited in my submitted crashes
1. http://www.scamadviser.com/is-baba-mail.com-a-fake-site.html
2. http://www.softpedia.com/get/Internet/Other-Internet-Related/SpeedyFox.shtml
3. http://crowdbooster.com/
OTOH, a couple hours ago I crashed bp-c0a77f10-7adc-4e45-b8bf-e8f592150128 and assessing all the non-BMO web pages I'm fairly certain THIS time none of them had flash bits.  But I am seeing a lot of doorhangers of "A web page is causing Nightly to run slowly. What would you like to do?"
Most common callers based on a sample of the last 100 submitted crashes - 

46	mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &)
24	mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *)
9	mozilla::dom::PBrowserChild::SendGetWidgetNativeData(unsigned int *)
8	mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *)
4	mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *)
2	mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
2	mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::PBrowserChild *,mozilla::dom::ScreenDetails *,bool *)
2	mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *)
1	mozilla::dom::PBrowserChild::SendCreateWindow(mozilla::dom::PBrowserChild *,unsigned int const &,bool const &,bool const &,bool const &,nsString const &,nsString const &,nsString const &,nsString const &,bool *,nsTArray<mozilla::dom::FrameScriptInfo> *,nsCString *)
1	mozilla::dom::PBrowserChild::SendSetInputContext(int const &,int const &,nsString const &,nsString const &,nsString const &,int const &,int const &)
Depends on: 1127374
Depends on: 1127378
Depends on: 1127379
Keywords: meta
Crash Signature: [@ WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [@ WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] → [@ WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [@ WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [@ WaitForSingleObjectEx | CLayeredObject<CDeviceContextState>::CCo…
latest sample:

35      mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &)
20      mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *)
16      mozilla::dom::PBrowserChild::SendGetWidgetNativeData(unsigned int *)
13      mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *)
5       mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *)
3       mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
3       mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
2       mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::PBrowserChild *,mozilla::dom::ScreenDetails *,bool *)
1       mozilla::dom::PBrowserChild::SendCreateWindow(mozilla::dom::PBrowserChild *,unsigned int const &,bool const &,bool const &,bool const &,nsString const &,nsString const &,nsString const &,nsString const &,bool *,nsTArray<mozilla::dom::FrameScriptInfo> *,nsCString *)
1       mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *)
Depends on: 1128253
Depends on: 1128254
2015-01-31, 100 samples:
34      mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &)
20      mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *)
17      mozilla::dom::PBrowserChild::SendGetWidgetNativeData(unsigned int *)
13      mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *)
5       mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *)
3       mozilla::dom::PBrowserChild::SendCreateWindow(mozilla::dom::PBrowserChild *,unsigned int const &,bool const &,bool const &,bool const &,nsString const &,nsString const &,nsString const &,nsString const &,bool *,n
sTArray<mozilla::dom::FrameScriptInfo> *,nsCString *)
2       mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
2       mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::PBrowserChild *,mozilla::dom::ScreenDetails *,bool *)
1       mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *)
1       mozilla::dom::PContentChild::SendClipboardHasText(int const &,bool *)
1       mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
2015-02-01:
22|27.160494|mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned __int64 *)
17|20.987654|mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *)
11|13.580247|mozilla::dom::PBrowserChild::SendCreateWindow(mozilla::dom::PBrowserChild *,unsigned int const &,bool const &,bool const &,bool const &,nsString const &,nsString const &,nsString const &,nsString const &,bool *,nsTArray<mozilla::dom::FrameScriptInfo> *,nsCString *)
10|12.345679|mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *)
5|6.172840|mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *)
4|4.938272|mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
3|3.703704|mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *)
3|3.703704|mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &,tag_nsresult *)
2|2.469136|mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
1|1.234568|mozilla::PRemoteSpellcheckEngineChild::SendCheck(nsString const &,bool *)
1|1.234568|mozilla::dom::PStorageChild::SendPreload(nsCString const &,unsigned int const &,nsTArray<nsString> *,nsTArray<nsString> *,tag_nsresult *)
1|1.234568|mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::PBrowserChild *,mozilla::dom::ScreenDetails *,bool *)
1|1.234568|mozilla::dom::PBrowserChild::SendPPluginWidgetConstructor(mozilla::plugins::PPluginWidgetChild *)

bugs 1127374 and 1127379 clearly helped.
Depends on: 1128454
Depends on: 1128457
bugs 1127374 and 1127379 knocked these crashes down a good bit. I've filed follow up bug 1128454 to investigate problems with RecvLoadPlugin. There are few more worst offenders:

PPluginWidgetChild::SendGetNativePluginPort (bug 1127378)
PCookieServiceChild::SendGetCookieString (bug 1128457)
PBrowserChild::SendCreateWindow (bug 1128253)
PContentChild::SendLoadPlugin <- some sort of routing issue on the parent side
2015-02-01:
29.9	mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned __int64 *)
22.8	mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *)
20.4	mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *)
6.6	mozilla::dom::PBrowserChild::SendCreateWindow(mozilla::dom::PBrowserChild *,unsigned int const &,bool const &,bool const &,bool const &,nsString const &,nsString const &,nsString const &,nsString const &,bool *,nsTArray<mozilla::dom::FrameScriptInfo> *,nsCString *)
4.2	mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &,tag_nsresult *)
3.6	mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
3.0	mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::PBrowserChild *,mozilla::dom::ScreenDetails *,bool *)
3.0	mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *)
2.4	mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *)
2.4	mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
0.6	mozilla::dom::PBrowserChild::SendPPluginWidgetConstructor(mozilla::plugins::PPluginWidgetChild *)
0.6	mozilla::PRemoteSpellcheckEngineChild::SendCheck(nsString const &,bool *)
0.6	mozilla::dom::PStorageChild::SendPreload(nsCString const &,unsigned int const &,nsTArray<nsString> *,nsTArray<nsString> *,tag_nsresult *)
SendGetNativePluginPort has a patch on inbound, SendGetCookieString diagnosis is waiting on annotation data which should start showing up today.

33.2	mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *)
27.6	mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned __int64 *)
19.1	mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *)
6.0	mozilla::dom::PBrowserChild::SendCreateWindow(mozilla::dom::PBrowserChild *,unsigned int const &,bool const &,bool const &,bool const &,nsString const &,nsString const &,nsString const &,nsString const &,bool *,nsTArray<mozilla:
4.5	mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *)
3.0	mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
2.0	mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,__int64 *)
2.0	mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *)
0.5	mozilla::dom::PBrowserChild::SendPPluginWidgetConstructor(mozilla::plugins::PPluginWidgetChild *)
0.5	mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::PBrowserChild *,mozilla::dom::ScreenDetails *,bool *)
0.5	mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &,tag_nsresult *)
0.5	mozilla::dom::PContentChild::SendGetRandomValues(unsigned int const &,nsTArray<unsigned char> *)
0.50	mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
The ipc error data coming back from bug 1124933 is rather interesting. I don't have enough to cover everything yet (just about 30 reports), but the prelim data is interesting:

- Most of the failures in PPluginWidgetChild::SendGetNativePluginPort are actually failures in PPluginWidget::Create method, which is async. I think we should switch this call to sync and make creation failures passive.

- PCookieServiceChild::SendGetCookieString errors are MsgNotAllowed errors.

- PBrowserChild::SendGetRenderFrameInfo errors are MsgValueError errors.
Depends on: 1129575
Blocks: 1123817
No longer blocks: 1123817
Depends on: 1123817
Signature breakdown:
41|28.873239|mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned __int64 *)
35|24.647887|mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *)
29|20.422535|mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *)
15|10.563380|mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &,nsresult *)
8|5.633803|mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *)
5|3.521127|mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *)
3|2.112676|mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
1|0.704225|mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
1|0.704225|mozilla::jsipc::PJavaScriptChild::SendCallOrConstruct(unsigned __int64 const &,nsTArray<mozilla::jsipc::JSParam> const &,bool const &,mozilla::jsipc::ReturnStatus *,mozilla::jsipc::JSVariant *,nsTArray<mozilla::jsipc::JSParam> *)
1|0.704225|mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,__int64 *)
1|0.704225|mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,unsigned __int64 const &,mozilla::layers::TargetConfig const &,nsTArray<mozilla::layers::PluginWindowData> const &,bool const &,bool cons
t &,unsigned int const &,bool const &,mozilla::TimeStamp const &,nsTArray<mozilla::layers::EditReply> *)
1|0.704225|mozilla::dom::PBrowserChild::SendEndIMEComposition(bool const &,nsString *)
1|0.704225|mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::IdType<mozilla::dom::TabParent> const &,mozilla::dom::ScreenDetails *,bool *)

IPC error breakdown per signature:
mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned __int64 *):
 '(msgtype=0x840003,name=PPluginWidget::Msg_Create) Processing error: message was deserialized, but the handler returned false (indicating failure)'
mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *):
 '(msgtype=0x340001,name=???) Message not allowed: cannot be sent/recvd in this state'
mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *):
 '(msgtype=0x840003,name=PPluginWidget::Msg_Create) Processing error: message was deserialized, but the handler returned false (indicating failure)'
mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &,nsresult *):
 '(msgtype=0x300040,name=PContent::Msg_LoadPlugin) Processing error: message was deserialized, but the handler returned false (indicating failure)'
mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *):
 '(msgtype=0x20001E,name=???) Message not allowed: cannot be sent/recvd in this state'
mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *):
 '(msgtype=0x200002,name=PBrowser::Msg_PRenderFrameConstructor) Value error: message was deserialized, but contained an illegal value'
mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *):
 '(msgtype=0x30007B,name=???) Message not allowed: cannot be sent/recvd in this state'
mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *):
 '(msgtype=0x20000E,name=???) Message not allowed: cannot be sent/recvd in this state'
mozilla::jsipc::PJavaScriptChild::SendCallOrConstruct(unsigned __int64 const &,nsTArray<mozilla::jsipc::JSParam> const &,bool const &,mozilla::jsipc::ReturnStatus *,mozilla::jsipc::JSVariant *,nsTArray<mozilla::jsipc::JSParam> *):
 '(msgtype=0x660016,name=???) Message not allowed: cannot be sent/recvd in this state'
mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,__int64 *):
 '(msgtype=0x20001E,name=???) Message not allowed: cannot be sent/recvd in this state'
mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,unsigned __int64 const &,mozilla::layers::TargetConfig const &,nsTArray<mozilla::layers::PluginWindowData> const &,bool const &,bool const &,unsigne
d int const &,bool const &,mozilla::TimeStamp const &,nsTArray<mozilla::layers::EditReply> *):
 '(msgtype=0x3E0004,name=PDocAccessible::Msg_ShowEvent) Processing error: message was deserialized, but the handler returned false (indicating failure)'
mozilla::dom::PBrowserChild::SendEndIMEComposition(bool const &,nsString *):
 '(msgtype=0x20001A,name=???) Message not allowed: cannot be sent/recvd in this state'
mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::IdType<mozilla::dom::TabParent> const &,mozilla::dom::ScreenDetails *,bool *):
 '(msgtype=0x3E0004,name=PDocAccessible::Msg_ShowEvent) Processing error: message was deserialized, but the handler returned false (indicating failure)'
No longer depends on: 1123817
The "message not allowed" errors are a result of CPOW timeouts. These might be potential deadlocks that were turned into crashes by the hang detector. Or they could just be bad luck. The crash stacks for both processes might give more information.
Depends on: 1130836
Signature breakdown:
---------------------
33|46.5%|mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *)
9|12.7%|mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &,nsresult *)
7|9.9%|mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *)
5|7.0%|mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
4|5.6%|mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
4|5.6%|mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::IdType<mozilla::dom::TabParent> const &,mozilla::dom::ScreenDetails *,bool *)
3|4.2%|mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *)
1|1.4%|mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *)
1|1.4%|mozilla::dom::PBrowserChild::SendNotifyIMEFocus(bool const &,nsIMEUpdatePreference *,unsigned int *)
1|1.4%|mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,unsigned __int64 const &,mozilla::layers::TargetConfig const &,nsTArray<mozilla::layers::PluginWindowData> const &,bool const &,bool const &,
unsigned int const &,bool const &,mozilla::TimeStamp const &,nsTArray<mozilla::layers::EditReply> *)
1|1.4%|mozilla::dom::PBrowserChild::SendSyncMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *)
1|1.4%|mozilla::plugins::PPluginWidgetChild::SendCreate(nsresult *)
1|1.4%|mozilla::PRemoteSpellcheckEngineChild::SendCheck(nsString const &,bool *)

IPC error breakdown per signature:
-----------------------------------
mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const &,bool const &,bool const &,IPC::SerializedLoadContext const &,nsCString *):
 '(msgtype=0x340001,name=???) Message not allowed: cannot be sent/recvd in this state'
 '(msgtype=0x3E0004,name=PDocAccessible::Msg_ShowEvent) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PContentChild::SendLoadPlugin(unsigned int const &,nsresult *):
 '(msgtype=0x300040,name=PContent::Msg_LoadPlugin) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PBrowserChild::SendGetRenderFrameInfo(mozilla::layout::PRenderFrameChild *,mozilla::layout::ScrollingBehavior *,mozilla::layers::TextureFactoryIdentifier *,unsigned __int64 *):
 '(msgtype=0x200002,name=PBrowser::Msg_PRenderFrameConstructor) Value error: message was deserialized, but contained an illegal value'

mozilla::dom::PContentChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *):
 '(msgtype=0x30007B,name=???) Message not allowed: cannot be sent/recvd in this state'

mozilla::dom::PBrowserChild::SendRpcMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *):
 '(msgtype=0x20000E,name=???) Message not allowed: cannot be sent/recvd in this state'

mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::IdType<mozilla::dom::TabParent> const &,mozilla::dom::ScreenDetails *,bool *):
 '(msgtype=0x980009,name=???) Message not allowed: cannot be sent/recvd in this state'
 '(msgtype=0x3E0004,name=PDocAccessible::Msg_ShowEvent) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PBrowserChild::SendGetInputContext(int *,int *,int *):
 '(msgtype=0x20001E,name=???) Message not allowed: cannot be sent/recvd in this state'

mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(unsigned int *):
 '(msgtype=0x3E0004,name=PDocAccessible::Msg_ShowEvent) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PBrowserChild::SendNotifyIMEFocus(bool const &,nsIMEUpdatePreference *,unsigned int *):
 '(msgtype=0x3E0004,name=PDocAccessible::Msg_ShowEvent) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,unsigned __int64 const &,mozilla::layers::TargetConfig const &,nsTArray<mozilla::layers::PluginWindowData> const &,bool const &,bool const &,unsigne
d int const &,bool const &,mozilla::TimeStamp const &,nsTArray<mozilla::layers::EditReply> *):
 '(msgtype=0x3E0004,name=PDocAccessible::Msg_ShowEvent) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PBrowserChild::SendSyncMessage(nsString const &,mozilla::dom::ClonedMessageData const &,nsTArray<mozilla::jsipc::CpowEntry> const &,IPC::Principal const &,nsTArray<nsString> *):
 '(msgtype=0x20000C,name=???) Message not allowed: cannot be sent/recvd in this state'

mozilla::plugins::PPluginWidgetChild::SendCreate(nsresult *):
 '(msgtype=0x840003,name=PPluginWidget::Msg_Create) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::PRemoteSpellcheckEngineChild::SendCheck(nsString const &,bool *):
 '(msgtype=0x900003,name=???) Message not allowed: cannot be sent/recvd in this state'
See Also: → 1130734
Depends on: 1134252
Whiteboard: [KillHard Windows signature]
Crash Signature: , nsTArray<nsCString>&, _NPSavedData*, short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] → , nsTArray<nsCString>&, _NPSavedData*, short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena]
Depends on: 1123464
Depends on: 1140714
Crash Signature: , nsTArray<nsCString>&, _NPSavedData*, short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] → , nsTArray<nsCString>&, _NPSavedData*, short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a]
Crash Signature: , nsTArray<nsCString>&, _NPSavedData*, short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] → , nsTArray<nsCString>&, _NPSavedData*, short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | moz…
Summary: crash in WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int) → [meta] crash in WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)
> Child protocol error abort messages:
> ------------------------------------
> 101	'(msgtype=0x980009,name=???) Route error: message sent to unknown actor ID'

This is PPluginWidget::ParentShutdown.

> 2	'(msgtype=0x52000D,name=???) Route error: message sent to unknown actor ID'

This is PFTPChannel::FlushedForDiversion

> 194	'(msgtype=0x940009,name=???) Route error: message sent to unknown actor ID'

I don't know what this is... My tree says it's in the PPluginStream protocol, but that protocol doesn't have 9 message types...
(In reply to Ben Turner [:bent] (use the needinfo flag!) from comment #50)
> > 194	'(msgtype=0x940009,name=???) Route error: message sent to unknown actor ID'
> 
> I don't know what this is... My tree says it's in the PPluginStream
> protocol, but that protocol doesn't have 9 message types...

PPluginWidget::ParentShutdown moves around whenever the protocols before PPluginWidget in the build change. :)
This is meta, we don't release track these bugs.
Depends on: 1139860
Depends on: 1120785
Depends on: 1140915
Depends on: 1139309
Depends on: 1121629
Assignee: jmathies → nobody
OS: Windows NT → Windows
Alias: WaitForSingleObjectEx
Summary: [meta] crash in WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int) → [meta] Crash signatures starting with: "WaitForSingleObjectEx |"
Crash Signature: , short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] → , short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] […
Version: 37 Branch → unspecified
Crash Signature: int)] [@ WaitForSingleObjectEx | js::GetElements(JSContext*, JS::Handle<JSObject*>, unsigned int, JS::Value*) ] → int)] [@ WaitForSingleObjectEx | js::GetElements(JSContext*, JS::Handle<JSObject*>, unsigned int, JS::Value*) ] [@ WaitForSingleObjectEx | js::frontend::Parser<js::frontend::FullParseHandler>::unaryExpr(js::frontend::Parser<js::frontend::FullParseHandl…
Tobias, no, please don't do that. WaitForSingleObjectEx is a frame that doesn't say anything at all, that's almost as good as saying "this is a bug for some crashes". This was filed for specific signatures, don't add any unless you *know* they are the same thing and don't change the summary unless you *know* that change is right.
Alias: WaitForSingleObjectEx
Crash Signature: , short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [… → , short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)]
Summary: [meta] Crash signatures starting with: "WaitForSingleObjectEx |" → [meta] crashes in mozilla::CondVar::Wait(unsigned int)
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #53)
> Summary: [meta] Crash signatures starting with: "WaitForSingleObjectEx |" → [meta] crashes in mozilla::CondVar::Wait(unsigned int)

Sorry! Add this sig?
[@ WaitForSingleObjectEx | arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ]
Crash Signature: , short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] → , short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] […
[@ WaitForSingleObjectEx | arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ]
https://crash-stats.mozilla.com/report/index/0a97d07c-4955-49d2-88a3-9d8eb2150607
KaiRo mentioned this as a top issue for Aurora/Dev edition in today's channel meeting. Adding a tracking flag for FF40.
[@ WaitForSingleObjectEx | XPCWrappedNative::HasInterfaceNoQI(nsID const&) ]

https://crash-stats.mozilla.com/report/index/310b13e5-333f-4a4f-bb9c-424012150612

Crashing Thread
Frame 	Module 	Signature 	Source
0 	ntdll.dll 	NtWaitForSingleObject 	
1 	kernelbase.dll 	WaitForSingleObjectEx 	
2 	xul.dll 	XPCWrappedNative::HasInterfaceNoQI(nsID const&) 	js/xpconnect/src/XPCInlines.h
3 		@0x7fe00000000 	
4 	nss3.dll 	PR_WaitCondVar 	nsprpub/pr/src/threads/combined/prucv.c
5 	xul.dll 	mozilla::CondVar::Wait(unsigned int) 	xpcom/glue/CondVar.h
6 	xul.dll 	mozilla::ipc::MessageChannel::WaitForSyncNotify() 	ipc/glue/WindowsMessageLoop.cpp
7 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*) 	ipc/glue/MessageChannel.cpp
8 	xul.dll 	mozilla::dom::PBrowserChild::SendRpcMessage(nsString const&, mozilla::dom::ClonedMessageData const&, nsTArray<mozilla::jsipc::CpowEntry> const&, IPC::Principal const&, nsTArray<nsString>*) 	obj-firefox/ipc/ipdl/PBrowserChild.cpp
9 	xul.dll 	mozilla::dom::TabChild::DoSendBlockingMessage(JSContext*, nsAString_internal const&, mozilla::dom::StructuredCloneData const&, JS::Handle<JSObject*>, nsIPrincipal*, nsTArray<nsString>*, bool) 	dom/ipc/TabChild.cpp
10 	xul.dll 	nsFrameMessageManager::SendMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>, bool) 	dom/base/nsFrameMessageManager.cpp
11 	xul.dll 	nsFrameMessageManager::SendRpcMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>) 	dom/base/nsFrameMessageManager.cpp
12 	xul.dll 	mozilla::dom::TabChildGlobal::SendRpcMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>) 	dom/ipc/TabChild.h
13 	xul.dll 	XPTC__InvokebyIndex 	xpcom/reflect/xptcall/md/win32/xptcinvoke_asm_x86_64.asm
14 		@0x27628e7 	
15 	xul.dll 	XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) 	js/xpconnect/src/XPCWrappedNativeJSOps.cpp
16 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
17 	xul.dll 	js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/src/proxy/CrossCompartmentWrapper.cpp
18 	xul.dll 	js::proxy_Call(JSContext*, unsigned int, JS::Value*) 	js/src/proxy/Proxy.cpp
19 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
20 	xul.dll 	js::jit::DoCallFallback 	js/src/jit/BaselineIC.cpp
21 		@0x4762bc
Crash Signature: int)] [@ WaitForSingleObjectEx | arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ] → int)] [@ WaitForSingleObjectEx | arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ] [@ WaitForSingleObjectEx | XPCWrappedNative::HasInterfaceNoQI(nsID const&) ]
[@ WaitForSingleObjectEx | js::TypeOfValue(JS::Value const&) ]

https://crash-stats.mozilla.com/report/index/7ad08603-dc0e-40f4-b228-c7d132150612

Crashing Thread
Frame 	Module 	Signature 	Source
0 	ntdll.dll 	NtWaitForSingleObject 	
1 	kernelbase.dll 	WaitForSingleObjectEx 	
2 	xul.dll 	js::TypeOfValue(JS::Value const&) 	js/src/vm/Interpreter.cpp
3 		@0x7fe00000000 	
4 	nss3.dll 	PR_WaitCondVar 	nsprpub/pr/src/threads/combined/prucv.c
5 	xul.dll 	mozilla::CondVar::Wait(unsigned int) 	xpcom/glue/CondVar.h
6 	xul.dll 	mozilla::ipc::MessageChannel::WaitForSyncNotify() 	ipc/glue/WindowsMessageLoop.cpp
7 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*) 	ipc/glue/MessageChannel.cpp
8 	xul.dll 	mozilla::dom::PBrowserChild::SendRpcMessage(nsString const&, mozilla::dom::ClonedMessageData const&, nsTArray<mozilla::jsipc::CpowEntry> const&, IPC::Principal const&, nsTArray<nsString>*) 	obj-firefox/ipc/ipdl/PBrowserChild.cpp
9 	xul.dll 	mozilla::dom::TabChild::DoSendBlockingMessage(JSContext*, nsAString_internal const&, mozilla::dom::StructuredCloneData const&, JS::Handle<JSObject*>, nsIPrincipal*, nsTArray<nsString>*, bool) 	dom/ipc/TabChild.cpp
10 	xul.dll 	nsFrameMessageManager::SendMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>, bool) 	dom/base/nsFrameMessageManager.cpp
11 	xul.dll 	nsFrameMessageManager::SendRpcMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>) 	dom/base/nsFrameMessageManager.cpp
12 	xul.dll 	mozilla::dom::TabChildGlobal::SendRpcMessage(nsAString_internal const&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, nsIPrincipal*, JSContext*, unsigned char, JS::MutableHandle<JS::Value>) 	dom/ipc/TabChild.h
13 	xul.dll 	XPTC__InvokebyIndex 	xpcom/reflect/xptcall/md/win32/xptcinvoke_asm_x86_64.asm
14 		@0x27630e7 	
15 	xul.dll 	XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) 	js/xpconnect/src/XPCWrappedNativeJSOps.cpp
16 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
17 	xul.dll 	js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/src/proxy/CrossCompartmentWrapper.cpp
18 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
19 	xul.dll 	js::jit::DoCallFallback 	js/src/jit/BaselineIC.cpp
20 		@0x3b62bc
Crash Signature: int)] [@ WaitForSingleObjectEx | arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ] [@ WaitForSingleObjectEx | XPCWrappedNative::HasInterfaceNoQI(nsID const&) ] → int)] [@ WaitForSingleObjectEx | arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ] [@ WaitForSingleObjectEx | XPCWrappedNative::HasInterfaceNoQI(nsID const&) ] [@ WaitForSingleObjectEx | js::TypeOfValue(JS::Value const&) ]
https://crash-stats.mozilla.com/report/list?product=Firefox&range_value=7&range_unit=days&date=2015-06-18&signature=shutdownhang+|+WaitForSingleObjectEx+|+WaitForSingleObject+|+PR_WaitCondVar+|+mozilla%3A%3ACondVar%3A%3AWait%28unsigned+int%29&version=Firefox%3A40.0a2

https://crash-stats.mozilla.com/report/list?product=Firefox&range_value=7&range_unit=days&date=2015-06-18&signature=shutdownhang+|+WaitForSingleObjectEx+|+WaitForSingleObject+|+PR_WaitCondVar+|+PR_JoinThread+|+SmartCardMonitoringThread%3A%3A~SmartCardMonitoringThread%28%29&version=Firefox%3A40.0a2
Crash Signature: int)] [@ WaitForSingleObjectEx | arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ] [@ WaitForSingleObjectEx | XPCWrappedNative::HasInterfaceNoQI(nsID const&) ] [@ WaitForSingleObjectEx | js::TypeOfValue(JS::Value const&) ] → int)] [@ WaitForSingleObjectEx | arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ] [@ WaitForSingleObjectEx | XPCWrappedNative::HasInterfaceNoQI(nsID const&) ] [@ WaitForSingleObjectEx | js::TypeOfValue(JS::Value const&) ] [@ shutdownhang |…
Depends on: 1176245
(In reply to Jim Mathies [:jimm] from comment #52)
> This is meta, we don't release track these bugs.
No longer depends on: 1176245
[@ WaitForSingleObjectEx | SetEvent ]

https://crash-stats.mozilla.com/report/index/98abfff4-8aa2-4587-95d4-080722150623
Crash Signature: SmartCardMonitoringThread::~SmartCardMonitoringThread()] → SmartCardMonitoringThread::~SmartCardMonitoringThread()] [@ WaitForSingleObjectEx | SetEvent ]
Untracking it as it is a meta bug and it is hard to do anything on those.
[@ WaitForSingleObjectEx | xul.dll@0x17e90bf | js::TypeScript::ThisTypes(JSScript*) ]

https://crash-stats.mozilla.com/report/index/08637eba-ed04-440b-b88b-482252150702

Crashing Thread
Frame 	Module 	Signature 	Source
0 	ntdll.dll 	NtWaitForSingleObject 	
1 	kernelbase.dll 	WaitForSingleObjectEx 	
2 	xul.dll 	xul.dll@0x17e90bf 	
3 	xul.dll 	js::TypeScript::ThisTypes(JSScript*) 	js/src/vm/TypeInference-inl.h
4 		@0xfffffffe 	
5 	nss3.dll 	PR_WaitCondVar 	nsprpub/pr/src/threads/combined/prucv.c
6 	xul.dll 	mozilla::CondVar::Wait(unsigned int) 	xpcom/glue/CondVar.h
7 	xul.dll 	mozilla::ipc::MessageChannel::WaitForSyncNotify() 	ipc/glue/WindowsMessageLoop.cpp
8 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*) 	ipc/glue/MessageChannel.cpp
9 	xul.dll 	mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const&, bool const&, bool const&, IPC::SerializedLoadContext const&, nsCString*) 	obj-firefox/ipc/ipdl/PCookieServiceChild.cpp
10 	xul.dll 	mozilla::net::CookieServiceChild::GetCookieStringInternal(nsIURI*, nsIChannel*, char**, bool) 	netwerk/cookie/CookieServiceChild.cpp
11 	xul.dll 	mozilla::net::CookieServiceChild::GetCookieString(nsIURI*, nsIChannel*, char**) 	netwerk/cookie/CookieServiceChild.cpp
12 	xul.dll 	nsHTMLDocument::GetCookie(nsAString_internal&, mozilla::ErrorResult&) 	dom/html/nsHTMLDocument.cpp
13 	xul.dll 	mozilla::dom::HTMLDocumentBinding::get_cookie 	obj-firefox/dom/bindings/HTMLDocumentBinding.cpp
14 	xul.dll 	mozilla::dom::GenericBindingGetter(JSContext*, unsigned int, JS::Value*) 	dom/bindings/BindingUtils.cpp
15 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
16 	xul.dll 	js::GetProperty(JSContext*, JS::Handle<JSObject*>, JS::Handle<JSObject*>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	js/src/vm/NativeObject.h
17 	xul.dll 	mozilla::dom::GetPropertyOnPrototype(JSContext*, JS::Handle<JSObject*>, JS::Handle<jsid>, bool*, JS::MutableHandle<JS::Value>) 	dom/bindings/BindingUtils.cpp
18 	xul.dll 	mozilla::dom::HTMLDocumentBinding::DOMProxyHandler::get(JSContext*, JS::Handle<JSObject*>, JS::Handle<JSObject*>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	obj-firefox/dom/bindings/HTMLDocumentBinding.cpp
19 	xul.dll 	js::Proxy::get(JSContext*, JS::Handle<JSObject*>, JS::Handle<JSObject*>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	js/src/proxy/Proxy.cpp
20 	xul.dll 	js::jit::ComputeGetPropResult 	js/src/jit/BaselineIC.cpp
21 	xul.dll 	js::jit::DoGetPropFallback 	js/src/jit/BaselineIC.cpp
22 		@0x336633
Crash Signature: SmartCardMonitoringThread::~SmartCardMonitoringThread()] [@ WaitForSingleObjectEx | SetEvent ] → SmartCardMonitoringThread::~SmartCardMonitoringThread()] [@ WaitForSingleObjectEx | SetEvent ] [@ WaitForSingleObjectEx | xul.dll@0x17e90bf | js::TypeScript::ThisTypes(JSScript*) ]
Depends on: 1180667
My theory (not for every case but hopefully many) on this bug making it a top content crash is;
It is occurring when users restart to install new builds.
Content process is spamming sync messages to browser.
async shutdown is sent to content.
sync spam continues
Browser continues restart, shutting off it's ability to respond to the sync message spam.
On restart user is immediately presented with content crash.
(I'm unsure if the content crash is happening before or after restart.)

Perhaps a small custom add-on could be written to try simulate this.

bp-ff28bb5f-9d30-40d9-a5bb-12e072150708
bp-4646bf93-4853-4bcf-bb6f-401062150708
Steps (not great)
Linux processor overloaded, was using
(while true;do :;done)|(while true;do :;done)|(while true;do :;done)|(while true;do :;done)|(while true;do :;done)|(while true;do :;done)
nightly with many add-ons installed including one of the restart add-ons. (More is better, crashes above are after I removed many and crash frequency dropped.)
At least one tab that is non-remote (e.g. newtab)
Active content (eg start load a link heavy page) (I think crashes above were from one of my add-ons fetching content)
Use restart 
A non-remote tab shows crash page. no submit :(
(repeat restart if crash not shown )
I submitted crashes using script in bug 1138399
Depends on: 1184276
Depends on: 1185065
Depends on: 1187736
Depends on: 1189277
Depends on: 1194808
Summary: [meta] crashes in mozilla::CondVar::Wait(unsigned int) → [meta] e10s KillHard aborts on Windows
[@ WaitForSingleObjectEx | mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ]

https://crash-stats.mozilla.com/report/index/0255a866-abbb-4201-8271-4419f2150817

Crashing Thread
Frame 	Module 	Signature 	Source
0 	ntdll.dll 	NtWaitForSingleObject 	
1 	kernelbase.dll 	WaitForSingleObjectEx 	
2 	xul.dll 	mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
3 		@0xfffffffe 	
4 	xul.dll 	mozilla::CondVar::Wait(unsigned int) 	xpcom/glue/CondVar.h
5 	xul.dll 	mozilla::ipc::MessageChannel::WaitForSyncNotify() 	ipc/glue/WindowsMessageLoop.cpp
6 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*) 	ipc/glue/MessageChannel.cpp
7 	xul.dll 	mozilla::jsipc::PJavaScriptChild::SendCallOrConstruct(unsigned __int64 const&, nsTArray<mozilla::jsipc::JSParam> const&, bool const&, mozilla::jsipc::ReturnStatus*, mozilla::jsipc::JSVariant*, nsTArray<mozilla::jsipc::JSParam>*) 	obj-firefox/ipc/ipdl/PJavaScriptChild.cpp
8 	xul.dll 	mozilla::jsipc::JavaScriptBase<mozilla::jsipc::PJavaScriptChild>::SendCallOrConstruct(mozilla::jsipc::ObjectId const&, nsTArray<mozilla::jsipc::JSParam> const&, bool const&, mozilla::jsipc::ReturnStatus*, mozilla::jsipc::JSVariant*, nsTArray<mozilla::jsipc::JSParam>*) 	js/ipc/JavaScriptBase.h
9 	xul.dll 	mozilla::jsipc::WrapperOwner::callOrConstruct(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&, bool) 	js/ipc/WrapperOwner.cpp
10 	xul.dll 	CPOWProxyHandler::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/ipc/WrapperOwner.cpp
11 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
12 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
13 	xul.dll 	JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) 	js/src/jsapi.cpp
14 	xul.dll 	mozilla::dom::Function::Call(JSContext*, JS::Handle<JS::Value>, nsTArray<JS::Value> const&, JS::MutableHandle<JS::Value>, mozilla::ErrorResult&) 	obj-firefox/dom/bindings/FunctionBinding.cpp
15 	xul.dll 	mozilla::dom::Function::Call<nsCOMPtr<nsISupports> >(nsCOMPtr<nsISupports> const&, nsTArray<JS::Value> const&, JS::MutableHandle<JS::Value>, mozilla::ErrorResult&, char const*, mozilla::dom::CallbackObject::ExceptionHandling, JSCompartment*) 	obj-firefox/dist/include/mozilla/dom/FunctionBinding.h
16 	xul.dll 	nsGlobalWindow::RunTimeoutHandler(nsTimeout*, nsIScriptContext*) 	dom/base/nsGlobalWindow.cpp
17 	xul.dll 	nsGlobalWindow::RunTimeout(nsTimeout*) 	dom/base/nsGlobalWindow.cpp
18 	xul.dll 	nsGlobalWindow::TimerCallback(nsITimer*, void*) 	dom/base/nsGlobalWindow.cpp
19 	xul.dll 	nsTimerImpl::Fire() 	xpcom/threads/nsTimerImpl.cpp
20 	xul.dll 	nsTimerEvent::Run() 	xpcom/threads/TimerThread.cpp
21 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
22 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/glue/nsThreadUtils.cpp
23 	xul.dll 	nsThread::Shutdown() 	xpcom/threads/nsThread.cpp
24 	xul.dll 	nsRunnableMethodImpl<nsresult ( nsIInterceptedChannel::*)(void), 1>::Run() 	xpcom/glue/nsThreadUtils.h
25 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
26 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/glue/nsThreadUtils.cpp
27 	xul.dll 	nsThread::Shutdown() 	xpcom/threads/nsThread.cpp
28 	xul.dll 	nsRunnableMethodImpl<nsresult ( nsIInterceptedChannel::*)(void), 1>::Run() 	xpcom/glue/nsThreadUtils.h
29 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
30 	xul.dll 	NS_ProcessNextEvent(nsIThread*, bool) 	xpcom/glue/nsThreadUtils.cpp
31 	xul.dll 	nsThread::Shutdown() 	xpcom/threads/nsThread.cpp
32 	xul.dll 	nsRunnableMethodImpl<nsresult ( nsIInterceptedChannel::*)(void), 1>::Run() 	xpcom/glue/nsThreadUtils.h
33 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
34 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
35 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
36 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
37 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
38 	xul.dll 	nsBaseAppShell::Run() 	widget/nsBaseAppShell.cpp
39 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp
40 	xul.dll 	XRE_RunAppShell 	toolkit/xre/nsEmbedFunctions.cpp
41 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
42 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
43 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
44 	xul.dll 	XRE_InitChildProcess 	toolkit/xre/nsEmbedFunctions.cpp
45 	plugin-container.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp
46 	plugin-container.exe 	__tmainCRTStartup 	f:/dd/vctools/crt/crtw32/startup/crt0.c:255
47 	kernel32.dll 	BaseThreadInitThunk 	
48 	ntdll.dll 	RtlUserThreadStart
Crash Signature: SmartCardMonitoringThread::~SmartCardMonitoringThread()] [@ WaitForSingleObjectEx | SetEvent ] [@ WaitForSingleObjectEx | xul.dll@0x17e90bf | js::TypeScript::ThisTypes(JSScript*) ] → SmartCardMonitoringThread::~SmartCardMonitoringThread()] [@ WaitForSingleObjectEx | SetEvent ] [@ WaitForSingleObjectEx | xul.dll@0x17e90bf | js::TypeScript::ThisTypes(JSScript*) ] [@ WaitForSingleObjectEx | mozilla::ipc::MessageChannel::ProcessPendin…
Depends on: 1195940
[@ ntdll.dll@0x4be7a ]
https://crash-stats.mozilla.com/report/index/49e27e73-d9a8-464f-85a8-07c542150901
Crash Signature: mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ] → mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ] [@ ntdll.dll@0x4be7a ]
Depends on: 1205289
Get a lot of [@ ntdll.dll@0x4be7a ] in the last ~30 days ... Any news to this bug ???
[@ ntdll.dll@0x4be7a ]

Very often signature while shutting down FF, but in plugin-container.exe, so it doesn't get auto send out to MCR.

One of the last examples:
https://crash-stats.mozilla.com/report/index/2c2c8cf6-7f6d-43cd-88fe-28d642151004

Crashing Thread:
Frame 	Module 		Signature 		Source
Ø 0 	ntdll.dll 	ntdll.dll@0x4be7a 	
Ø 1 	kernelbase.dll 	kernelbase.dll@0x10db 	
2 	xul.dll 	mozilla::jsipc::ObjectVariant::MaybeDestroy(mozilla::jsipc::ObjectVariant::Type)
						obj-firefox/ipc/ipdl/JavaScriptTypes.cpp

There are "only" 36 reports for Windows 7 in the last 28 days, but I guess they are all from me, because I send out at least 1 by day manual ... :D
[@ shutdownhang | ntdll.dll@0x4be7a ]

https://crash-stats.mozilla.com/report/index/280dd4f5-b4fa-4857-9d4c-060182151010

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	mozilla::`anonymous namespace'::RunWatchdog(void*) 	toolkit/components/terminator/nsTerminator.cpp
1 	nss3.dll 	PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c
2 	nss3.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c
3 	msvcr120.dll 	_callthreadstartex 	f:\dd\vctools\crt\crtw32\startup\threadex.c:376
4 	msvcr120.dll 	_threadstartex 	f:\dd\vctools\crt\crtw32\startup\threadex.c:354
Ø 5 	kernel32.dll 	kernel32.dll@0x159dc 	
Ø 6 	ntdll.dll 	ntdll.dll@0x2a630 	
Ø 7 	kernel32.dll 	kernel32.dll@0x9ba5f 	
Ø 8 	kernel32.dll 	kernel32.dll@0x9ba5f
Crash Signature: mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ] [@ ntdll.dll@0x4be7a ] → mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ] [@ ntdll.dll@0x4be7a ] [@ shutdownhang | ntdll.dll@0x4be7a ]
Keywords: topcrash-plugin
[@ shutdownhang | ReleaseData(void*, unsigned int) ]

https://crash-stats.mozilla.com/report/index/600185e4-6f14-417e-9018-5e29d2151011

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	mozilla::`anonymous namespace'::RunWatchdog(void*) 	toolkit/components/terminator/nsTerminator.cpp
1 	nss3.dll 	PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c
2 	nss3.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c
3 	msvcr120.dll 	_callthreadstartex 	f:\dd\vctools\crt\crtw32\startup\threadex.c:376
4 	msvcr120.dll 	_threadstartex 	f:\dd\vctools\crt\crtw32\startup\threadex.c:354
Ø 5 	kernel32.dll 	kernel32.dll@0x159dc 	
Ø 6 	ntdll.dll 	ntdll.dll@0x2a630 	
Ø 7 	kernel32.dll 	kernel32.dll@0x9ba5f 	
Ø 8 	kernel32.dll 	kernel32.dll@0x9ba5f
Crash Signature: mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ] [@ ntdll.dll@0x4be7a ] [@ shutdownhang | ntdll.dll@0x4be7a ] → mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ] [@ ntdll.dll@0x4be7a ] [@ shutdownhang | ntdll.dll@0x4be7a ] [@ shutdownhang | ReleaseData(void*, unsigned int) ]
https://crash-stats.mozilla.com/report/index/d8da871b-7965-4f35-bb7b-7a37b2151013

FF44.0a1

Crashing Thread
Frame 	Module 	Signature 	Source
Ø 0 	ntdll.dll 	ntdll.dll@0x4be7a 	
Ø 1 	kernelbase.dll 	kernelbase.dll@0x10db 	
2 	mozglue.dll 	je_arena_malloc 	memory/jemalloc/src/include/jemalloc/internal/arena.h
3 	xul.dll 	mozilla::CondVar::Wait(unsigned int) 	xpcom/glue/CondVar.h
4 	xul.dll 	mozilla::ipc::MessageChannel::WaitForSyncNotify() 	ipc/glue/WindowsMessageLoop.cpp
5 	xul.dll 	mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*) 	ipc/glue/MessageChannel.cpp
6 	xul.dll 	mozilla::net::PCookieServiceChild::SendGetCookieString(mozilla::ipc::URIParams const&, bool const&, bool const&, IPC::SerializedLoadContext const&, nsCString*) 	obj-firefox/ipc/ipdl/PCookieServiceChild.cpp
7 	xul.dll 	mozilla::net::CookieServiceChild::GetCookieStringInternal(nsIURI*, nsIChannel*, char**, bool) 	netwerk/cookie/CookieServiceChild.cpp
8 	xul.dll 	mozilla::net::CookieServiceChild::GetCookieString(nsIURI*, nsIChannel*, char**) 	netwerk/cookie/CookieServiceChild.cpp
9 	xul.dll 	nsHTMLDocument::GetCookie(nsAString_internal&, mozilla::ErrorResult&) 	dom/html/nsHTMLDocument.cpp
10 	xul.dll 	mozilla::dom::HTMLDocumentBinding::get_cookie 	obj-firefox/dom/bindings/HTMLDocumentBinding.cpp
11 	xul.dll 	mozilla::dom::GenericBindingGetter(JSContext*, unsigned int, JS::Value*) 	dom/bindings/BindingUtils.cpp
12 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
13 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
14 	xul.dll 	mozilla::dom::GetPropertyOnPrototype(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, bool*, JS::MutableHandle<JS::Value>) 	dom/bindings/BindingUtils.cpp
15 	xul.dll 	mozilla::dom::HTMLDocumentBinding::DOMProxyHandler::get(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	obj-firefox/dom/bindings/HTMLDocumentBinding.cpp
16 	xul.dll 	js::Proxy::get(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	js/src/proxy/Proxy.cpp
17 	xul.dll 	js::GetProperty(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	js/src/vm/NativeObject.h
18 	xul.dll 	js::jit::ComputeGetPropResult 	js/src/jit/BaselineIC.cpp
19 	xul.dll 	js::jit::DoGetPropFallback 	js/src/jit/BaselineIC.cpp
20 		@0x1992e21dd3
Crash Signature: mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ] [@ ntdll.dll@0x4be7a ] [@ shutdownhang | ntdll.dll@0x4be7a ] [@ shutdownhang | ReleaseData(void*, unsigned int) ] → mozilla::ipc::MessageChannel::ProcessPendingRequest(IPC::Message const&) ] [@ ntdll.dll@0x4be7a ] [@ shutdownhang | ntdll.dll@0x4be7a ] [@ shutdownhang | ReleaseData(void*, unsigned int) ] [@ WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar:…
Tobias, it might be better to file these as new bugs and make this meta bug depend on them.
Flags: needinfo?(Tobias.Besemer)
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #74)
> Tobias, it might be better to file these as new bugs and make this meta bug
> depend on them.

OK, will do this in future ... Didn't it, because I was thinking that it wouldn't be helpful to have for every signature a new bug report, if I have nothing more then the signature and can't deliver a test case ...

If a dev don't need more then the sig, this is my topcrash:
(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #70)
> [@ ntdll.dll@0x4be7a ]
> 
> Very often signature while shutting down FF, but in plugin-container.exe, so
> it doesn't get auto send out to MCR.
> 
> One of the last examples:
> https://crash-stats.mozilla.com/report/index/2c2c8cf6-7f6d-43cd-88fe-
> 28d642151004
> 
> Crashing Thread:
> Frame 	Module 		Signature 		Source
> Ø 0 	ntdll.dll 	ntdll.dll@0x4be7a 	
> Ø 1 	kernelbase.dll 	kernelbase.dll@0x10db 	
> 2 	xul.dll 
> mozilla::jsipc::ObjectVariant::MaybeDestroy(mozilla::jsipc::ObjectVariant::
> Type)
> 						obj-firefox/ipc/ipdl/JavaScriptTypes.cpp
> 
> There are "only" 36 reports for Windows 7 in the last 28 days, but I guess
> they are all from me, because I send out at least 1 by day manual ... :D

Example for FF44.0a1:
https://crash-stats.mozilla.com/report/index/535e9b09-e0a8-4ef8-9123-293832151014
Flags: needinfo?(Tobias.Besemer)
Jim, are these reports useful? They seem like close variants of the same crash.
Flags: needinfo?(jmathies)
Crash Signature: [@ WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [@ WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [@ WaitForSingleObjectEx | CLayeredObject<CDeviceContextState>::CCo… → [@ WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [@ WaitForSingleObjectEx | CLayeredObject<CDeviceContextState>::CContainedObject::Release()] [@ WaitForSingleObjectEx | DrawingContext::FlushBatch(Fl…
Crash Signature: , short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ WaitForSingleObjectEx | choose_arena] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [… → , short*)] [@ WaitForSingleObjectEx | CD3DDeviceLevel1::EndDeviceStateChange()] [@ ntdll.dll@0x90e1a] [@ WaitForSingleObjectEx | arena_bin_malloc_hard | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)] [@ WaitForSingleObjectEx | arena_dalloc_sma…
No longer depends on: 1120785
No longer depends on: 1121629
Crash Signature: arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ] [@ WaitForSingleObjectEx | XPCWrappedNative::HasInterfaceNoQI(nsID const&) ] [@ WaitForSingleObjectEx | js::TypeOfValue(JS::Value const&) ] [@ shutdownhang | WaitForSingleObjectEx | WaitForSi… → arena_dalloc_small | mozilla::CondVar::Wait(unsigned int) ] [@ WaitForSingleObjectEx | XPCWrappedNative::HasInterfaceNoQI(nsID const&) ] [@ WaitForSingleObjectEx | js::TypeOfValue(JS::Value const&) ] [@ WaitForSingleObjectEx | SetEvent ] [@ WaitForSi…
Flags: needinfo?(jmathies)
This wasn't supposed to be a meta for shutdown hangs. I've removed all of those signatures. Shutdown hangs should be filed individually.
This is what I removed:

[@ shutdownhang | WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | mozilla::CondVar::Wait]
[@ shutdownhang | WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | PR_JoinThread | SmartCardMonitoringThread::~SmartCardMonitoringThread]
[@ shutdownhang | ReleaseData ]
[@ shutdownhang | ntdll.dll@0x4be7a ]
[@ shutdownhang | ReleaseData(void*, unsigned int) ]
[@ shutdownhang | WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)]
[@ shutdownhang | WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | PR_JoinThread | SmartCardMonitoringThread::~SmartCardMonitoringThread()]
No longer depends on: 1215016
(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #75)
> (In reply to Liz Henry (:lizzard) (needinfo? me) from comment #74)
> > Tobias, it might be better to file these as new bugs and make this meta bug
> > depend on them.
> 
> OK, will do this in future ... Didn't it, because I was thinking that it
> wouldn't be helpful to have for every signature a new bug report, if I have
> nothing more then the signature and can't deliver a test case ...
> 
> If a dev don't need more then the sig, this is my topcrash:
> (In reply to Tobias B. Besemer [:BesTo] (QA) from comment #70)
> > [@ ntdll.dll@0x4be7a ]
> > 
> > Very often signature while shutting down FF, but in plugin-container.exe, so
> > it doesn't get auto send out to MCR.
> > 
> > One of the last examples:
> > https://crash-stats.mozilla.com/report/index/2c2c8cf6-7f6d-43cd-88fe-
> > 28d642151004
> > 
> > Crashing Thread:
> > Frame 	Module 		Signature 		Source
> > Ø 0 	ntdll.dll 	ntdll.dll@0x4be7a 	
> > Ø 1 	kernelbase.dll 	kernelbase.dll@0x10db 	
> > 2 	xul.dll 
> > mozilla::jsipc::ObjectVariant::MaybeDestroy(mozilla::jsipc::ObjectVariant::
> > Type)
> > 						obj-firefox/ipc/ipdl/JavaScriptTypes.cpp
> > 
> > There are "only" 36 reports for Windows 7 in the last 28 days, but I guess
> > they are all from me, because I send out at least 1 by day manual ... :D
> 
> Example for FF44.0a1:
> https://crash-stats.mozilla.com/report/index/535e9b09-e0a8-4ef8-9123-
> 293832151014

Since this is specific to your install, you might try disabling individual addons to see if an addon you have installed is the cause.
(In reply to Jim Mathies [:jimm] from comment #77)
> This wasn't supposed to be a meta for shutdown hangs. I've removed all of
> those signatures. Shutdown hangs should be filed individually.

(In reply to Jim Mathies [:jimm] from comment #78)
> This is what I removed:
> 
> [@ shutdownhang | WaitForSingleObjectEx | WaitForSingleObject |
> PR_WaitCondVar | mozilla::CondVar::Wait]
> [@ shutdownhang | WaitForSingleObjectEx | WaitForSingleObject |
> PR_WaitCondVar | PR_JoinThread |
> SmartCardMonitoringThread::~SmartCardMonitoringThread]
> [@ shutdownhang | ReleaseData ]
> [@ shutdownhang | ntdll.dll@0x4be7a ]
> [@ shutdownhang | ReleaseData(void*, unsigned int) ]
> [@ shutdownhang | WaitForSingleObjectEx | WaitForSingleObject |
> PR_WaitCondVar | mozilla::CondVar::Wait(unsigned int)]
> [@ shutdownhang | WaitForSingleObjectEx | WaitForSingleObject |
> PR_WaitCondVar | PR_JoinThread |
> SmartCardMonitoringThread::~SmartCardMonitoringThread()]

I'm pretty sure, that at least some of this shutdownhangs are killhards.
As far as I can see & understand it, FF tries to shutdown the plugin-container.exe with e10s, can't remove all from the memory and ends in a killhard.

Some signatures also changed the names ...
E.g.: [@ ntdll.dll@0x4be7a ] -> [@ shutdownhang | ntdll.dll@0x4be7a ]

> No longer depends on: 1215016

(In reply to Jim Mathies [:jimm] from comment #79)
> (In reply to Tobias B. Besemer [:BesTo] (QA) from comment #75)
> > (In reply to Liz Henry (:lizzard) (needinfo? me) from comment #74)
> > > Tobias, it might be better to file these as new bugs and make this meta bug
> > > depend on them.
> > 
> > OK, will do this in future ... Didn't it, because I was thinking that it
> > wouldn't be helpful to have for every signature a new bug report, if I have
> > nothing more then the signature and can't deliver a test case ...
> > 
> > If a dev don't need more then the sig, this is my topcrash:
> > (In reply to Tobias B. Besemer [:BesTo] (QA) from comment #70)
> > > [@ ntdll.dll@0x4be7a ]
> > > 
> > > Very often signature while shutting down FF, but in plugin-container.exe, so
> > > it doesn't get auto send out to MCR.
> > > 
> > > One of the last examples:
> > > https://crash-stats.mozilla.com/report/index/2c2c8cf6-7f6d-43cd-88fe-
> > > 28d642151004
> > > 
> > > Crashing Thread:
> > > Frame 	Module 		Signature 		Source
> > > Ø 0 	ntdll.dll 	ntdll.dll@0x4be7a 	
> > > Ø 1 	kernelbase.dll 	kernelbase.dll@0x10db 	
> > > 2 	xul.dll 
> > > mozilla::jsipc::ObjectVariant::MaybeDestroy(mozilla::jsipc::ObjectVariant::
> > > Type)
> > > 						obj-firefox/ipc/ipdl/JavaScriptTypes.cpp
> > > 
> > > There are "only" 36 reports for Windows 7 in the last 28 days, but I guess
> > > they are all from me, because I send out at least 1 by day manual ... :D
> > 
> > Example for FF44.0a1:
> > https://crash-stats.mozilla.com/report/index/535e9b09-e0a8-4ef8-9123-
> > 293832151014
> 
> Since this is specific to your install, you might try disabling individual
> addons to see if an addon you have installed is the cause.

I normally don't run any addons ...
This is still the "FF can't make a good GC with FB"-Thing ...
... think you should ask for a feedback by someone who worked on the GC stuff before ...
Flags: needinfo?(jmathies)
(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #80)
> I'm pretty sure, that at least some of this shutdownhangs are killhards.
> As far as I can see & understand it, FF tries to shutdown the
> plugin-container.exe with e10s, can't remove all from the memory and ends in
> a killhard.

The same terminate helper function might be used but the type of problem is very different.

> > Since this is specific to your install, you might try disabling individual
> > addons to see if an addon you have installed is the cause.
> 
> I normally don't run any addons ...
> This is still the "FF can't make a good GC with FB"-Thing ...
> ... think you should ask for a feedback by someone who worked on the GC
> stuff before ...

Is there a specific bug on this issue filed that I can look at?
Flags: needinfo?(jmathies)
(In reply to Jim Mathies [:jimm] from comment #81)
> (In reply to Tobias B. Besemer [:BesTo] (QA) from comment #80)
> > I'm pretty sure, that at least some of this shutdownhangs are killhards.
> > As far as I can see & understand it, FF tries to shutdown the
> > plugin-container.exe with e10s, can't remove all from the memory and ends in
> > a killhard.
> The same terminate helper function might be used but the type of problem is
> very different.

OK, I see you changed the summary after comment #65 ...
> Summary: [meta] crashes in mozilla::CondVar::Wait(unsigned int) → [meta] e10s KillHard aborts on Windows
... and I added all with "mozilla::CondVar::Wait(unsigned int)", but this is the terminate helper function ... (?)


> > > Since this is specific to your install, you might try disabling individual
> > > addons to see if an addon you have installed is the cause.
> > I normally don't run any addons ...

Correction: "I normally don't run any plugins ..."
But my extensions are just some common like NoScript, FlashGet, Nightly Tester Tool, ...


> > This is still the "FF can't make a good GC with FB"-Thing ...
> > ... think you should ask for a feedback by someone who worked on the GC
> > stuff before ...
> Is there a specific bug on this issue filed that I can look at?

I mentioned that on some bugs in the last time ... and I know that someone (or more) must have worked on it, because it was getting a bit better ... but also one of the problems/bugs, where just someone did anything and I never get a feedback to it ... There are two more bugs open from me, where shortly after I filled them someone have changed anything ... never get a feedback ... bugs still open ... Yes, sometimes it is really fun to be a volunteer for Mozilla ... ;-) ... (Guess I will never get a "We have fixed it." one these bugs ... ;-) )

Look e.g. at: https://bugzilla.mozilla.org/show_bug.cgi?id=1131330#c37
[@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ]

Win7, FF44.0a1, 64bit:
https://crash-stats.mozilla.com/report/index/c74606ed-bad1-4e34-8207-827ea2151024

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	[thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}' (nsID const&, void**)
Crash Signature: mozilla::ipc::MessageChannel::ProcessPendingRequest ] → mozilla::ipc::MessageChannel::ProcessPendingRequest ] [@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ]
[@ je_arena_salloc | free_impl | nsXPCWrappedJS::cycleCollection::DeleteCycleCollectable ]

Win7, FF44.0a1, 64bit:
https://crash-stats.mozilla.com/report/index/8b1c6a45-f04b-4a0a-b905-6099e2151023

Crashing Thread
Frame 	Module 	Signature 	Source
0 	mozglue.dll 	je_arena_salloc 	memory/jemalloc/src/include/jemalloc/internal/arena.h
1 	mozglue.dll 	free_impl 	memory/build/replace_malloc.c
2 	xul.dll 	nsXPCWrappedJS::cycleCollection::DeleteCycleCollectable(void*) 	js/xpconnect/src/xpcprivate.h
3 	xul.dll 	SnowWhiteKiller::~SnowWhiteKiller() 	xpcom/base/nsCycleCollector.cpp
4 	xul.dll 	nsCycleCollector::FreeSnowWhite(bool) 	xpcom/base/nsCycleCollector.cpp
5 	xul.dll 	AsyncFreeSnowWhite::Run() 	js/xpconnect/src/XPCJSRuntime.cpp
6 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
7 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
8 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
9 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
10 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
11 	xul.dll 	nsBaseAppShell::Run() 	widget/nsBaseAppShell.cpp
12 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp
13 	xul.dll 	XRE_RunAppShell 	toolkit/xre/nsEmbedFunctions.cpp
14 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
15 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
16 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
17 	xul.dll 	XRE_InitChildProcess 	toolkit/xre/nsEmbedFunctions.cpp
18 	plugin-container.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp
19 	plugin-container.exe 	__tmainCRTStartup 	f:/dd/vctools/crt/crtw32/startup/crt0.c:255
Ø 20 	kernel32.dll 	kernel32.dll@0x159dc 	
Ø 21 	ntdll.dll 	ntdll.dll@0x2a630
Crash Signature: mozilla::ipc::MessageChannel::ProcessPendingRequest ] [@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ] → mozilla::ipc::MessageChannel::ProcessPendingRequest ] [@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ] [@ je_arena_salloc | free_impl | nsXPCWrappedJS::cycleCollection::DeleteCycleCollectable ]
[@ nsStyleSet::ClearSelectors ]

Win7, FF44.0a1, 64bit:
https://crash-stats.mozilla.com/report/index/1910dc78-951b-42cb-8df1-9f34b2151023

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	nsStyleSet::ClearSelectors() 	layout/style/nsStyleSet.cpp
1 	xul.dll 	nsStyleSet::GatherRuleProcessors(mozilla::SheetType) 	layout/style/nsStyleSet.cpp
2 	xul.dll 	PresShell::RemovePreferenceStyles() 	layout/base/nsPresShell.cpp
3 	xul.dll 	PresShell::Destroy() 	layout/base/nsPresShell.cpp
4 	xul.dll 	nsDocumentViewer::DestroyPresShell() 	layout/base/nsDocumentViewer.cpp
5 	xul.dll 	nsDocumentViewer::Destroy() 	layout/base/nsDocumentViewer.cpp
6 	xul.dll 	nsDocShell::Destroy() 	docshell/base/nsDocShell.cpp
7 	xul.dll 	nsWebBrowser::SetDocShell(nsIDocShell*) 	embedding/browser/nsWebBrowser.cpp
8 	xul.dll 	nsWebBrowser::InternalDestroy() 	embedding/browser/nsWebBrowser.cpp
9 	xul.dll 	nsWebBrowser::Destroy() 	embedding/browser/nsWebBrowser.cpp
10 	xul.dll 	mozilla::dom::TabChild::DestroyWindow() 	dom/ipc/TabChild.cpp
11 	xul.dll 	mozilla::dom::TabChild::RecvDestroy() 	dom/ipc/TabChild.cpp
12 	xul.dll 	mozilla::dom::PBrowserChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PBrowserChild.cpp
13 	xul.dll 	mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PContentChild.cpp
14 	xul.dll 	mozilla::ipc::MessageChannel::DispatchAsyncMessage(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
15 	xul.dll 	mozilla::ipc::MessageChannel::DispatchMessageW(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
16 	xul.dll 	mozilla::ipc::MessageChannel::OnMaybeDequeueOne() 	ipc/glue/MessageChannel.cpp
17 	xul.dll 	RunnableMethod<mozilla::ipc::MessageChannel, void ( mozilla::ipc::MessageChannel::*)(void), Tuple0>::Run() 	ipc/chromium/src/base/task.h
18 	xul.dll 	MessageLoop::DoWork() 	ipc/chromium/src/base/message_loop.cc
19 	xul.dll 	mozilla::ipc::DoWorkRunnable::Run() 	ipc/glue/MessagePump.cpp
20 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
21 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
22 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
23 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
24 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
25 	xul.dll 	nsBaseAppShell::Run() 	widget/nsBaseAppShell.cpp
26 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp
27 	xul.dll 	XRE_RunAppShell 	toolkit/xre/nsEmbedFunctions.cpp
28 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
29 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
30 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
31 	xul.dll 	XRE_InitChildProcess 	toolkit/xre/nsEmbedFunctions.cpp
32 	plugin-container.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp
33 	plugin-container.exe 	__tmainCRTStartup 	f:/dd/vctools/crt/crtw32/startup/crt0.c:255
Ø 34 	kernel32.dll 	kernel32.dll@0x159dc 	
Ø 35 	ntdll.dll 	ntdll.dll@0x2a630
Crash Signature: mozilla::ipc::MessageChannel::ProcessPendingRequest ] [@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ] [@ je_arena_salloc | free_impl | nsXPCWrappedJS::cycleCollection::DeleteCycleCollectable ] → mozilla::ipc::MessageChannel::ProcessPendingRequest ] [@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ] [@ je_arena_salloc | free_impl | nsXPCWrappedJS::cycleCollection::DeleteCycleCollectable ] [@ nsStyleSet::ClearSelectors ]
[@ XPCStringConvert::FinalizeDOMString ]

Win7, FF44.0a1, 64bit:
https://crash-stats.mozilla.com/report/index/dd0714b1-c0c8-448f-9c4e-d788d2151023

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	XPCStringConvert::FinalizeDOMString(JSStringFinalizer const*, wchar_t*) 	js/xpconnect/src/XPCString.cpp
1 		@0x80000000008ce704
Crash Signature: ] → ] [@ XPCStringConvert::FinalizeDOMString ]
[@ nsIContent::GetClasses ]

Win7, FF44.0a1, 64bit:
https://crash-stats.mozilla.com/report/index/d6e70a3a-8fbf-4741-9749-b81452151023

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	nsIContent::GetClasses() 	dom/base/nsIContent.h
1 		@0x0 	
2 		@0x4 	
3 	xul.dll 	js::jit::AssemblerX86Shared::jmpSrc(js::jit::Label*) 	js/src/jit/x86-shared/Assembler-x86-shared.h
4 	xul.dll 	js::GetProperty(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	js/src/vm/NativeObject.h
5 	xul.dll 	mozilla::dom::GetPropertyOnPrototype(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, bool*, JS::MutableHandle<JS::Value>) 	dom/bindings/BindingUtils.cpp
6 	xul.dll 	mozilla::dom::HTMLCollectionBinding::DOMProxyHandler::get(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	obj-firefox/dom/bindings/HTMLCollectionBinding.cpp
7 	xul.dll 	js::Proxy::get(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	js/src/proxy/Proxy.cpp
8 	xul.dll 	js::GetProperty(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>) 	js/src/vm/NativeObject.h
9 	xul.dll 	js::GetLengthProperty(JSContext*, JS::Handle<JSObject*>, unsigned int*) 	js/src/jsarray.cpp
10 	xul.dll 	js::fun_apply(JSContext*, unsigned int, JS::Value*) 	js/src/jsfun.cpp
11 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
12 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
13 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
14 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
15 	xul.dll 	js::fun_apply(JSContext*, unsigned int, JS::Value*) 	js/src/jsfun.cpp
16 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
17 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
18 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
19 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
20 	xul.dll 	js::fun_apply(JSContext*, unsigned int, JS::Value*) 	js/src/jsfun.cpp
21 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
22 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
23 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
24 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
25 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
26 	xul.dll 	mozilla::dom::EventListener::HandleEvent(JSContext*, JS::Handle<JS::Value>, mozilla::dom::Event&, mozilla::ErrorResult&) 	obj-firefox/dom/bindings/EventListenerBinding.cpp
27 	xul.dll 	mozilla::EventListenerManager::HandleEventInternal(nsPresContext*, mozilla::WidgetEvent*, nsIDOMEvent**, mozilla::dom::EventTarget*, nsEventStatus*) 	dom/events/EventListenerManager.cpp
28 	xul.dll 	mozilla::EventTargetChainItem::HandleEventTargetChain(nsTArray<mozilla::EventTargetChainItem>&, mozilla::EventChainPostVisitor&, mozilla::EventDispatchingCallback*, mozilla::ELMCreationDetector&) 	dom/events/EventDispatcher.cpp
29 	xul.dll 	mozilla::EventDispatcher::Dispatch(nsISupports*, nsPresContext*, mozilla::WidgetEvent*, nsIDOMEvent*, nsEventStatus*, mozilla::EventDispatchingCallback*, nsTArray<mozilla::dom::EventTarget*>*) 	dom/events/EventDispatcher.cpp
30 	xul.dll 	mozilla::EventDispatcher::DispatchDOMEvent(nsISupports*, mozilla::WidgetEvent*, nsIDOMEvent*, nsPresContext*, nsEventStatus*) 	dom/events/EventDispatcher.cpp
31 	xul.dll 	nsDocument::DispatchPageTransition(mozilla::dom::EventTarget*, nsAString_internal const&, bool) 	dom/base/nsDocument.cpp
32 	xul.dll 	nsDocument::OnPageHide(bool, mozilla::dom::EventTarget*) 	dom/base/nsDocument.cpp
33 	xul.dll 	nsDocumentViewer::PageHide(bool) 	layout/base/nsDocumentViewer.cpp
34 	xul.dll 	nsDocShell::FirePageHideNotification(bool) 	docshell/base/nsDocShell.cpp
35 	xul.dll 	nsDocShell::Destroy() 	docshell/base/nsDocShell.cpp
36 	xul.dll 	nsWebBrowser::SetDocShell(nsIDocShell*) 	embedding/browser/nsWebBrowser.cpp
37 	xul.dll 	nsWebBrowser::InternalDestroy() 	embedding/browser/nsWebBrowser.cpp
38 	xul.dll 	nsWebBrowser::Destroy() 	embedding/browser/nsWebBrowser.cpp
39 	xul.dll 	mozilla::dom::TabChild::DestroyWindow() 	dom/ipc/TabChild.cpp
40 	xul.dll 	mozilla::dom::TabChild::RecvDestroy() 	dom/ipc/TabChild.cpp
41 	xul.dll 	mozilla::dom::PBrowserChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PBrowserChild.cpp
42 	xul.dll 	mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PContentChild.cpp
43 	xul.dll 	mozilla::ipc::MessageChannel::DispatchAsyncMessage(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
44 	xul.dll 	mozilla::ipc::MessageChannel::DispatchMessageW(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
45 	xul.dll 	mozilla::ipc::MessageChannel::OnMaybeDequeueOne() 	ipc/glue/MessageChannel.cpp
46 	xul.dll 	RunnableMethod<mozilla::ipc::MessageChannel, void ( mozilla::ipc::MessageChannel::*)(void), Tuple0>::Run() 	ipc/chromium/src/base/task.h
47 	xul.dll 	MessageLoop::DoWork() 	ipc/chromium/src/base/message_loop.cc
48 	xul.dll 	mozilla::ipc::DoWorkRunnable::Run() 	ipc/glue/MessagePump.cpp
49 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
50 	xul.dll 	XPTC__InvokebyIndex 	xpcom/reflect/xptcall/md/win32/xptcinvoke_asm_x86_64.asm
51 		@0x9 	
52 	xul.dll 	nsXPConnect::XPConnect() 	js/xpconnect/src/xpcprivate.h
53 	xul.dll 	XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode) 	js/xpconnect/src/XPCWrappedNative.cpp
54 	xul.dll 	XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) 	js/xpconnect/src/XPCWrappedNativeJSOps.cpp
55 		@0x3a3aad23bcb
Crash Signature: ] [@ XPCStringConvert::FinalizeDOMString ] → ] [@ XPCStringConvert::FinalizeDOMString ] [@ nsIContent::GetClasses ]
[@ nsINode::QuerySelectorAll ]

Win7, FF44.0a1, 64bit:
https://crash-stats.mozilla.com/report/index/012e509d-5e3d-48a7-8181-2d9712151019

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	nsINode::QuerySelectorAll(nsAString_internal const&, mozilla::ErrorResult&) 	dom/base/nsINode.cpp
1 	xul.dll 	mozilla::dom::DocumentBinding::querySelectorAll 	obj-firefox/dom/bindings/DocumentBinding.cpp
2 	xul.dll 	mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) 	dom/bindings/BindingUtils.cpp
3 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
4 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
5 	xul.dll 	js::jit::InvokeFunction(JSContext*, JS::Handle<JSObject*>, bool, unsigned int, JS::Value*, JS::MutableHandle<JS::Value>) 	js/src/jit/VMFunctions.cpp
6 		@0x304c0128dcc 	
7 	xul.dll 	mozilla::dom::WindowBinding::get_document 	obj-firefox/dom/bindings/WindowBinding.cpp
8 		@0x40d4ec3f
Crash Signature: ] [@ XPCStringConvert::FinalizeDOMString ] [@ nsIContent::GetClasses ] → ] [@ XPCStringConvert::FinalizeDOMString ] [@ nsIContent::GetClasses ] [@ nsINode::QuerySelectorAll ]
[@ GetNameOperation ]

Win7, FF44.0a1, 64bit:
https://crash-stats.mozilla.com/report/index/bdfe01e9-bab7-4f88-965a-31e9e2151027

Crashing Thread
Frame 	Module 	Signature 	Source
0 	xul.dll 	GetNameOperation 	js/src/vm/Interpreter.cpp
1 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
2 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
3 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
4 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
5 	xul.dll 	js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/src/proxy/CrossCompartmentWrapper.cpp
6 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
7 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
8 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
9 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
10 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
11 	xul.dll 	js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/src/proxy/CrossCompartmentWrapper.cpp
12 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
13 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
14 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
15 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
16 	xul.dll 	JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) 	js/src/jsapi.cpp
17 	xul.dll 	nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS*, unsigned short, XPTMethodDescriptor const*, nsXPTCMiniVariant*) 	js/xpconnect/src/XPCWrappedJSClass.cpp
18 	xul.dll 	nsXPCWrappedJS::CallMethod(unsigned short, XPTMethodDescriptor const*, nsXPTCMiniVariant*) 	js/xpconnect/src/XPCWrappedJS.cpp
19 	xul.dll 	PrepareAndDispatch 	xpcom/reflect/xptcall/md/win32/xptcstubs_x86_64.cpp
20 	xul.dll 	SharedStub 	xpcom/reflect/xptcall/md/win32/xptcstubs_asm_x86_64.asm
21 	xul.dll 	nsObserverList::NotifyObservers(nsISupports*, char const*, wchar_t const*) 	xpcom/ds/nsObserverList.cpp
22 	xul.dll 	nsObserverService::NotifyObservers(nsISupports*, char const*, wchar_t const*) 	xpcom/ds/nsObserverService.cpp
23 	xul.dll 	XPTC__InvokebyIndex 	xpcom/reflect/xptcall/md/win32/xptcinvoke_asm_x86_64.asm
24 		@0x3696145f 	
25 	xul.dll 	XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode) 	js/xpconnect/src/XPCWrappedNative.cpp
26 	xul.dll 	XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) 	js/xpconnect/src/XPCWrappedNativeJSOps.cpp
27 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
28 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
29 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
30 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
31 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
32 	xul.dll 	js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/src/proxy/CrossCompartmentWrapper.cpp
33 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
34 	xul.dll 	Interpret 	js/src/vm/Interpreter.cpp
35 	xul.dll 	js::RunScript(JSContext*, js::RunState&) 	js/src/vm/Interpreter.cpp
36 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
37 	xul.dll 	js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) 	js/src/vm/Interpreter.cpp
38 	xul.dll 	js::CrossCompartmentWrapper::call(JSContext*, JS::Handle<JSObject*>, JS::CallArgs const&) 	js/src/proxy/CrossCompartmentWrapper.cpp
39 	xul.dll 	js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) 	js/src/vm/Interpreter.cpp
40 	xul.dll 	JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) 	js/src/jsapi.cpp
41 	xul.dll 	nsFrameMessageManager::ReceiveMessage(nsISupports*, nsIFrameLoader*, bool, nsAString_internal const&, bool, mozilla::dom::ipc::StructuredCloneData*, mozilla::jsipc::CpowHolder*, nsIPrincipal*, nsTArray<mozilla::dom::ipc::StructuredCloneData>*) 	dom/base/nsFrameMessageManager.cpp
42 	xul.dll 	nsFrameMessageManager::ReceiveMessage(nsISupports*, nsIFrameLoader*, nsAString_internal const&, bool, mozilla::dom::ipc::StructuredCloneData*, mozilla::jsipc::CpowHolder*, nsIPrincipal*, nsTArray<mozilla::dom::ipc::StructuredCloneData>*) 	dom/base/nsFrameMessageManager.cpp
43 	xul.dll 	mozilla::dom::ContentChild::RecvAsyncMessage(nsString const&, mozilla::dom::ClonedMessageData const&, nsTArray<mozilla::jsipc::CpowEntry>&&, IPC::Principal const&) 	dom/ipc/ContentChild.cpp
44 	xul.dll 	mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) 	obj-firefox/ipc/ipdl/PContentChild.cpp
45 	xul.dll 	mozilla::ipc::MessageChannel::DispatchAsyncMessage(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
46 	xul.dll 	mozilla::ipc::MessageChannel::DispatchMessageW(IPC::Message const&) 	ipc/glue/MessageChannel.cpp
47 	xul.dll 	mozilla::ipc::MessageChannel::OnMaybeDequeueOne() 	ipc/glue/MessageChannel.cpp
48 	xul.dll 	RunnableMethod<mozilla::ipc::MessageChannel, void ( mozilla::ipc::MessageChannel::*)(void), Tuple0>::Run() 	ipc/chromium/src/base/task.h
49 	xul.dll 	MessageLoop::DoWork() 	ipc/chromium/src/base/message_loop.cc
50 	xul.dll 	mozilla::ipc::DoWorkRunnable::Run() 	ipc/glue/MessagePump.cpp
51 	xul.dll 	nsThread::ProcessNextEvent(bool, bool*) 	xpcom/threads/nsThread.cpp
52 	xul.dll 	mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
53 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
54 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
55 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
56 	xul.dll 	nsBaseAppShell::Run() 	widget/nsBaseAppShell.cpp
57 	xul.dll 	nsAppShell::Run() 	widget/windows/nsAppShell.cpp
58 	xul.dll 	XRE_RunAppShell 	toolkit/xre/nsEmbedFunctions.cpp
59 	xul.dll 	mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) 	ipc/glue/MessagePump.cpp
60 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
61 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
62 	xul.dll 	XRE_InitChildProcess 	toolkit/xre/nsEmbedFunctions.cpp
63 	plugin-container.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp
64 	plugin-container.exe 	__tmainCRTStartup 	f:/dd/vctools/crt/crtw32/startup/crt0.c:255
Ø 65 	kernel32.dll 	kernel32.dll@0x159dc 	
Ø 66 	ntdll.dll 	ntdll.dll@0x2a630
Crash Signature: ] [@ XPCStringConvert::FinalizeDOMString ] [@ nsIContent::GetClasses ] [@ nsINode::QuerySelectorAll ] → ] [@ XPCStringConvert::FinalizeDOMString ] [@ nsIContent::GetClasses ] [@ nsINode::QuerySelectorAll ] [@ GetNameOperation ]
Please stop adding shutdown hang signatures to this bug. None of these are properly filed here:

[@ nsINode::QuerySelectorAll ]
[@ nsIContent::GetClasses ]
[@ XPCStringConvert::FinalizeDOMString ]
[@ nsStyleSet::ClearSelectors ]
[@ je_arena_salloc | free_impl | nsXPCWrappedJS::cycleCollection::DeleteCycleCollectable ]
[@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ]

These are all shutdown hangs. You can tell this by looking at the ipc_channel_error under meta data, which will have 'ShutDownKill' as the text.
Crash Signature: mozilla::ipc::MessageChannel::ProcessPendingRequest ] [@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ] [@ je_arena_salloc | free_impl | nsXPCWrappedJS::cycleCollection::DeleteCycleCollectable ] [@ nsStyleSet::ClearSelectors … → mozilla::ipc::MessageChannel::ProcessPendingRequest ]
Whiteboard: [KillHard Windows signature] → [KillHard Windows signature][do not file ShutdownKill signatures here]
Depends on: 719114
(In reply to Jim Mathies [:jimm] from comment #90)
> Please stop adding shutdown hang signatures to this bug. None of these are
> properly filed here:

OK, seems I don't get it how you define "KillHard".
If the container crashes while browsing?
(In reply to Jim Mathies [:jimm] from comment #90)
> These are all shutdown hangs. You can tell this by looking at the
> ipc_channel_error under meta data, which will have 'ShutDownKill' as the
> text.

Do we have another bug on those?
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #93)
> (In reply to Jim Mathies [:jimm] from comment #90)
> > These are all shutdown hangs. You can tell this by looking at the
> > ipc_channel_error under meta data, which will have 'ShutDownKill' as the
> > text.
> 
> Do we have another bug on those?

I don't think we have a tracker, you could create one with individual shutdown hangs filed under it. We can expect that shutdown hangs are individual issues vs. a generic problem. This bug keeps track of a a specific class of KillHard aborts related to ipc protocol errors that occur while the browser is running.
BTW, in terms of phrasing, it's very confusing to me when we call anything a "shutdown hang" that does not have a "shutdownhang | …" signature. ;-)
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #95)
> BTW, in terms of phrasing, it's very confusing to me when we call anything a
> "shutdown hang" that does not have a "shutdownhang | …" signature. ;-)

I'm not sure where that comes from. Crashes with that added to the signature all appear to have a crash reason annotation that looks like -

MozCrashReason 	MOZ_CRASH(Shutdown too long, probably frozen, causing a crash.)

I don't think the 'shutdownhang' text comes from firefox, looks like it's added by socorro post processing. Whatever is doing it doesn't pick up the KillHard shutdown hangs. But we could probably fix that. I'll file a bug.
(In reply to Jim Mathies [:jimm] from comment #96)
> I don't think the 'shutdownhang' text comes from firefox, looks like it's
> added by socorro post processing. Whatever is doing it doesn't pick up the
> KillHard shutdown hangs. But we could probably fix that. I'll file a bug.

Actually I take that back, looking at these for example, the ipc related kill hards appear to have shutdownhang in the sig.

[@ nsINode::QuerySelectorAll ]
[@ nsIContent::GetClasses ]
[@ XPCStringConvert::FinalizeDOMString ]
[@ nsStyleSet::ClearSelectors ]
[@ je_arena_salloc | free_impl | nsXPCWrappedJS::cycleCollection::DeleteCycleCollectable ]
[@ [thunk]:mozilla::dom::HTMLImageElement::QueryInterface`adjustor{136}'' ]
We add "shutdownhang" (and generate the signature from Thread 0 instead of the crashing thread) when we kill the process after 60s of inactivity on shutdown.
No longer depends on: 719114
Alias: KillHards-Win
Blocks: 899758
No longer blocks: 1118517, 1124064
See Also: → 1118517, 1124064
See Also: → 1180160
See Also: shutdownkill
For documenting the old split of the bug -> Link to the removed sigs
See Also: → shutdownkill
¡Hola Jim!

FWIW, on Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0 ID:20160126030244 CSet: c0ba5835ca489d15f8f170d5deb01f8dad92709a the about:sessionrestore tab crashes at times here and causes severe data loss.

Is this crash this bug or shall I file a separate one?

Might it be that https://bugzilla.mozilla.org/show_bug.cgi?id=1217904 somehow morphed into this crash?

¡Gracias!
Alex

Report ID 	Date Submitted
bp-fda41717-6272-4981-969e-f94372160127
	27/01/2016	12:47
Flags: needinfo?(jmathies)
> Report ID 	Date Submitted
> bp-fda41717-6272-4981-969e-f94372160127
> 	27/01/2016	12:47

This is a ShutDownKill abort. Possibly fits under bug 1219672.
Flags: needinfo?(jmathies)
Alias: KillHards-Win → killhard-win
Depends on: 1268881
(In reply to Jim Mathies [:jimm] from comment #102)
> > Report ID 	Date Submitted
> > bp-fda41717-6272-4981-969e-f94372160127
> > 	27/01/2016	12:47
> 
> This is a ShutDownKill abort. Possibly fits under bug 1219672.

Sig is:
[@ WaitForSingleObjectEx | PR_WaitCondVar | mozilla::CondVar::Wait ]
...and this sig is added here, too.
Anyway: ipc_channel_error = ShutDownKill
...and I add the sig to bug 1219672, too.
Priority: -- → P2
Closing because no crash reported since 12 weeks.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
Closing because no crash reported since 12 weeks.
You need to log in before you can comment on or make changes to this bug.