693451 - [10.7] Empty plugin side crash reports

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, defect)

Description

[Marcia Knous [:marcia]]

Seen while looking at chofmann's Mac report. This hang consistently shows in the top 5 10.7 specific issues in chofmann's report.

https://crash-stats.mozilla.com/report/list?signature=hang%20|%20__psynch_mutexwait

https://crash-stats.mozilla.com/report/index/b5653167-f5bf-45f2-8813-8fa6a2111010

Frame 	Module 	Signature [Expand] 	Source
0 	libsystem_kernel.dylib 	__psynch_mutexwait 	
1 	libsystem_c.dylib 	pthread_mutex_lock 	
2 	XUL 	google_breakpad::ExceptionHandler::WriteMinidump 	toolkit/crashreporter/google-breakpad/src/client/mac/handler/exception_handler.cc:298
3 	XUL 	CrashReporter::CreatePairedMinidumps 	toolkit/crashreporter/nsExceptionHandler.cpp:2098
4 	XUL 	mozilla::plugins::PluginModuleParent::ShouldContinueFromReplyTimeout 	dom/plugins/ipc/PluginModuleParent.cpp:259
5 	XUL 	mozilla::plugins::PPluginModuleParent::OnReplyTimeout 	obj-firefox/x86_64/ipc/ipdl/PPluginModuleParent.cpp:1125
6 	XUL 	mozilla::ipc::SyncChannel::ShouldContinueFromTimeout 	ipc/glue/SyncChannel.cpp:264
7 	XUL 	mozilla::ipc::RPCChannel::Call 	ipc/glue/RPCChannel.cpp:210
8 	XUL 	mozilla::plugins::PPluginInstanceParent::CallNPP_HandleEvent 	obj-firefox/x86_64/ipc/ipdl/PPluginInstanceParent.cpp:496
9 	XUL 	mozilla::plugins::PluginInstanceParent::NPP_HandleEvent 	dom/plugins/ipc/PluginInstanceParent.cpp:1270
10 	XUL 	mozilla::plugins::PluginModuleParent::NPP_HandleEvent 	dom/plugins/ipc/PluginModuleParent.cpp:546
11 	XUL 	nsNPAPIPluginInstance::HandleEvent 	dom/plugins/base/nsNPAPIPluginInstance.cpp:576
12 	XUL 	nsPluginInstanceOwner::ProcessEvent 	dom/plugins/base/nsPluginInstanceOwner.cpp:2060
13 	XUL 	nsPluginInstanceOwner::DispatchMouseToPlugin 	dom/plugins/base/nsPluginInstanceOwner.cpp:1819
14 	XUL 	nsPluginInstanceOwner::HandleEvent 	dom/plugins/base/nsPluginInstanceOwner.cpp:1864
15 	XUL 	nsEventListenerManager::HandleEventSubType 	content/events/src/nsEventListenerManager.cpp:865
16 	XUL 	nsEventListenerManager::HandleEventInternal 	content/events/src/nsEventListenerManager.cpp:919
17 	XUL 	nsEventTargetChainItem::HandleEventTargetChain 	content/events/src/nsEventListenerManager.h:147
18 	XUL 	nsEventDispatcher::Dispatch 	content/events/src/nsEventDispatcher.cpp:672
19 	XUL 	PresShell::HandleEventInternal 	layout/base/nsPresShell.cpp:7087
20 	XUL 	PresShell::HandlePositionedEvent 	layout/base/nsPresShell.cpp:6920
21 	XUL 	PresShell::HandleEvent 	layout/base/nsPresShell.cpp:6755
22 	XUL 	PresShell::HandleEvent 	layout/base/nsPresShell.cpp:6510
23 	XUL 	nsViewManager::DispatchEvent 	view/src/nsViewManager.cpp:1029
24 	XUL 	HandleEvent 	view/src/nsView.cpp:159
25 	XUL 	nsChildView::DispatchEvent 	widget/src/cocoa/nsChildView.mm:1505
26 	XUL 	nsChildView::DispatchWindowEvent 	widget/src/cocoa/nsChildView.mm:1515
27 	XUL 	-[ChildView mouseUp:] 	widget/src/cocoa/nsChildView.mm:3161
28 	AppKit 	-[NSWindow sendEvent:] 	
29 	XUL 	-[ToolbarWindow sendEvent:] 	widget/src/cocoa/nsCocoaWindow.mm:2396
30 	AppKit 	-[NSApplication sendEvent:] 	
31 	XUL 	-[GeckoNSApplication sendEvent:] 	widget/src/cocoa/nsAppShell.mm:192
32 	AppKit 	-[NSApplication run] 	
33 	XUL 	nsAppShell::Run 	widget/src/cocoa/nsAppShell.mm:771
34 	XUL 	nsAppStartup::Run 	toolkit/components/startup/nsAppStartup.cpp:224
35 	XUL 	XRE_main 	toolkit/xre/nsAppRunner.cpp:3544
36 	firefox 	main 	browser/app/nsBrowserApp.cpp:198
37 	firefox 	firefox@0xac3

Comment 1

[Marcia Knous [:marcia]]

Add on correlations from 7.0, but don't really show enough volume:

7% (2/27) vs.   0% (7/1858) [e-mail] (Alexa Sparky, https://addons.mozilla.org/addon/5362)
      7% (2/27) vs.   1% (15/1858) [e-mail] (Ant.com Toolbar - Tube Downloader, https://addons.mozilla.org/addon/8174)
      7% (2/27) vs.   1% (19/1858) {a3a5c777-f583-4fef-9380-ab4add1bc2a8}
      7% (2/27) vs.   2% (36/1858) {19503e42-ca3c-4c27-b1e2-9cdb2170ee34} (FlashGot, https://addons.mozilla.org/addon/220)

Comment 2

[Steven Michaud [:smichaud] (Retired)]

Note that this is a "hang" while processing a minidump for a crash.

Comment 3

[(not currently active) Ted Mielczarek]

No, that's not true. The stack you see in the parent is what it always is, the parent writing out its half of the minidump pair. The child minidump is usually more interesting, but in the report from comment 0, the matching child dump is:
https://crash-stats.mozilla.com/report/index/d2d9802a-f9db-46a8-9297-fcf4b2111010

Which is not at all useful. It looks like we're missing a module list. Do either of you know if we get useful reports on 10.7 at all?

Comment 4

[(not currently active) Ted Mielczarek]

I should say "useful plugin reports", because clearly the browser process report is fine.

Comment 5

[Steven Michaud [:smichaud] (Retired)]

>> Note that this is a "hang" while processing a minidump for a crash.
>
> No, that's not true.

You're saying the parent process isn't hung, but just behaving as it normally would when the plugin crashes?

If so, then this bug's signature is spurious, and so are most (if not all) of the browser-side signatures of our "top hangs" that are plugin related.  In other words, these signatures don't correspond to hangs.

They're still useful, though, if they can be paired up with stack traces from plugin crashes/hangs.

Comment 6

[Benjamin Smedberg]

Steven, I think you misunderstand plugin hang reports. Every plugin hang reports comes with two crash report IDs: we take a snapshot of both the plugin process and the browser process at the time of the hang, and they both end up with a "hang | ..." signature. This is because the cause of the hang may be in either process, so we need often to correlate them both to diagnose the problem.

Comment 7

[Steven Michaud [:smichaud] (Retired)]

I understood most of this.  

> and they both end up with a "hang | ..." signature. This is because
> the cause of the hang may be in either process, so we need often to
> correlate them both to diagnose the problem.

I didn't understand this part ... and is it completely true? :-)

In other words,	might some (or even all) plugin	crashes	end up with a
"hang | ..." in	their signature, at least on the browser-process side?

In any case, though, it's (apparently) a mistake to say	that this
bug's signature corresponds to a hang on the browser side.

Comment 8

[Benjamin Smedberg]

Currently we don't report any browser-only hangs. We only report plugin hangs. That is going to change soon.

Real plugin crashes *never* have a "hang | " in their summary. Note that plugin hangs appear to the user as a crash (we wait for 45 seconds and then kill the plugin, but the user gets the same plugin frowny-face).

Comment 9

[Steven Michaud [:smichaud] (Retired)]

(In reply to comment #3)

> It looks like we're missing a module list. Do either of you know if
> we get useful reports on 10.7 at all?

Looks like we aren't, on any version of OS X 10.7:

bp-74303d8f-3775-43c6-8a8d-063b32111009
bp-b741539e-ea83-4045-a6bb-e474c2111011
bp-9d69c3fc-260b-43ce-9541-684fd2111006
bp-d85dee60-65cb-4568-b12e-c90122111010

(In reply to comment #8)

Thanks!

Comment 10

[(not currently active) Ted Mielczarek]

Looks like Google already filed this issue upstream:
http://code.google.com/p/google-breakpad/issues/detail?id=447

Comment 11

[Cameron McCormack (:heycam)]

Is this the same issue that results in minidumps not being written out properly on Lion that I see in this log?

https://tbpl.mozilla.org/php/getParsedLog.php?id=7738963&tree=Firefox#error0

Assertion failed: (count && length), function AllocateObjectAndArray, file /builds/slave/m-cen-osx64-dbg/build/toolkit/crashreporter/google-breakpad/src/client/mac/handler/../../../client/minidump_file_writer-inl.h, line 66.

Comment 12

[Scoobidiver (away)]

The issue here is that there are empty plugin side crash reports on Mac OS X 10.7.

Comment 13

[(not currently active) Ted Mielczarek]

This appears to have been fixed upstream in Breakpad, although I don't know specifically what fixed it. We should be able to update our Breakpad snapshot to fix this.

Comment 14

[Scoobidiver (away)]

It's fixed in versions where hangs are no longer available in crash stats!