Last Comment Bug 422308 - Breakpad does not catch crash when Flash Player 9 plugin is active (even crashes unrelated to Flash)
: Breakpad does not catch crash when Flash Player 9 plugin is active (even cras...
Status: VERIFIED WORKSFORME
: relnote
Product: Core
Classification: Components
Component: Plug-ins (show other bugs)
: Trunk
: x86 Windows XP
: -- normal with 15 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
: 386343 422966 434525 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2008-03-11 20:36 PDT by Brian Polidoro
Modified: 2010-01-28 07:40 PST (History)
61 users (show)
ted: blocking1.9-
hskupin: in‑litmus?
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
(300.49 KB, application/octet-stream)
2009-04-23 08:39 PDT, nichole
no flags Details

Description Brian Polidoro 2008-03-11 20:36:20 PDT
Breakpad wasn't catching my crashes so I investigated.  From what I've found having yahoo mail in the sessiosn restore keeps breakpad from working.

STR:
1. Create a new profile.
2. Open yahoo mail.  I've only tested the new version so far.
3. Open the testcase from Bug 410198 in a second tab as a means to crash. 
https://bugzilla.mozilla.org/attachment.cgi?id=294858
Hopefully you don't crash when it loads.  Otherwise you can control when you crash because then the crash occurs when you resize the window.
4. Close Minefield to get those sites into session restore.
5. Start Minefield and restore the session.
6. Use the testcase to crash by resizing the window.

Actual Result:
Microsoft's crash tool appears instead of breakpad.
Comment 1 Brian Polidoro 2008-03-11 20:41:47 PDT
It also happens with Yahoo Mail Classic.
Comment 2 Brian Polidoro 2008-03-11 20:49:31 PDT
Oh yeah I need a step in there to say turn on session restore after making the new profile.
Comment 3 Brian Polidoro 2008-03-11 21:02:01 PDT
Well turning on session restore isn't necessary because the session restore file is still generated to recover from crashes.

But more importantly is that Flash seems to be in play here too.  I tried a new profile and even that profile in safe mode.  But I noticed plugins are still enabled in safe mode.  So from disabling plugins I see that without flash enabled the problem does not happen. I have Flash 9.0 r115.
Comment 4 Brian Polidoro 2008-03-11 21:04:38 PDT
> So from disabling plugins I see that without flash enabled the problem does not happen.

That grammar may not be clear.  The problem does not occur with flash disabled.
Comment 5 timeless 2008-03-12 00:16:25 PDT
http://developer.mozilla.org/en/docs/How_to_get_a_stacktrace_with_WinDbg

you should be able to get a stack trace for the crash, and also later w/ ted's help figure out if breakpad isn't initializing and why (unless the stack trace shows that breakpad wouldn't catch it).
Comment 6 Ted Mielczarek [:ted.mielczarek] 2008-03-12 03:30:28 PDT
I am definitely interested in this. Brian: thanks for taking the time to come up with STR! Hopefully I'll have time to look at this today.
Comment 7 Ted Mielczarek [:ted.mielczarek] 2008-03-12 05:11:35 PDT
To clarify, you need to login to Yahoo Mail in step 2. With that, I can reproduce.

Comment 8 Ted Mielczarek [:ted.mielczarek] 2008-03-12 05:17:37 PDT
The crash is just the same crash as bug 410198 (since we're using the testcase there). I can actually reproduce this using my "Crash Me Now!" extension too. More investigation warranted...
Comment 9 Ted Mielczarek [:ted.mielczarek] 2008-03-12 05:18:09 PDT
http://ted.mielczarek.org/code/mozilla/crashme.xpi btw
Comment 10 Ted Mielczarek [:ted.mielczarek] 2008-03-12 05:27:32 PDT
In fact, I can reproduce this by doing (with Crash me now installed):

1) Load data:text/html,<embed type="application/x-shockwave-flash" src="http://www.adobe.com//shockwave/welcome/flash.swf" height="120" width="300">
2) Close (and save session)
3) Restart
4) Tools -> Crash me now

Breakpad doesn't trigger.
Comment 11 Ted Mielczarek [:ted.mielczarek] 2008-03-12 05:39:22 PDT
Ok, so I can reproduce this if I load that data URI *at all* and crash, no session restore needed. Wow. Are we broken if you have flash running in any tab? That's pretty horrible. This is the flash plugin screwing us, apparently:

0012ef68 300cb378 kernel32!SetUnhandledExceptionFilter
WARNING: Stack unwind information not available. Following frames may be wrong.
0012ef88 300be168 NPSWF32!native_ShockwaveFlash_TCallLabel+0x741e
0012efb4 60af3f86 NPSWF32!NP_Initialize+0x3c
0012f100 60afb8dd xul!Create4xPlugin(class nsIServiceManagerObsolete * aServiceManager = 0x0012f164, class nsPluginTag * aPluginTag = 0x022654a0, class nsIPlugin ** aOut4xPlugnin = 0x0012f164)+0x1f4 [e:\builds\tinderbox\fx-trunk\winnt_5.2_depend\mozilla\modules\plugin\base\src\nspluginhostimpl.cpp @ 4717]
Comment 12 Ted Mielczarek [:ted.mielczarek] 2008-03-12 05:43:22 PDT
It looks like the flash plugin calls SetUnhandledExceptionFilter
 any time the plugin code calls into it. This is not good for our crash reporting.
Comment 13 Radek 'sysKin' Czyz 2008-03-12 07:43:25 PDT
Come to think of it, we *could* have noticed the complete lack of NPSWF32.dll crashes in breakpad crash reports :)

I see no QuickTime.qts crashes either (despite being #6 on branch).
Comment 14 Radek 'sysKin' Czyz 2008-03-12 07:46:06 PDT
Um apologies, QuickTime.qts does happen (just not on beta4 yet).
Comment 15 timeless 2008-03-12 08:04:06 PDT
fwiw, chaining is very risky, the testcase is:

plugin 1 sets up a chain to our handler
plugin 2 sets up a chain to current handler (plugin 1)
plugin 1 unloads but can't clear the handler because it isn't plugin 1's
exception happens and reaches plugin 2, which tries to chain, which lands in code in unloaded plugin 1. disaster strikes here. plugin may need to have its own code to unload a dll if we don't currently unload libraries.
Comment 16 Ted Mielczarek [:ted.mielczarek] 2008-03-12 08:10:49 PDT
We have plenty of crash reports in NPSWF32.dll:
http://crash-stats.mozilla.com/?do_query=1&query_search=signature&query_type=contains&query=NPSWF&date=&range_value=1&range_unit=weeks

bsmedberg recalled that in MMgc, it uses exception handling for its guard pages:
http://mxr-test.landfill.bugzilla.org/tamarin-central/source/core/GrowableBuffer.cpp#272

The code there actually looks correct, and should only handle its own thread's exceptions. The code in Flash 9 must be calling SetUnhandledExceptionFilter directly, which would break us. Actually, they obviously are, since if I break on it, it gets hit a lot.
Comment 17 Benjamin Smedberg AWAY UNTIL 2-AUG-2016 [:bsmedberg] 2008-03-12 08:17:32 PDT
Steven, I at first thought this might be caused by http://mxr-test.landfill.bugzilla.org/tamarin-central/source/core/GrowableBuffer.cpp#280 which doesn't chain the old exception handler properly. I will submit a patch for that. But that code is clearly not the cause of this particular problem, because it doesn't call SetUnhandledExceptionFilter.

Can you put me in contact with the correct person at Adobe so that we can get the Flash plugin to chain exception handlers correctly?
Comment 18 Michelle Sintov 2008-03-12 09:37:17 PDT
Benjamin, I'll hook you up with the right person at Adobe.
Comment 19 Michelle Sintov 2008-03-12 11:14:28 PDT
This bug is related to bug 407958, which has been fixed on the Flash Player side (slated for a future release). The Flash Player will now no longer call SetUnhandledExceptionFilter() every time the Flash Player gets called by the browser.
Comment 20 Ted Mielczarek [:ted.mielczarek] 2008-03-12 14:26:34 PDT
*** Bug 386343 has been marked as a duplicate of this bug. ***
Comment 21 Ted Mielczarek [:ted.mielczarek] 2008-03-14 11:04:00 PDT
*** Bug 422966 has been marked as a duplicate of this bug. ***
Comment 22 Ted Mielczarek [:ted.mielczarek] 2008-03-17 08:06:06 PDT
Can't block on this, it's not our bug. :-/ I really hope that Adobe will get a new version of the plugin out before we ship though. Ideally ASAP so we can determine if we're missing out on any bad crashes.
Comment 23 u88484 2008-03-17 08:25:15 PDT
Is there anyway Adobe can provide an expected release time, as in a certain month or quarter?
Comment 24 David Weitz 2008-03-23 15:34:00 PDT
Well hopefully Adobe would be nice enough to give someone a build of the release so that testing can ensue.
Comment 25 Tony Chung [:tchung] 2008-03-24 10:13:48 PDT
This is unfortunate that we can't block on this, since it is a critical and common exposure to our users.    I've noticed myself my crashes stopped coming in after they 3/20 nightly build, having a page with a tab having a flash banner open.

Relnoting for beta 5.
Comment 26 Michael Morgan [:morgamic] 2008-04-02 17:06:00 PDT
Michelle - any update on when the Flash update might happen?  Also, is there a workaround in the client that could gracefully prevent a plugin from calling SetUnhandledExceptionFilter()?
Comment 27 Michelle Sintov 2008-04-03 09:20:20 PDT
Our apologies for this bug; this is an ugly situation. We do have a fix for it, but unfortunately, you should assume there will be no Flash Player update that includes this bug fix prior to the release of Firefox 3.
Comment 28 u88484 2008-04-03 09:33:18 PDT
Michelle, thank you for responding.  I'm just a regular ole user of Firefox but I am wondering if you mean release as in a final release of the Flash Player or does your response even mean that not even a beta version of the Flash Player (released before Firefox 3.0) will include a fix?
Comment 29 Ted Mielczarek [:ted.mielczarek] 2008-04-09 07:30:11 PDT
Flash 9.0 r124 still contains this issue, FWIW.
Comment 30 Michelle Sintov 2008-04-09 09:30:18 PDT
Indeed, the Flash Player with the fix is not Flash Player 9r124. It will be a later release. I will update this bug when the fixed player is live.
Comment 31 Step Christopher 2008-04-10 13:52:01 PDT
I've seen this bug an awful lot. Hard to know what's happening when Breakpad doesn't even get launched(and I'm sort of an "average user" in some ways).  

I guess the way to avoid it on my end is to block all flash, and not leave tabs open if they have flash on the page?  I've got noscript, but obviously sometimes I actually do want to allow the flash object to run.
Comment 32 Unknown W. Brackets 2008-04-10 20:14:48 PDT
(In reply to comment #31)
> I've seen this bug an awful lot. Hard to know what's happening when Breakpad
> doesn't even get launched(and I'm sort of an "average user" in some ways).  
> 
> I guess the way to avoid it on my end is to block all flash, and not leave tabs
> open if they have flash on the page?  I've got noscript, but obviously
> sometimes I actually do want to allow the flash object to run.

I also get this a lot (now that I switched to Minefield as my primary browser.)  However, my experience tells me that once Flash loads, Breakpad can no longer catch exceptions or crashes.

Even if I close that tab or navigate away, the damage is done.  So, unless you're willing to live entirely without Flash Player you will most likely see this bug quite frequently.  I know that almost all of my crashes end this way.

-[Unknown]
Comment 33 timeless 2008-04-10 23:49:58 PDT
there's really nothing breakpad can do.

if you care about crashes and really want to get reports, there's something you can do:

http://developer.mozilla.org/en/docs/How_to_get_a_stacktrace_with_WinDbg

1. open firefox
2. run windbg
3. debug>attach to process
4. select firefox
5. debug>go

when it crashes, read the rest of the instructions from the url.
Comment 34 Jenn Scott 2008-05-15 12:54:12 PDT
It looks to me that the beta for Flash Player 10 (10.0.1.218) has the fix for this, although if people want to plug the beta as a fix for this beyond nightly testers is up to higher-ups:

1. Go to the demos on the Adobe Labs site (http://labs.adobe.com/technologies/flashplayer10/demos/index.html).
2. Click on any of the videos at the bottom to play them.
3. While the video is playing, in another tab, load the third testcase in Bug 398332 (https://bugzilla.mozilla.org/attachment.cgi?id=316234).
4. Firefox does crash and Breakpad does come up.
Comment 35 Michelle Sintov 2008-05-15 12:57:37 PDT
(In reply to comment #34)
> It looks to me that the beta for Flash Player 10 (10.0.1.218) has the fix for
> this<snip>

You are correct. The Flash Player 10 beta now contains the fix for this bug: http://labs.adobe.com/downloads/flashplayer10.html
Comment 36 Dave Garrett 2008-05-15 13:37:31 PDT
Are there any plans to ever release an update to Flash Player 9 with this fix in it, or will most people only get it when the final version 10 is released?  

Is it possible to get a rough guess of how many months until this might be ready?
Comment 37 Michelle Sintov 2008-05-15 13:54:36 PDT
There are no plans to put this fix into Flash Player 9, so yes, people must have Flash Player 10. Fortunately, upgrades to Flash Player 10 happen quickly.

Flash Player 10 has no set final release date.
Comment 38 Martin 2008-05-21 05:20:40 PDT
*** Bug 434525 has been marked as a duplicate of this bug. ***
Comment 39 Liam K. 2008-06-08 21:40:38 PDT
So if this is an Adobe bug and is going to be fixed in a future release, isn't it INVALID or WONTFIX?
Comment 40 Samuel Sidler (old account; do not CC) 2008-06-08 21:44:18 PDT
(In reply to comment #39)
> So if this is an Adobe bug and is going to be fixed in a future release, isn't
> it INVALID or WONTFIX?

We're using this bug to track the progress. It will be closed as INVALID when a final release (non-beta) version of the fixed plug-in is out.
Comment 41 Wayne Mery (:wsmwk, NI for questions) 2008-08-24 13:59:12 PDT
RC was announced 8/11 http://labs.adobe.com/technologies/flashplayer10/
Comment 42 Jim Jeffery not reading bug-mail 1/2/11 2008-08-24 16:13:36 PDT
Sorry to report that the issue with Flash blocking the startup of Breakpad has returned with the RC build.  

I've been using it for awhile now and once in a while I get a Breakpad report on crash, but I have just found one site I crash on consistently while testing new builds in Minefield/Firefox 3.1a2pre will not fire the Breakpad unless I disable Flash... 

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1a2pre) Gecko/20080824031931 Minefield/3.1a2pre Firefox/3.0 ID:20080824031931
Comment 43 Philip Chee 2008-08-24 18:41:45 PDT
And that site is?
Comment 44 Jim Jeffery not reading bug-mail 1/2/11 2008-08-25 04:42:23 PDT
Using any of recent builds since 'tracemonkey' was added enter about:config and filter for JIT - toggle the 'content' to true

Then visit google documents..  instant crash for me, no breakpad.  Disable Flash RC in Tools->Addons:  then I get a breakpad on crash.

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1a2pre) Gecko/20080825031951 Minefield/3.1a2pre Firefox/3.0 ID:20080825031951
Comment 45 Jeremy Greene 2008-10-27 01:58:30 PDT
I had this problem with Flash Player 9. Flash Player 10.0.12.36 resolved it.

Firefox version is:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 (CK-MetaCarta) Firefox/3.0
Comment 46 AndrewM 2008-11-16 20:41:54 PST
With Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b2pre) Gecko/20081115 Minefield/3.1b2pre ID:20081115150852 and Flash Player 10.0.12.36, when I have Flash enabled, and I load the Flash movie in comment 10 and then crash (using the "Crash Me Now!" extension), Breakpad pops up as expected.

Jim Jeffery, are you able to reproduce using those steps on a recent Firefox build (which I think has tracemonkey turned on by default now) and the release version of Flash 10?
Comment 47 Ted Mielczarek [:ted.mielczarek] 2008-11-17 04:00:54 PST
Per comment 35, Flash Player 10 fixes this bug. Going to resolve this WFM, as Flash 10 is readily available now.
Comment 48 Henrik Skupin (:whimboo) 2008-11-17 16:00:16 PST
Verified with Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b2pre) Gecko/20081114 Minefield/3.1b2pre ID:20081114034305 and Flash 10.0.12.36.

The crash is captured now and Breakpad comes up as expected.

Marcia, something we could have a test for on Litmus? We could use crashme.xpi to trigger that crash while a webpage with flash content is open.
Comment 49 Nochum Sossonko [:Natch] 2008-12-16 16:48:23 PST
Can this be removed from the relnotes?
Comment 50 Dave Garrett 2008-12-16 16:51:49 PST
(In reply to comment #49)
> Can this be removed from the relnotes?

I'd recommend keeping a relnote telling users to simply upgrade to Flash 10 to avoid the problem. (not to mention other issues fixed in that release)
Comment 51 Wayne Mery (:wsmwk, NI for questions) 2008-12-16 16:58:56 PST
(In reply to comment #48)
>
> The crash is captured now and Breakpad comes up as expected.
> 
> Marcia, something we could have a test for on Litmus? We could use crashme.xpi
> to trigger that crash while a webpage with flash content is open.

Henrik, it is *partly* there - crashme is mentioned in the test, but perhaps something should be added about flash.  https://litmus.mozilla.org/show_test.cgi?id=5118
Comment 52 Wayne Mery (:wsmwk, NI for questions) 2009-04-06 10:44:43 PDT
updated https://litmus.mozilla.org/show_test.cgi?id=5118
Comment 53 Henrik Skupin (:whimboo) 2009-04-06 11:34:27 PDT
Sorry Wayne, but I think you misunderstood this bug. Breakpad isn't able to catch the crash if a webpage containing any flash content is open - means Flash is active. It's not related if it is installed or not. We should better create a new Litmus test.
Comment 54 Wayne Mery (:wsmwk, NI for questions) 2009-04-06 12:50:53 PDT
OK - The test WITH flash regarding this bug should be a new testcase as Henrik states.  And only for FFT as discussed on IRC.  Will leave this for someone else, but modified existing testcase to just mention this bug. Also updated that 64bit is not supported. The added text reads

Note: Windows-only - Breakpad will not catch crashes if a flash image has been opened and Flash Player version is older than v10 (see regression bug 422308).
Note: Do not test in 64 bit environment. 64 bit builds are not supported by crashme nor breakpad.
Comment 55 nichole 2009-04-23 08:39:59 PDT
Created attachment 374270 [details]
Comment 56 u88484 2009-04-23 08:44:13 PDT
(In reply to comment #55)
> Created an attachment (id=374270) [details]
> bug stablelizer
I'm not sure what the point of your attachment is but this bug is fixed and if you are still seeing it, you need to upgrade your flash.
Comment 57 Ubermonkey 2009-04-23 09:02:37 PDT
The attachment in #55 is probably malicious.  Can someone delete it?
Comment 58 timeless 2009-04-23 09:26:03 PDT
Comment on attachment 374270 [details]
 

So, this file is actually signed, I think it really is firefox 3.0.9's firefox.exe, however it's totally useless and should not have been attached.

Note You need to log in before you can comment on or make changes to this bug.