Closed Bug 1082670 Opened 5 years ago Closed 5 years ago

[adbe 3839666] Flash plugin crashes/hangs every time in Nightly, after bug 1061335

Categories

(Core :: Plug-ins, defect, blocker)

36 Branch
x86_64
Windows 8.1
defect
Not set
blocker

Tracking

()

VERIFIED FIXED
mozilla36
Tracking Status
firefox35 --- unaffected
firefox36 - verified

People

(Reporter: streetwolf, Assigned: dmajor)

References

(Blocks 1 open bug)

Details

(Keywords: flashplayer, regression, site-compat)

Crash Data

Attachments

(1 file, 1 obsolete file)

Blocks: 1061335
Severity: normal → blocker
Keywords: regression
CSET: https://hg.mozilla.org/mozilla-central/rev/54217864bae9

This video playing fine - http://www.dailymotion.com/video/x27x16j with Adobe Flash 15.0.0.189 Stable. 

DLL File: NPSWF32_15_0_0_189.dll

Windows 8.1 Update 1 64-bit but 32-bit Nightly.
Confirmed  - setting to NEW
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flash plug-in is crashing, but getting the 'generic' Flash crash report we've been seeing for ages now:

https://crash-stats.mozilla.com/report/index/78d98011-1762-458f-844a-f6bc42141014
I'm not getting any crashes, just no video on sites.
WFM with Flash Player 15.0.0.189.  Got specific examples?
Any video at cnn.com
(In reply to IU from comment #5)
> WFM with Flash Player 15.0.0.189.  Got specific examples?


Well, I tested on CNN's web-site, and was getting flash-player crashes.

A quick test just now on Youtube, flashplayer does not crash.  So maybe its related to CNN's player... I believe they use some extra coding on their video.  Octoshape add-in for Flashplayer is what they use as I recall.
CNN WFM.  Try updating your Flash Player to 15.0.0.189, if you haven't already.

And in case it matters, I'm using Windows 8.1.

YouTube is no longer Flash by default (unless you've done something to force it), so make sure it's actually a Flash video you're viewing.
YouTube uses HTML5 which is not affected.  Flash doesn't play on other sites like weather.com
The latest beta Flash is 15.0.0.199.  Give that a try.
(In reply to Jim Jeffery not reading bug-mail 1/2/11 from comment #7)
> (In reply to IU from comment #5)
> > WFM with Flash Player 15.0.0.189.  Got specific examples?
> 
> 
> Well, I tested on CNN's web-site, and was getting flash-player crashes.
> 
> A quick test just now on Youtube, flashplayer does not crash.  So maybe its
> related to CNN's player... I believe they use some extra coding on their
> video.  Octoshape add-in for Flashplayer is what they use as I recall.

I do not think that the problem is not limited in CNN.

 
http://www.adobe.com/software/flash/about/ site should show Flash Version.
However, Flash crashes or just empty or gray rectangle.

bp-7c6d6c00-b9c5-416d-8bec-9b88e2141014
bp-eea5dae6-91dc-4b30-8e02-585642141014
sorry, my English translation error , 

I think that the problem is not limited in CNN.
(In reply to IU from comment #8)
> CNN WFM.  Try updating your Flash Player to 15.0.0.189, if you haven't
> already.
> 
> And in case it matters, I'm using Windows 8.1.
> 
> YouTube is no longer Flash by default (unless you've done something to force
> it), so make sure it's actually a Flash video you're viewing.

I'm already running 15.0.0..189 as of today, but... I'm on win7 x64.  This is getting confusing quickly...

NO crash on Adobe site from comment #11 here.

Youtube is NOT all HTML5 all the time, lately I've been getting a lot of fall-backs to Flash for some reason.
As Alice mentioned over at MozillaZine setting environment variable MOZ_DISABLE_OOP_PLUGINS=1 makes Flash work for me.
I can't reproduce this either on http://www.adobe.com/software/flash/about/ or the dailymotion URL, using a win32 build on win7x64.
Does it work if you disable protected mode?
http://kb.mozillazine.org/Flash#Disabling_Protected_Mode_in_Flash_11.3
(In reply to IU from comment #16)
> Does it work if you disable protected mode?
> http://kb.mozillazine.org/Flash#Disabling_Protected_Mode_in_Flash_11.3

Yes, Flash works fine when disabling protected mode.  So there are two ways around the problem so far.
Perhaps obviously, those workarounds are not acceptable in the general case. We need to understand why this is actually failing.
Component: Build Config → Plug-ins
If there is anything I can do to help resolve this let me know. Keep in mind I am just an end-user.  Do you ever use something like TeamViewer to help with a problem?
I get a crash in FlashPlayerPlugin_15_0_0_199.exe (but not plugin-container, and not firefox) on Win7 SP1 at http://edition.cnn.com/video.

Near the crash site I see the standard sequence of tests for "is there a valid PE image at this address", but there's nothing at that address (it's reserved but not committed).

Odd that a change in FF would cause this in Flash's process. Are they mapping FF binaries into their exe's memory? Maybe that logic is getting messed up by the new CRT files or the new version linker flags on the other binaries.

bsmedberg do you have a way to reach somebody at Adobe?
Flags: needinfo?(benjamin)
Yes, I'll ping Adobe.
Flags: needinfo?(benjamin)
This has happened on today build and above I can give you my build ids
Ps happens on Facebook games from today 

so I will close my bug I opened and mark duplicate of this
Duplicate of this bug: 1083504
Interesting. When I go here: http://webhtb.sourceforge.net/video_demo.html (per bug 1083504), then I get my postmortem debugger for the crash in FlashPlayerPlugin_15_0_0_199.exe, but then later I see a frowny-face for the crash in plugin-container.exe, at the same F1398665248 symbol that others are seeing: bp-fd904735-3f90-4c01-887b-d92522141016.

I guess if one didn't have a postmortem debugger, only the plugin-container.exe crash would come up.
The plugin-container crash is an int 3, so I assume that's just a panic resulting from the original crash in FlashPlayerPlugin.
This crash stat may be of some interest since it is slightly different than the more generic one that everyone has been posting, though I also have plenty of those generic ones as well. Hope it helps.

https://crash-stats.mozilla.com/report/index/f4828673-045b-453e-ad24-4c8152141016
We're looking into it.  This is Adobe 3839666
Duplicate of this bug: 1083659
Crash Signature: [@ F1398665248_____________________________ ]
Duplicate of this bug: 1083812
(In reply to Jeromie Clark from comment #28)
> We're looking into it.  This is Adobe 3839666

Can we get a link to the Adobe bug report filed?  I can't seem to find it when I do a search on Adobe's bug site.
Duplicate of this bug: 1084048
Summary: Flash not working after Bug 1061335 → [adbe 3839666] Flash not working after Bug 1061335
When I use the 64bit version of Firefox 36.0A1, Flashplayer works perfectly, it seems that the 32-bit versions of firefox do not work with flashplayerd!
(In reply to Gary [:streetwolf] from comment #31)
> (In reply to Jeromie Clark from comment #28)
> > We're looking into it.  This is Adobe 3839666
> 
> Can we get a link to the Adobe bug report filed?  I can't seem to find it
> when I do a search on Adobe's bug site.

¡Hola Mr. Clark!

Any progress on this bug from Adobe?

¡Gracias!
Alex
Flags: needinfo?(jeclark)
Duplicate of this bug: 1084420
(In reply to Gary [:streetwolf] from comment #31)
> (In reply to Jeromie Clark from comment #28)
> > We're looking into it.  This is Adobe 3839666
> 
> Can we get a link to the Adobe bug report filed?  I can't seem to find it
> when I do a search on Adobe's bug site.

Me neither. Is it possible the bug report is private?
Duplicate of this bug: 1084452
We have separate external and internal bug repositories.  I can create an external bug, but it will basically show as Open until it's Closed, which doesn't really add any value for you.  I provide you guys with the internal bug numbers primarily just to speed communication between the two engineering groups.
Flags: needinfo?(jeclark)
I am not seeing videos when I try access videos from YouTube or http://www.chicagotribune.com/ for example.
I have Flash 15.0.0152.
I also tried Flash 15.0.0.189 and got the same results, black screen.

FF 34.1
(In reply to jlerner10 from comment #40)
> I am not seeing videos when I try access videos from YouTube or
> http://www.chicagotribune.com/ for example.
> I have Flash 15.0.0152.
> I also tried Flash 15.0.0.189 and got the same results, black screen.
> 
> FF 34.1

This bug is about FF36, not prior versions.
Nope.  This is not a dire emergency from our perspective, as you do have the option to roll back to the old project files.  I can tell you that updating Flash Player to VS2013 to resolve this is not an achievable outcome in the short-term.

This issue is in the queue and has been assigned to an engineer for investigation.  When we have finished our analysis and can provide you with useful feedback, we'll update this bug.
At its root, I believe this issue has something to do with the interaction between Flash Player protected mode and Gecko IPC.  That disabling one or the other resolves the issue is evidence of this.

I even have protected mode disabled permanently on two systems because it causes stalls and horribly jerky playback of lots of videos.  Granted, it could be IPC that's at fault, but Mozilla not longer supports having IPC disabled.

Sure there are risks, but I have click-to-play enabled for Flash by default.

It could be a Flash problem or it could simply be a side-effect of an IPC problem, so (rather than just waiting on Adobe) it would be prudent of Mozilla to simultaneously investigate Gecko IPC behavior.
Duplicate of this bug: 1084567
64bit Nightly seems OK for me. 32bit has constant hangs.
One of the interesting tidbits of feedback that I've been seeing lately for users in the Flash forums is that disabling PluginContainer (via the OOP-related environmental variables) resolves a lot of the stability problems that people associated with Flash Player Protected Mode.  I also noticed that when running in ProtectedMode, a crash in Flash Player doesn't take down the tab or browser when it crashes.  It might be a low-cost win that could be worth exploring independent of the current bug.
Putting  ProtectedMode=0
in mms.cfg  C:\Windows\SysWOW64\Macromed\Flash
works.
Setting MOZ_DISABLE_OOP_PLUGINS=1 in system environment works for me as well. Otherwise Flash player crashes.

Win 7 x64
Nightly 36 32bit
Confirming that setting ProtectedMode=0 in mms.cfg works for beta build 15.0.0.207 as well.
Duplicate of this bug: 1085040
Duplicate of this bug: 1085339
We will not be disabling plugin-container, and in fact it will soon be the only available option, since it is required to support multi-process content.
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #54)
> We will not be disabling plugin-container, and in fact it will soon be the
> only available option, since it is required to support multi-process content.

Could you guys at least troubleshoot/inspect Gecko IPC with respect to this bug?  I strongly believe this is actually an IPC bug and not a Flash Player protected mode bug.
I managed to narrow this down: the crash appears to be related to the crash reporting code that Firefox injects into the FlashPlayerPlugin.exe processes (bug 769048 has history). Likely due to the CRT upgrade, but we'll need to debug further to figure out exactly what's going on.
Cool, let us know if there's anything we can do to help.  Happy to provide symbols, etc.
Duplicate of this bug: 1085580
Keywords: site-compat
The crash is indeed happening in the crash reporting DLL:

breakpadinjector!_ValidateImageBase
breakpadinjector!_IsNonwritableInCurrentImage
breakpadinjector!_cinit
breakpadinjector!_CRT_INIT
breakpadinjector!Start

The relevant piece of _cinit is:
        if (_FPinit != NULL &&
            _IsNonwritableInCurrentImage((PBYTE)&_FPinit))
        {
            (*_FPinit)(initFloatingPrecision);
        }

That code initializes the CRT's floating-point support, if the program expects to use it. The breakpadinjector.dll from VS2010 had a null _FPinit, so we never called _IsNonwritableInCurrentImage. With VS2013 we have a non-null _FPinit routine. (I don't know why -- maybe some SDK or CRT header used by injector.cpp added a float variable somewhere.)

_ValidateImageBase crashes because the memory at the start of breakpadinjector.dll is decommitted. I suspect that our FinalizeSections [1] didn't map the PE header because it's not part of any section.

Interestingly, _ValidateImageBase protects itself with a try/catch, but because the handler code is not part of a bona-fide MEM_IMAGE, the NT exception dispatcher says no-go.

I propose that we add an extra VirtualAllocEx to commit breakpadinjector's image header.

[1] http://dxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/LoadLibraryRemote.cpp#112
Assignee: nobody → dmajor
Attached patch Commit the header (obsolete) — Splinter Review
Attachment #8508371 - Flags: review?(benjamin)
Comment on attachment 8508371 [details] [diff] [review]
Commit the header

Er, it would help if I followed it up with a WriteProcessMemory :)
Attachment #8508371 - Attachment is obsolete: true
Attachment #8508371 - Flags: review?(benjamin)
Now with actually initialized data.

0:011> db .-159a
00ec0000  4d 5a 90 00 03 00 00 00-04 00 00 00 ff ff 00 00  MZ..............
00ec0010  b8 00 00 00 00 00 00 00-40 00 00 00 00 00 00 00  ........@.......
00ec0020  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00  ................
00ec0030  00 00 00 00 00 00 00 00-00 00 00 00 f8 00 00 00  ................
00ec0040  0e 1f ba 0e 00 b4 09 cd-21 b8 01 4c cd 21 54 68  ........!..L.!Th
00ec0050  69 73 20 70 72 6f 67 72-61 6d 20 63 61 6e 6e 6f  is program canno
00ec0060  74 20 62 65 20 72 75 6e-20 69 6e 20 44 4f 53 20  t be run in DOS 
00ec0070  6d 6f 64 65 2e 0d 0d 0a-24 00 00 00 00 00 00 00  mode....$.......
Attachment #8508378 - Flags: review?(benjamin)
Comment on attachment 8508378 [details] [diff] [review]
Copy the breakpadinjector image header

Review of attachment 8508378 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/crashreporter/injector/injector.cpp
@@ +23,5 @@
>  
>  extern "C"
>  __declspec(dllexport) DWORD Start(void* context)
>  {
> +__debugbreak();

Bah, forgot to qref this. I'll remove before submitting. Same for the trailing whitespace in the previous file.
Duplicate of this bug: 1078000
Duplicate of this bug: 1086192
(Making summary more keyword rich, to try and avoid dupes).
Summary: [adbe 3839666] Flash not working after Bug 1061335 → [adbe 3839666] Flash plugin crashes/hangs every time in Nightly, after bug 1061335
Attachment #8508378 - Flags: review?(benjamin) → review+
Duplicate of this bug: 1086607
Based on the feedback, I'm going to close the bug on our side.  Let me know if you need anything further from Adobe.
Sounds good. Thanks for keeping an eye on this, Jeromie.
https://hg.mozilla.org/mozilla-central/rev/ac44aef9c4d4
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla36
Firefox 36.0a1 is still affected...
(In reply to Achwaq Khalid from comment #72)
> Firefox 36.0a1 is still affected...

Are you running e10s? I had to fix a plugin-related crash for that in bug 1087410.
(In reply to Aaron Klotz [:aklotz] from comment #73)
> (In reply to Achwaq Khalid from comment #72)
> > Firefox 36.0a1 is still affected...
> 
> Are you running e10s? I had to fix a plugin-related crash for that in bug
> 1087410.

Yes Aaron, I have E10S enabled
Duplicate of this bug: 1083164
Are you sure this has been solved?

https://crash-stats.mozilla.com/report/index/4a221aa4-146b-4558-a467-9766a2141202

Still occurs in Ff33 on Dec 02:
dgapi.dll 	dgapi.dll@0xde824
dgapi.dll 	dgapi.dll@0x33fab
Date Processed	2014-12-02 13:38:47.256006
Process Type 	plugin Shockwave Flash Version:15.0.0.239 Filename:NPSWF32_15_0_0_239.dll
(In reply to Suburp from comment #76)
> Are you sure this has been solved?
> 
> https://crash-stats.mozilla.com/report/index/4a221aa4-146b-4558-a467-
> 9766a2141202
> 
> Still occurs in Ff33 on Dec 02:


This bug is about Flash crashes in FF36, not FF33, file a new bug, please.
Blocks: 1137664
didn't see this one for a while until the latest stable 36.0 came out. Took only a few hours to show up:
https://crash-stats.mozilla.com/report/index/b840c375-976d-40c9-8170-0ee682150227
(https://bugzilla.mozilla.org/show_bug.cgi?id=1137664)
(In reply to Loic from comment #78)
> (In reply to Suburp from comment #76)
> > Are you sure this has been solved? seems to do the same in V36, too.
> > 
> > https://crash-stats.mozilla.com/report/index/4a221aa4-146b-4558-a467-
> > 9766a2141202
> > 
> > Still occurs in Ff33 on Dec 02:
> 
> 
> This bug is about Flash crashes in FF36, not FF33, file a new bug, please.

Loic, filed those separately, but this is a 36.0 issue:
https://crash-stats.mozilla.com/report/index/b840c375-976d-40c9-8170-0ee682150227
AFAIK, this Flash crash signature is generic, not sure if it would help a lot. Look at the bug report list, it's huge.
Restrict Comments: true
You need to log in before you can comment on or make changes to this bug.