Bug 774281 (F1398665248)

Generic bug: Flash plugin-container crash in F1398665248

NEW
Unassigned

Status

External Software Affecting Firefox
Flash (Adobe)
--
critical
5 years ago
3 months ago

People

(Reporter: bsmedberg, Unassigned)

Tracking

(Depends on: 1 bug, Blocks: 1 bug, 7 keywords)

unspecified
x86
Windows
crash, crashreportid, flashplayer, thirdparty, topcrash, topcrash-plugin, topcrash-win
Dependency tree / graph

Firefox Tracking Flags

(firefox47 affected, firefox48 affected, firefox49 affected, firefox50 affected, firefox51 affected, firefox52 wontfix, firefox53 affected, firefox54 affected, firefox55 affected, firefox-esr45 affected, firefox-esr52 affected)

Details

(Whiteboard: [Flash-11.3] generic signature bug, please file bugs blocking this one, crash signature)

Attachments

(2 attachments)

(Reporter)

Description

5 years ago
This is a generic bug covering the Flash 11.3 signature F1398665248_____________________________. If you have specific steps to reproduce a Flash crash, please do not comment in this bug! Instead, please file a *new* bug and mark it blocking this one.

The F1398665248_____________________________ signature in Flash crashes is a generic signature. It happens when Flash protected mode for Firefox is enabled and one of the other FlashPlayerPlugin processes crashes or is killed. Currently in Firefox 13 and 14, we only track crashes in the plugin-container process, so almost all Flash 11.3 crashes show up as this signature.

In Firefox 15 and 16 we have implemented crash reporting for the FlashPlayerPlugin processes as well, so this signature will usually not be reported; we'll see the more interesting crash signature in the Flash process instead.

It appears that this crash is uncontrolled behavior, and Adobe is working with us to make the behavior more controlled when the Flash process crashes, something like MOZ_CRASH, so this signature will very likely change in future versions of Flash 11.3.

Updated

5 years ago
Depends on: 774092
(Reporter)

Comment 1

5 years ago
I can confirm that with the 11.3.300.267 build killing a FlashPlayerPlugin process while hulu is running produces a different (controlled) EXCEPTION_BREAKPOINT report.

Comment 2

5 years ago
Has it been tried with Flash 11.3.300.268?

Comment 3

5 years ago
(In reply to Arthur K. from comment #2)
> Has it been tried with Flash 11.3.300.268?
It's a generic bug, so there are no STR to test
(Reporter)

Comment 4

5 years ago
Well actually, you can usually reproduce the generic bug by playing a hulu.com movie (vimeo would probably work as well) and then killing one of the FlashPlayerPlugin.exe processes in the task manager.

Updated

5 years ago
Depends on: 782395

Comment 5

5 years ago
Any improvement with 11.4.402.265?

Comment 6

5 years ago
(In reply to Arthur K. from comment #5)
> Any improvement with 11.4.402.265?
No. See https://crash-stats.mozilla.com/report/list?product=Firefox&query_search=signature&query_type=contains&reason_type=contains&process_type=plugin&plugin_field=filename&plugin_query_type=exact&plugin_query=NPSWF32_11_4_402_265.dll&do_query=1&signature=F1398665248_____________________________

Updated

5 years ago
Depends on: 787014

Updated

5 years ago
Depends on: 789880

Comment 7

5 years ago
We implemented the suggested fix, similar to what is in MOZ_CRASH, for 11.3 and 11.4.402.265.

When we detect an error that requires us to shut down plugin-container, we do the following to exit the process:

__debugbreak();
::TerminateProcess(::GetCurrentProcess(), 3);

This is based on the MOZ_CRASH macro defined in http://mxr.mozilla.org/mozilla-central/source/mfbt/Assertions.h#153

Is there something else we should be doing to more cleanly exit the plugin-container process when we detect a fatal error so that this is not classified as a crash for this process?
(Reporter)

Comment 8

5 years ago
I'm not sure what you mean... we now see that the crash reason is correct (EXCEPTION_BREAKPOINT) and we can track the volume in a useful way, so I believe that the current behavior is as expected.

Comment 9

5 years ago
Ah, perhaps there was a misunderstanding then. This bug/signature was in one of the top crashers lists we were looking at, so we thought that we still needed to do something to address it.

However, if this is behaving as expected, then I suppose we will just continue to see this crash as long as we still have other crashes that cause the sandbox process to go away unexpectedly.
(Reporter)

Comment 10

5 years ago
Yes. To the user it still represents "Flash player crashing", but to us it indicates that there was a problem in one of the other processes which was not caught usefully by the crash reporter and so what's left is a bucket of crashes which are good for volume measurements but often aren't useful in terms of engineering.

If we have new bugs with STR to reproduce this signature, I'd like to see them because I'd like to figure out why the crash reporter didn't catch the other process dying.

Updated

5 years ago
Depends on: 790368
Depends on: 789696
Duplicate of this bug: 790153

Comment 12

5 years ago
Any improvement with 11.5.500.80?

Comment 13

5 years ago
FWIW, friend who'd encountered crasher (11.3 and 11.4 Flash in Windows 7) reports that the fix for him was disabling acceleration in Flash.

Well, maybe more of a workaround, but given the increasingly limited utility of Flash, really more of a fix for him.

Perhaps Adobe could push out a hotfix 11.4.1 w/ acceleration disabled by default? :)

Comment 14

5 years ago
Have a second report of disabling acceleration being a workaround, also a Windows user.

Updated

5 years ago
Depends on: 812047

Comment 15

5 years ago
@Nemo: Please attach dxdiag output for affected machines, and ideally, URLs to sites where the crash is occurring consistently.  We'll attempt to build out a similar configuration and reproduce the issue.  Once we're sure that the crash is related to the HW/Driver combination, we can disable GPU acceleration for that particular profile.

Comment 16

5 years ago
Welp. I asked one of 'em, but he didn't answer me, and I'm betting he isn't too interested in doing debugging for Adobe when he already has a fix.  I might be able to convince him to pastebin about:support or otherwise give card info.

The other fellow I can't contact.

Updated

5 years ago
Depends on: 814003

Updated

5 years ago
Depends on: 842077

Updated

5 years ago
No longer depends on: 842077

Updated

5 years ago
Depends on: 846126

Comment 17

4 years ago
Created attachment 748509 [details]
dmp/extra files & dxdiag

After playing http://www.youtube.com/watch?v=NWoHju_ii4c I scrolled down the page -> crash
https://crash-stats.mozilla.com/report/index/bp-12c89852-ea69-4ac2-95bf-cb1672130512
https://crash-stats.mozilla.com/report/index/bp-8e450e15-61a7-4ef6-ad8f-9a9092130512

Comment 18

4 years ago
(In reply to MrX1980 from comment #17)
I doesn't crash for me with Flash 11.7

> https://crash-stats.mozilla.com/report/index/bp-12c89852-ea69-4ac2-95bf-
> cb1672130512
It's bug 772097 that spiked with Flash 11.8.

> https://crash-stats.mozilla.com/report/index/bp-8e450e15-61a7-4ef6-ad8f-
> 9a9092130512
It's not this bug which a generic one but will likely be fixed along with bug 772097.

Updated

4 years ago
Depends on: 891035

Updated

4 years ago
No longer depends on: 891035

Updated

4 years ago
Depends on: 891813
with Flash 11.8.800.94 is number one on https://crash-analysis.mozilla.com/bsmedberg/flash-topcrash-release.html

Updated

4 years ago
Keywords: thirdparty

Updated

4 years ago
Duplicate of this bug: 935493

Updated

4 years ago
Keywords: topcrash → topcrash-plugin, topcrash-win
Tracy noticed a crash spike across branches over the weekend.

Looking at the explosive reports list for Fx29, the number of crashes with this signature doubled in the last few days, starting on 1/31.

The last Flash update was a couple of weeks ago. Also, the version of Flash reported in the reports I looked at varied from 11.9x to the latest 12.0.0.43, so it doesn't seem to be version specific.

Perhaps a change elsewhere, like a major website?

Comment 22

4 years ago
I think one reason could be "watching Super Bowl".
https://crash-stats.mozilla.com/report/index/a0d22cb0-887c-4ec7-9fea-035522140203

Comment 23

4 years ago
Flash keeps on crashing with this error in my VS2013 builds of Fx29 on Win 8.1 x64.

Here is my latest crash report https://crash-stats.mozilla.com/report/index/9935723b-25b2-4282-819f-6c1232140226

In earlier builds around 1/11, Flash did not crash. It started somewhen between 1/12 and 1/23 (with Fx28).

Comment 24

4 years ago
https://crash-stats.mozilla.com/report/index/213e784c-5927-4aab-a232-3ffcc2140227
Depends on: 713138

Updated

3 years ago
Duplicate of this bug: 713138
Comment hidden (off-topic)
FYI, this signature is now the #1 explosive crash in Firefox 31:
https://crash-analysis.mozilla.com/rkaiser/2014-05-04/2014-05-04.firefox.31.explosiveness.html

I'm not sure if that indicates a worse situation or not, and I'm not sure what the next step should be. I just thought I'd document it here in case anyone has an idea about what to do.

Comment 28

3 years ago
Unless we see this as a permanent trend that probably relates to a patch on our side, I wouldn't care about those generic Flash issues on Aurora or Nightly. And we did not see any significant increase on the other channels at this point.
Comment hidden (me-too)
Comment hidden (me-too)
Comment hidden (me-too)

Updated

3 years ago
Depends on: 1083164
No longer depends on: 1083164
Depends on: 1082670

Updated

3 years ago
Depends on: 1120011

Updated

3 years ago
Depends on: 1099944

Updated

3 years ago
Depends on: 1122646

Updated

3 years ago
Depends on: 1120286
(Reporter)

Updated

3 years ago
Duplicate of this bug: 1133407

Updated

2 years ago
Depends on: 1137664
(Reporter)

Updated

2 years ago
Duplicate of this bug: 1099944
Comment hidden (spam)

Updated

2 years ago
Depends on: 1155606

Updated

2 years ago
No longer depends on: 1155606
Duplicate of this bug: 761597
No longer depends on: 774092
Duplicate of this bug: 774092
No longer depends on: 1099944
Duplicate of this bug: 1078000
No longer depends on: 789696
No longer depends on: 789880
Depends on: 1157078
No longer depends on: 1120286

Updated

2 years ago
Summary: Generic bug: Flash plugin-container crash in F1398665248_____________________________ → Generic bug: Flash plugin-container crash in F1398665248
Blocks: 1150657

Comment 38

2 years ago
Flash Player has shipped a number of changes to mitigate this as part of a recent collaborative engineering effort with Mozilla.  Those changes are dependent on additional changes in Firefox, and my understanding is that we're waiting on the corresponding Firefox changes to make their way through the various stability channels at this point in time.  

I can't get a clear read on whether the situation is improved looking at the data in crash-stats, but we would naturally welcome additional engineering engagements to work on this as necessary moving forward.

Comment 39

2 years ago
Aaron, in which Firefox version do or did the changes ship that Jeromie is talking about?

Jeromie, can you tell us at what time or in which Flash version your side of the improvements shipped?
Flags: needinfo?(aklotz)
(In reply to Robert Kaiser (:kairo@mozilla.com) - on vacation or slow to reply until the end of June from comment #39)
> Aaron, in which Firefox version do or did the changes ship that Jeromie is
> talking about?

39.
Flags: needinfo?(aklotz)

Comment 41

2 years ago
This signature dropped significantly in volume with the Flash release that was shipped 2015-06-24.

Comment 42

2 years ago
(In reply to Robert Kaiser (:kairo@mozilla.com) - on vacation or slow to reply until the end of June from comment #41)
> This signature dropped significantly in volume with the Flash release that
> was shipped 2015-06-24.

Oops, sorry, probably not true, forgot for a moment that we are missing Flash symbols for the new release, so of course Flash crashes with a symbolized signature seem to have dropped.

Updated

2 years ago
Duplicate of this bug: 1188631

Updated

2 years ago
Duplicate of this bug: 1188682

Updated

2 years ago
Duplicate of this bug: 1188683

Updated

2 years ago
Crash Signature: [@ F1398665248_____________________________] [@ @0x0 | F1398665248_____________________________ ] → [@ F1398665248_____________________________] [@ @0x0 | F1398665248_____________________________ ] [@ npswf32_18_0_0_209.dll@0x3624bd]
Comment hidden (spam)
Comment hidden (spam)
Duplicate of this bug: 1221838
Duplicate of this bug: 1249959
Keywords: crashreportid, flashplayer, topcrash
Crash Signature: [@ F1398665248_____________________________] [@ @0x0 | F1398665248_____________________________ ] [@ npswf32_18_0_0_209.dll@0x3624bd] → [@ F1398665248_____________________________] [@ @0x0 | F1398665248_____________________________ ] [@ npswf32_18_0_0_209.dll@0x3624bd] [@ npswf32_18_0_0_194.dll@0x35e84d] [@ npswf32_18_0_0_203.dll@0x35ed9d]

Updated

a year ago
Duplicate of this bug: 1250116

Updated

a year ago
Depends on: 1275682
Comment hidden (spam)

Comment 52

a year ago
(In reply to Yorgos from comment #51)
> https://crash-stats.mozilla.com/report/index/b04d4faf-edd8-4beb-a730-
> 26b5a2160601

AdapterVendorID: 0x10de, AdapterDeviceID: 0x0fc6, AdapterSubsysID: 00000000, AdapterDriverVersion: 10.18.13.6191

Could possibly be fixed by updating to (v365.19) 10.18.13.6519 drivers.

Comment 53

a year ago
there is something in the changelog from nvidia?
because this issue arised a lot of years ago.

Comment 54

a year ago
Now I have 10.18.13.6519 drivers.
Crash volume for signature 'F1398665248_____________________________':
  - nightly (50): 11899
  - aurora (49): 31684
  - beta (48): 16372
  - release (47): 92596
  - esr (45): 16611

Affected platform: Windows
status-firefox47: --- → affected
status-firefox48: --- → affected
status-firefox49: --- → affected
status-firefox50: --- → affected
status-firefox-esr45: --- → affected

Comment 56

a year ago
(In reply to Yorgos from comment #54)
> Now I have 10.18.13.6519 drivers.

Did upgrading drivers reduce or eliminate crashes? You're on Flash 22.0.0.209 as well?

Comment 57

a year ago
A bit reduced, not fully resolved but very well reduced.
I am on 209.

Comment 58

a year ago
(In reply to Yorgos from comment #57)
> A bit reduced, not fully resolved but very well reduced.
> I am on 209.

This is good to know as it implies the driver on Vista still has room for improvement. Thanks for reporting back.

Comment 59

a year ago
this is the final driver for Vista.
Crash volume for signature 'F1398665248_____________________________':
 - nightly (version 51): 5697 crashes from 2016-08-01.
 - aurora  (version 50): 13400 crashes from 2016-08-01.
 - beta    (version 49): 47525 crashes from 2016-08-02.
 - release (version 48): 18140 crashes from 2016-07-25.
 - esr     (version 45): 22863 crashes from 2016-05-02.

Crash volume on the last weeks (Week N is from 08-22 to 08-28):
            W. N-1  W. N-2  W. N-3
 - nightly    2023    1497     412
 - aurora     4369    3907    2449
 - beta       4194   25129   15981
 - release    6004    5246    2874
 - esr        1464    1588    1814

Affected platform: Windows

Crash rank on the last 7 days:
           Browser     Content Plugin
 - nightly #14                 #1
 - aurora  #1                  #1
 - beta    #181                #1
 - release #1083               #1
 - esr     #1094               #1
status-firefox51: --- → affected
Duplicate of this bug: 1299582
Duplicate of this bug: 1249958
Crash volume for signature 'F1398665248_____________________________':
 - nightly (version 52): 2645 crashes from 2016-09-19.
 - aurora  (version 51): 8019 crashes from 2016-09-19.
 - beta    (version 50): 3383 crashes from 2016-09-20.
 - release (version 49): 16033 crashes from 2016-09-05.
 - esr     (version 45): 26082 crashes from 2016-06-01.

Crash volume on the last weeks (Week N is from 10-03 to 10-09):
            W. N-1  W. N-2
 - nightly    2223     422
 - aurora     7564     455
 - beta       2739     644
 - release   12016    4016
 - esr        1853    1731

Affected platform: Windows

Crash rank on the last 7 days:
           Browser     Content Plugin
 - nightly #13                 #1
 - aurora  #3                  #1
 - beta    #1868               #1
 - release #2096               #1
 - esr     #3692               #1
status-firefox52: --- → affected

Updated

9 months ago
Duplicate of this bug: 1312750
This crash is getting worse in 51 aurora and it's #2 in top crash. 
One of the crashes is caused by Flash Version: 23.0.0.185 Filename: NPSWF32_23_0_0_185.dll

We can also see the volume grows from 604 to 1109 on 2016-10-23 to 2016-10-24.
(Reporter)

Comment 66

9 months ago
Gerry, there are many possibilities here.
* A Firefox change caused a problem
* A Flash change caused a problem (either Flash release or Flash beta)
* Some common website changed their code and caused a problem
* We changed something about crash submission

Can you help with digging in and figuring out which of these is most likely? I don't know of any plugin changes that landed to aurora recently, and I don't think any of the auto-submission bits changed recently either (NI?mconley for confirmation). If this was a Flash or website change, it's likely that we'd see similar volume changes in beta/release.
Flags: needinfo?(mconley)
Created attachment 8805166 [details]
Plugin crash rate graph on Aurora

Here's the plugin crash rate on Aurora according to Telemetry: https://chutten.github.io/telemetry_crashes/

The spike you mention around September 23rd / 24th, I believe, makes an appearance on this chart. It's the big spike in the middle - but it drops off dramatically.

Interestingly, this was very shortly after 51 uplifted to Aurora (2016-09-20), according to https://wiki.mozilla.org/RapidRelease/Calendar.

So it looks like the crashes were real, but have tapered off. Either something got patched in Gecko (I'm not familiar with what that might be), or something got patched in Flash, or something got patched on some website somewhere.
Flags: needinfo?(mconley)
The only change to auto-submission in the last release is that we started showing users the "Submit Unsubmitted Crash Reports" notification, for which users can opt-in to auto-submission.
(In reply to Mike Conley (:mconley) - (digging out of review / needinfo backlog) from comment #68)
> The only change to auto-submission in the last release is that we started
> showing users the "Submit Unsubmitted Crash Reports" notification, for which
> users can opt-in to auto-submission.

Did we enable auto-submission? Which bug? I see bug 1270553 is still open.
(In reply to Marco Castelluccio [:marco] from comment #69)
> (In reply to Mike Conley (:mconley) - (digging out of review / needinfo
> backlog) from comment #68)
> > The only change to auto-submission in the last release is that we started
> > showing users the "Submit Unsubmitted Crash Reports" notification, for which
> > users can opt-in to auto-submission.
> 
> Did we enable auto-submission? Which bug? I see bug 1270553 is still open.

Bug 1287178 is where we exposed the preference and the button in the notification bar to allow users to auto-submit backlogged crash reports.

Comment 71

8 months ago
Are we agglomerating all bugs on this topic here, even after more than 4 years?
(Reporter)

Comment 72

8 months ago
Suburp: if you have specific steps to reproduce one of these crashes, please keep it separate. But if there's a bug that's "there are lots of crashes at F139..." or "I crashed once at F139... but can't reproduce" then they should be duped to this bug.
Comment hidden (spam)

Comment 74

5 months ago
Just got three crashes in one minute with this address:

https://crash-stats.mozilla.com/report/index/ca11fcbb-285e-4993-ba63-b54f92170215#tab-details

https://crash-stats.mozilla.com/report/index/1fb5c29f-3b4f-4d74-b40d-c57d52170215

https://crash-stats.mozilla.com/report/index/6663af52-4af6-4d3a-be6c-1d93c2170215

Comment 75

5 months ago
I would like to point out that this crash occurs with the browser just sitting there doing nothing, for instance, I go to bed and leave a couple of webpages open and come back and this crash has occurred. I generally have a couple of pages open and there is no Flash on either of them.

Comment 76

5 months ago
https://crash-stats.mozilla.com/report/index/e169b771-2365-4c58-b604-848262170215

Crash occurred with the browser just sitting there.

Comment 77

5 months ago
(In reply to Robert Lindsay from comment #76)
> https://crash-stats.mozilla.com/report/index/e169b771-2365-4c58-b604-
> 848262170215
> 
> Crash occurred with the browser just sitting there.

I suspect it is your ancient VGA (945G). From the crash report, the driver version you have is 6.14.10.4299 which dates from 2005. While it may not help much, try updating to v8.15.10.1930 which is from 2009 (https://tinyurl.com/zecsve2).

Comment 78

5 months ago
the same here (gtx650 drivers 2015 and 2016).
Component: Plug-ins → Flash (Adobe)
Product: Core → External Software Affecting Firefox
Duplicate of this bug: 1306910
Too late for firefox 52, mass-wontfix.
status-firefox52: affected → wontfix

Updated

3 months ago
Depends on: 1358147

Comment 81

3 months ago
[Tracking Requested - why for this release]:
status-firefox53: --- → affected
status-firefox54: --- → affected
status-firefox55: --- → affected
status-firefox-esr52: --- → affected
OS: Windows 7 → Windows

Comment 82

3 months ago
Showing results from 7 days ago

Windows 7 	15328 	46.4%
Windows 10 	12007 	36.3%
Windows 8.1 	3538 	10.7%
Windows Vista 	1449 	4.4%
Windows 8 	733 	2.2%
Windows XP 	6 	0.0%

Firefox 	55.0a1 	1338 	4.0% 	219
Firefox 	54.0a2 	5838 	17.7% 	567
Firefox 	54.0a1 	70 	0.2% 	3
Firefox 	53.0b99 	1335 	4.0% 	548
Firefox 	53.0b6 	702 	2.1% 	65
Firefox 	53.0 	493 	1.5% 	298
Firefox 	53.0b5 	410 	1.2% 	51
Firefox 	53.0b10 	358 	1.1% 	52
Firefox 	53.0b9 	258 	0.8% 	53
Firefox 	53.0b2 	189 	0.6% 	33
Firefox 	53.0b4 	181 	0.5% 	41
Firefox 	53.0b1 	175 	0.5% 	46
Firefox 	53.0b8 	40 	0.1% 	25
Firefox 	52.1.0esr 	735 	2.2% 	240
Firefox 	52.0.2 	8829 	26.7% 	7918
Firefox 	52.0.2esr 	3862 	11.7% 	1124
Firefox 	52.0.1 	1718 	5.2% 	817

Flash Version
25.0.0.148 	21897 	66.2%
25.0.0.127 	5602 	16.9%
24.0.0.221 	802 	2.4%
22.0.0.209 	551 	1.7%
24.0.0.194 	547 	1.7%
24.0.0.186 	461 	1.4%
16.0.0.228 	343 	1.0%
25.0.0.156 	335 	1.0%
23.0.0.207 	267 	0.8%
23.0.0.162 	175 	0.5%
18.0.0.209 	112 	0.3%
You need to log in before you can comment on or make changes to this bug.