If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

[adbe 3318859] crash in F_835252764

REOPENED
Unassigned

Status

External Software Affecting Firefox
Flash (Adobe)
P3
critical
REOPENED
5 years ago
6 months ago

People

(Reporter: Benjamin Smedberg, Unassigned)

Tracking

({crash})

unspecified
x86
Windows NT
crash

Firefox Tracking Flags

(firefox47 affected, firefox48 affected, firefox49 affected, firefox-esr45 affected, firefox50 affected, firefox51 affected, firefox52 wontfix, firefox53 affected)

Details

(Whiteboard: [Flash 11.3], crash signature)

Attachments

(1 attachment)

27.37 KB, application/octet-stream
Details
(Reporter)

Description

5 years ago
Topcrash found in the FlashPlayerPlugin* process after landing crashreporting for those processes in bug 769048.

report bp-18936d0f-4052-43a9-a2f9-a17402120708

More reports can be found at https://crash-stats.mozilla.com/report/list?product=Firefox&query_search=signature&query_type=contains&query=F_835252764&reason_type=contains&range_value=1&range_unit=weeks&hang_type=any&process_type=any&do_query=1&signature=F_835252764_________________________________________________________

Updated

5 years ago
Summary: crash in F_835252764_________________________________________________________ → crash in F_835252764
Whiteboard: [Flash-11.3] → [Flash-11.3][startupcrash]
Version: unspecified → 15 Branch
(Reporter)

Comment 1

5 years ago
This signature also appears to be a low-volume crash signature which has been around in prior versions of Flash, but has spiked significantly with protected mode.
When I look at the breakdown by Firefox version, it seems almost all of the Aurora crashes are startup crashes, but those on 13.0.1 and trunk have uptime.
(Reporter)

Comment 3

5 years ago
Here are the steps to reproduce, reported by Adobe:

Method 
1:Clear  firefox cache and Go to http://www.vudu.com/movies/#
2:Choose New Trailers
3:Player tailers one by one  -- Once the video starts playing, you can seek to 5-10 seconds before the end of the video, then let it complete.

Result:
Crash after playback  3 or more of the trailers (eg. play The Loved Ones->Madagascar3->Magic Mike->The Loved Ones)

Apparently this isn't an issue in Firefox 12.

I tried this but can't get Vudu to load at all... I'll try again on a different box. In the meantime, Marcia is it possible to get other people to try this and get a nightly regression window? Specifically checking the nightlies before and after bug 90268 landed, since it's the obvious candidate.
Keywords: regressionwindow-wanted
(Reporter)

Comment 4

5 years ago
Not sure whether this is revelant. I ran these STR in nightly under a debugger and got a different stack trace:

>	NPSWF32_11_3_300_265.dll!F1046602669___________________________()  Line 2423	C++
 	NPSWF32_11_3_300_265.dll!F1092009539__________()  Line 3811	C++
 	NPSWF32_11_3_300_265.dll!F661466345____________________()  Line 3756	C++
 	NPSWF32_11_3_300_265.dll!NPP_SetWindow()  Line 976	C++
 	FlashPlayerPlugin_11_3_300_265.exe!F_952353214____________________()  Line 200	C++

The disassembly is:

--- F_1885880_________________________________________________________ ---------
675623C0  mov         edx,dword ptr [ecx+4Ch]  
675623C3  mov         eax,dword ptr [esp+4]  
675623C7  mov         dword ptr [eax+4],edx  
675623CA  mov         edx,dword ptr [ecx+4Ch]  
675623CD  test        edx,edx  
675623CF  je          F1046602669___________________________+14h (675623D4h)  
* 675623D1  mov         dword ptr [edx+8],eax   <--crash here, EDX = 00D66660
675623D4  mov         dword ptr [ecx+4Ch],eax  
675623D7  ret         4

0xD66660 is accessible but readonly memory, this is as access-violation-write
(Reporter)

Comment 5

5 years ago
Oddly, following these steps I can consistently reproduce the F1046602699 crash when in a debugger, and the F_835252764 signature when running Firefox normally with the crash reporter.
(Reporter)

Comment 6

5 years ago
hrm! I crashed with crash report https://crash-stats.mozilla.com/report/index/bp-9386a084-5e5d-4c65-84c1-48adc2120717 which says that the stack is:

0: F_835252764_________________________________________________________ (F_1885880_________________________________________________________:2423)
1: F1018235994______________________________________ (F_1963533604__________________________________________________:3810)
2: F_1417225323______________________________________ (F_1963533604__________________________________________________:3756)
3: NPP_SetWindow (F_1417895389__________________________________________________________________:976)

MSVC says the stack for the same crash is:

>	NPSWF32_11_3_300_265.dll!F1046602669___________________________()  Line 2423	C++
 	NPSWF32_11_3_300_265.dll!F1092009539__________()  Line 3811	C++
 	NPSWF32_11_3_300_265.dll!F661466345____________________()  Line 3756	C++
 	NPSWF32_11_3_300_265.dll!NPP_SetWindow()  Line 976	C++

So the source/line information matches, but the function names don't. This is confusing; it must be something about the obfuscation technique?
Adding qawanted, but I will take a look at the STR in Comment 3 and see if I can help find a regression range.
Keywords: qawanted
QA Contact: mozillamarcia.knous
Using the latest nightly in the lab (Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Firefox/17.0), I keep freezing while the video is buffering - http://www.vudu.com/movies/# and playing the three stooges video.

I will try another machine next.
So far on several different Windows 7 machines I have not been able to reproduce the crash using the STR in Comment 3.
(Reporter)

Comment 10

5 years ago
Hrm, I can almost always reproduce it, though it sometimes requires repeating the trailer load 10 or more times. It was easier to reproduce when I tied all the processes to a single CPU to emulate a single-CPU system. Perhaps it's easier to see with slower hardware?
I can reproduce the bug in today's Nightly and Windows 7. I followed the steps in #3. 
The crashes are: 

https://crash-stats.mozilla.com/report/index/bp-0d4bba26-adab-4ab5-858a-bdcfd2120720

https://crash-stats.mozilla.com/report/index/bp-345cfca6-84c0-4d45-b9ba-6d48a2120720

https://crash-stats.mozilla.com/report/index/bea313b5-cd9d-4abd-9d55-0b4682120720

https://crash-stats.mozilla.com/report/index/bp-0d4bba26-adab-4ab5-858a-bdcfd2120720

https://crash-stats.mozilla.com/report/index/bp-0d9d5a07-b379-4091-982f-877e42120720

https://crash-stats.mozilla.com/report/index/bp-67fe1dc8-5a7a-4ee8-82a8-2a3822120720

https://crash-stats.mozilla.com/report/index/bp-0d4bba26-adab-4ab5-858a-bdcfd2120720

https://crash-stats.mozilla.com/report/index/bp-4f2bc468-c1f9-4453-8760-875802120720

Graphics
       
Adapter DescriptionSiS Mirage 3 GraphicsVendor ID0x1039Device ID0x6351Adapter RAMUnknownAdapter DriversSISGRUMD SiSClone SiSFunc SiSKrl SiSGlvDriver Version7.14.10.5290Driver Date12-15-2010Direct2D EnabledBlocked for your graphics card because of unresolved driver issues.DirectWrite Enabledfalse (6.1.7600.16972)ClearType ParametersGamma: 2200 Pixel Structure: RGB ClearType Level: 0 Enhanced Contrast: 400 WebGL RendererBlocked for your graphics card because of unresolved driver issues.GPU Accelerated Windows0. Blocked for your graphics card because of unresolved driver issues.

I have a 6 years old computer, it's quite fast but....
I apologize to all. My above comment had been done using the wrong Flash version.

I updated to the correct one and I can still reproduce the bug.

https://crash-stats.mozilla.com/report/index/bp-6d1220f4-a172-4358-ae37-0a57b2120720

https://crash-stats.mozilla.com/report/index/bp-4f2bc468-c1f9-4453-8760-875802120720

https://crash-stats.mozilla.com/report/index/bp-a330b113-d234-4f27-96ed-6c1a22120720

https://crash-stats.mozilla.com/report/index/bp-9bed73a6-7724-4b0f-a70c-c0fbf2120720

https://crash-stats.mozilla.com/report/index/bp-a330b113-d234-4f27-96ed-6c1a22120720

https://crash-stats.mozilla.com/report/index/bp-da98f3df-a691-457b-ad52-9d28e2120720


Same computer, same graphics of course.

Comment 13

5 years ago
It's #6 top plugin crasher in 15.0b1.
Keywords: topcrash
(Reporter)

Comment 14

5 years ago
This crash should be fixed in Flash 11.3.300.268.
Gabriela did some preliminary testing last evening and her testing indicates that .268 did address this crash. She indicated she would also do some additional testing this evening.
Crashes were reduced in 11.3.300.268, but are still around with .270 in smaller numbers.

Comment 17

5 years ago
our internal bugs (#3286062/3219934) have been marked resolved fixed in Dolores 11.4.  we shipped 11.4.402.265 on 8/21.  closing.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED

Comment 18

5 years ago
There are still crashes in 11.4.402.265. Here is the breakdown per Flash version:
11.3.300.262 	32.172 % 	157
11.3.300.265 	26.844 % 	131
11.3.300.271 	12.295 % 	60
11.3.300.257 	9.836 % 	48
11.4.402.265 	7.582 % 	37
11.4.400.252 	3.074 % 	15
11.1.102.55 	1.844 % 	9
11.4.400.231 	1.639 % 	8
11.3.300.268 	1.434 % 	7
11.3.300.231 	0.82 % 	        4
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 19

5 years ago
BTW, you can find this breakdown at the bottom of the Signature Summary tab in https://crash-stats.mozilla.com/report/list?signature=F_835252764_________________________________________________________

Comment 20

5 years ago
thanks.  bug#3318859 has been opened>for review in ellis...
Summary: crash in F_835252764 → [adbe 3318859] crash in F_835252764

Comment 21

5 years ago
Any additional reproducible scenarios would be extremely helpful in getting this resolved.

Comment 22

5 years ago
It's #64 top crasher in 15.0.1.
Keywords: topcrash
This is still present in latest builds for last 4 weeks at it shows in Soccoro.

Version:  Percentage      Nr.Crashes

18.0.1 	   35.625 %	      1404
18.0.2 	   31.337 %	      1235
19.0b4 	    5.303 %	       209
19.0b3 	    3.349 %	       132
19.0b6 	    3.324 %	       131
19.0b5 	    3.096 %	       122
19.0b2 	    2.614 %            103

Comment 24

5 years ago
MarioMi: As this is a crash in Flash, the versions of Firefox it happens in are not very useful. What is more useful is which versions of Flash it happens in.

Comment 25

5 years ago
There continues to be traffic on this signature for current Flash Player versions.  It happens under extreme low memory conditions.  Our audit of related code did not reveal opportunities for a blind fix.

I've pulled some 11.6 crash reports and added them to our bug in an attempt to get some new traction on this, but reproducible steps would be incredibly helpful as we're hitting the end of what we can glean from the stacks.
Keywords: steps-wanted
Priority: -- → P2
Whiteboard: [Flash-11.3][startupcrash] → [Flash-11.3][Flash-11.6][startupcrash]

Comment 26

5 years ago
Let's keep the Flash 11.6 whiteboard for regressions in Flash 11.6.
Whiteboard: [Flash-11.3][Flash-11.6][startupcrash] → [Flash-11.3][startupcrash]
There are still thousands crashes in last 4 weeks via soccoro, most of them on 11.6:

Flash Version	Percentage	Number Of Crashes
11.6.602.180	 61.838 %	      2369
11.3.300.262	   9.58 %	       367
11.3.300.265	   7.93 %	       304
11.3.300.257	   7.70 %	       295

Comment 28

4 years ago
Matt Talistu noticed that I had put these comments in the wrong bug.  They belong here. Sorry for the confusion!

From Magnus:
It would be useful to have the call stack for the PIC process.  The current attached dumps are in the sandbox process.  Difficult to know what's going on as Jimson said earlier in the notes.

From Jimson: 
Seems to be due to a corrupt AbortUnwindList. May be because we don't remove the last AbortUnwindObject from the list. This would indicate that the player didn't invoke RemoveAbortUnwindObject() for one of the added objects. However, no missing invocations stand out.
Priority: P2 → P3

Comment 29

4 years ago
It's currently #2 top Flash crasher in Flash 11.8 for current Firefox channels: https://crash-stats.mozilla.com/query/query?product=Firefox&version=Firefox%3A23.0a1&version=Firefox%3A22.0a2&version=Firefox%3A21.0b6&version=Firefox%3A20.0.1&query_search=signature&query_type=contains&process_type=plugin&hang_type=any&plugin_field=filename&plugin_query_type=exact&plugin_query=NPSWF32_11_8_800_42.dll&do_query=1
Whiteboard: [Flash-11.3][startupcrash] → [Flash 11.3][Flash 11.8]

Comment 30

4 years ago
Created attachment 748513 [details]
dmp/extra file and dxdiag

https://www.youtube.com/watch?v=o9GLl6kI4hQ
switching from fullscreen to normal mode with flash player hardware acceleration (hwa) enabled (with hwa disabled I don't have those crashes)

Comment 31

4 years ago
I forgot the crash report-id
https://crash-stats.mozilla.com/report/index/bp-3cbc80df-a259-4dd4-a55c-803bb2130512

Comment 32

4 years ago
https://bugzilla.mozilla.org/show_bug.cgi?id=774281#c17

fyi: Both reports were created at the same time and belong together (correlation).
2 reports for 1 action
Thanks MrX1980. How reproducible are these steps for you?

Georg, is there anything actionable for you in comment 30-32? If not, what would help here? A Firefox regression window? A Flash regression window?
Keywords: qawanted

Comment 34

4 years ago
While there's a common signature in play, my guess is that the huge uptick we're seeing with Flash Player 11.8 has a unique root-cause.  

We reopened this last week as 3550712 and are currently investigating.  There has been persistent low-volume activity on this signature for years that I don't believe will be resolved by this fix.  I expect we'll have a fix for the immediate issue in a future Flash Player 11.8 beta build, and will be looking for parity with Flash Player 11.7 as our short-term success case.

Updated

4 years ago
Summary: [adbe 3318859] crash in F_835252764 → [adbe 3318859][adbe 3550712 for Flash 11.8 spike] crash in F_835252764

Comment 35

4 years ago
a) switching from fullscreen to normal mode = always
b) scrolling after playing = once (until yet)

Comment 36

4 years ago
I can repro this, thanks!
(In reply to MrX1980 from comment #35)
> a) switching from fullscreen to normal mode = always
I confirm this with Flash 11.8.800.50 on FF 21, Win 7 x64.
Keywords: steps-wanted

Updated

4 years ago
Keywords: reproducible
Jeromie, do you need us to find a Firefox regression window or do you have sufficient information to go on?

Comment 39

4 years ago
We believe this is fixed on our side, but we had a nasty installer bug that prevented us from shipping the Beta last week.  That issue is also resolved, and we should be on track for shipping a beta this week with this fix.

Comment 40

4 years ago
(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #38)
> Jeromie, do you need us to find a Firefox regression window or do you have
> sufficient information to go on?
The regression window is for adbe 3318859 (Firefox's fault), not adbe 3550712 (Flash-11.8 fault).

Comment 41

4 years ago
(In reply to Jeromie Clark from comment #39)
> We believe this is fixed on our side
It's back to Flash 11.7 volume in Flash 11.8.800.64: around #30 Flash crasher in current Firefox channels.
Keywords: reproducible
Summary: [adbe 3318859][adbe 3550712 for Flash 11.8 spike] crash in F_835252764 → [adbe 3318859] crash in F_835252764
Whiteboard: [Flash 11.3][Flash 11.8] → [Flash 11.3][Flash 11.8][fixed in Flash 11.8.800.64]

Comment 42

4 years ago
We shipped a blind fix for this in 11.8.800.64, and I see a sharp reduction in frequency as compared to 11.8.800.50.  The low volume in 11.8.800.75 is early, but also supports a positive resolution. 

I've closed 3550712 as Fixed.  

In terms of the baseline issue (3318859), I think we're stuck on the issues in Comment 28

Comment 43

4 years ago
We really should have had a separate bug for the 11.8 spike to have it cleanly tracked separately. But what's done is done, so we need to live with having both issues in this bug.

Comment 44

4 years ago
Removing the whiteboard that biases queries with Flash 11.8.
Whiteboard: [Flash 11.3][Flash 11.8][fixed in Flash 11.8.800.64] → [Flash 11.3]
Duplicate of this bug: 1185937
We're still hitting this with Flash 19, but I don't think finding a regression window from 2012 is going to help us much in resolving it at this point.
Keywords: regressionwindow-wanted
Crash volume for signature 'F_835252764_________________________________________________________':
 - nightly(version 50):5 crashes from 2016-06-06.
 - aurora (version 49):28 crashes from 2016-06-07.
 - beta   (version 48):647 crashes from 2016-06-06.
 - release(version 47):1304 crashes from 2016-05-31.
 - esr    (version 45):31 crashes from 2016-04-07.

Crash volume on the last weeks:
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly       1       1       1       0       0       1       0
 - aurora        3       9       3       2       5       4       1
 - beta         91     109     111      84      81      61      77
 - release     204     175     195     194     159     155     155
 - esr           5       5       3       4       3       1       1

Affected platform: Windows
status-firefox47: --- → affected
status-firefox48: --- → affected
status-firefox49: --- → affected
status-firefox50: --- → affected
status-firefox-esr45: --- → affected
Crash volume for signature 'F_835252764_________________________________________________________':
 - nightly (version 51): 2 crashes from 2016-08-01.
 - aurora  (version 50): 10 crashes from 2016-08-01.
 - beta    (version 49): 321 crashes from 2016-08-02.
 - release (version 48): 195 crashes from 2016-07-25.
 - esr     (version 45): 38 crashes from 2016-05-02.

Crash volume on the last weeks (Week N is from 08-22 to 08-28):
            W. N-1  W. N-2  W. N-3
 - nightly       0       0       0
 - aurora        2       4       2
 - beta         83     121      82
 - release      57      52      42
 - esr           1       3       3

Affected platform: Windows

Crash rank on the last 7 days:
             Browser   Content Plugin
 - nightly                     #63
 - aurora                      #70
 - beta                        #20
 - release                     #23
 - esr                         #83
status-firefox51: --- → affected
Crash volume for signature 'F_835252764_________________________________________________________':
 - nightly (version 54): 0 crashes from 2017-01-23.
 - aurora  (version 53): 1 crash from 2017-01-23.
 - beta    (version 52): 117 crashes from 2017-01-23.
 - release (version 51): 259 crashes from 2017-01-16.
 - esr     (version 45): 99 crashes from 2016-08-10.

Crash volume on the last weeks (Week N is from 02-06 to 02-12):
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly       0       0
 - aurora        0       0
 - beta         75      20
 - release     156      54       0
 - esr           3       6       5       3       2       0       0

Affected platform: Windows

Crash rank on the last 7 days:
           Browser   Content   Plugin
 - nightly
 - aurora                      #170
 - beta    #5824               #62
 - release                     #23
 - esr                         #109
status-firefox52: --- → affected
status-firefox53: --- → affected
Component: Plug-ins → Flash (Adobe)
Product: Core → External Software Affecting Firefox
Version: 15 Branch → unspecified
Mass wontfix for bugs affecting firefox 52.
status-firefox52: affected → wontfix
You need to log in before you can comment on or make changes to this bug.