Closed Bug 530989 Opened 15 years ago Closed 15 years ago

crashes [@ NPSWF32.dll@0x136a29] (in Flash 10.0.32.18), many on yoville facebook app

Categories

(Core Graveyard :: Plug-ins, defect)

Product:
Core Graveyard
Component:
Plug-ins
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(status1.9.2 beta5-fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
status1.9.2 --- beta5-fixed

People

(Reporter: dbaron, Unassigned)

References

Details

(Keywords: crash, topcrash, verified1.9.2, Whiteboard: [crashkill][#1 Firefox 3.6b4 topcrash])

Crash Data

The #2 topcrash in 3.6b3 (and likely to be #1 in 3.6b4) is crashes at NPSWF32.dll@0x136a29 .  That dll offset is an offset into Flash 10.0.32.18, which is the current released flash version.

The crashes are not on the main thread, although it's not uncommon for flash to be on the stack on the main thread as well.

The most common URLs are:
     44 http://apps.facebook.com/yoville/index.php?bypass_reminder=1
     19
     12 http://apps.facebook.com/yoville/index.php?poe=1
     10 http://apps.facebook.com/yoville/index.php?sgskip=1&bypass_reminder=1

These crashes do not seem to be present in any significant numbers in 3.5.* (there are a very small number), but they're extremely common in 3.6 betas.  (Of the ones that are on 3.5.5, none of the URLs are yoville.)
Flags: blocking1.9.2?
We see more total crashes with "yoville" in the URL field on 3.6b3 than we do on 3.5.5.
Signature	NPSWF32.dll@0x136a29
UUID	8fd54e19-9f6a-442c-8488-c872c2091124
Time 	2009-11-24 21:57:59.759432
Uptime	5946
Last Crash	5948 seconds before submission
Product	Firefox
Version	3.6b3
Build ID	20091115182845
Branch	1.9.2
OS	Windows NT
OS Version	6.1.7600
CPU	x86
CPU Info	GenuineIntel family 6 model 15 stepping 11
Crash Reason	EXCEPTION_ACCESS_VIOLATION
Crash Address	0x180c301c
User Comments	The browser just keeps crashing constantly on this entire website all the time.
Processor Notes 	
Related Bugs

Crashing Thread
Frame 	Module 	Signature [Expand] 	Source
0 	NPSWF32.dll 	NPSWF32.dll@0x136a29 	
1 	winmm.dll 	TimerCompletion 	
2 	winmm.dll 	timeThread 	
3 	kernel32.dll 	BaseThreadInitThunk 	
4 	ntdll.dll 	__RtlUserThreadStart 	
5 	ntdll.dll 	_RtlUserThreadStart 	

NPSWF32.dll 	10.0.32.18 	C569605C15E5448BBFD9E1FE262649B61 	NPSWF32.pdb
Assignee: nobody → msintov
Severity: normal → critical
Timeless:  please don't assign release-blocking bugs to people without their consent (or, really, any bugs).

Also, can you explain why this didn't happen in 3.5?
Assignee: msintov → nobody
I don't have to.

Signature	NPSWF32.dll@0x136a29
UUID	d6459674-1338-43c0-b11b-52ca72091125
Time 	2009-11-25 07:54:59.28426
Uptime	217
Last Crash	2269 seconds before submission
Product	Firefox
Version	3.5.5
Build ID	20091102152451
Branch	1.9.1
OS	Windows NT
OS Version	5.1.2600 Service Pack 3
CPU	x86
CPU Info	GenuineIntel family 15 model 4 stepping 1
Crash Reason	EXCEPTION_ACCESS_VIOLATION
Crash Address	0x101c
User Comments	
Processor Notes 	
Related Bugs

OPEN

        * 530989 NEW crashes [@ NPSWF32.dll@0x136a29] (in Flash 10.0.32.18) in Firefox 3.6 betas, many on yoville facebook app

Crashing Thread
Frame 	Module 	Signature [Expand] 	Source
0 	NPSWF32.dll 	NPSWF32.dll@0x136a29 	
1 	winmm.dll 	TimerCompletion 	
2 	winmm.dll 	timeThread 	
3 	kernel32.dll 	BaseThreadStart 	

NPSWF32.dll 	10.0.32.18 	C569605C15E5448BBFD9E1FE262649B61 	NPSWF32.pdb
Summary: crashes [@ NPSWF32.dll@0x136a29] (in Flash 10.0.32.18) in Firefox 3.6 betas, many on yoville facebook app → crashes [@ NPSWF32.dll@0x136a29] (in Flash 10.0.32.18), many on yoville facebook app
Timeless, you know that a crash in one module doesn't mean that module alone, or at all, is to blame. If the same Flash plugin crashes on 3.6 and not on 3.5, that suggests a bug in Mozilla code. It could still be a Flash bug; it could be a pair of bugs cooperating to bite. But we should look to our own code first.

/be
brendan: sure, but this was an existence proof:

> Also, can you explain why this didn't happen in 3.5?

It crashes at the same address with the same stack in 3.5.5 with the same plugin version -- see the rest of comment 5.

Crashes from Flash generally will change as Flash authors write new applets or update/enhance old ones. They'll tickle some previously unknown crash and won't necessarily immediately discover that their swf is crashing the flash runtime.

Offhand, it sounds like it's audio related. - My w32 is incredibly rusty.

Most likely the yoville people changed something audioish in their flash which happens to exercise some interesting bit of Flash code. The Flash team will figure this out.

From our side, we can basically move plugins out of process and let Flash crash on its own.
Sure, there were crashes at that address in 3.5.5; I know that.  But, as I said in comment 0, there are *more* in 3.6b3, with many fewer users.  That's a regression.  (And that statistic works both for crashes-at-this-signature and crashes-with-"yoville"-in-the-URL, IIRC.)
(In reply to comment #7)
> Most likely the yoville people changed something audioish in their flash which
> happens to exercise some interesting bit of Flash code. The Flash team will
> figure this out.

No, because then we'd see the crashes in 3.5.5 as well.  I'm comparing crashes in 3.5.5 vs. 3.6b3 on the same day.

> From our side, we can basically move plugins out of process and let Flash crash
> on its own.

You know quite well that (a) we're already working on that and (b) there's no way we can do that for 3.6.

Please stop dragging this bug into covering other things than what it was filed about.
Or, in other words, compare the ratios of the numbers in the following two matrices:

Firefox 3.6b3 crashes on 2009-11-24 (unthrottled):

                                    Crashes at
                                       all             Crashes at
                                    signatures     NPSWF32.dll@0x136a29

           Crashes at any URL:       14927                515
Crashes with "yoville" in URL:          82                 54


Firefox 3.5.3 crashes on 2009-11-24 (throttled):

                                    Crashes at
                                       all             Crashes at
                                    signatures     NPSWF32.dll@0x136a29

           Crashes at any URL:      119703                  5
Crashes with "yoville" in URL:         129                  0
It would be useful to get steps to reproduce here.  Then we could:

 * bisect to figure out when the problem started on the Mozilla side

 * probably have a better chance of getting help from Adobe
Keywords: qawanted
Whiteboard: [crashkill]
Whiteboard: [crashkill] → [crashkill][#1 Firefox 3.6b4 topcrash]
chofmann, can i get a list for testing ? :)
the high frequency sites seem to involve interaction.   here is a pretty good list that didn't require sanitization.   the high about:blank count might indicate leaving the page as part of the steps we should try.

 695 http://apps.facebook.com/yoville/index.php?bypass_reminder=1
 597 \N
 198 http://apps.facebook.com/yoville/index.php?sgskip=1&bypass_reminder=1
 134 about:blank
 126 http://combatarms.nexon.net/
 104 http://www.ustream.tv/discovery/live/sports
  65 http://profile.myspace.com/Modules/Applications/Pages/Canvas.aspx?appId=111238
  62 http://www.ustream.tv/discovery/live/all
  49 http://www.ustream.tv/
  48 http://dungeonfighter.nexon.net/
  41 
  34 http://www.baixaki.com.br/site/dwnld4866.htm
  33 http://www.baixaki.com.br/site/dwnld41034.htm
  28 http://www.baixaki.com.br/site/dwnld2657.htm
  28 http://passport.nexon.net/Login.aspx
  23 http://www.tagged.com/yoville.html
  22 http://www.emulinhabrasil.xpg.com.br/
  21 http://www.ustream.tv/discovery/live/entertainment
  21 http://www.baixaki.com.br/site/dwnld5576.htm
  20 http://www.baixaki.com.br/site/dwnld42942.htm
  19 http://www.orkut.com.br/Main#Home
  19 http://www.ebay.com/
  19 http://www.baixaki.com.br/site/dwnld33061.htm
  19 http://apps.facebook.com/onthefarm/index.php?ref=tab
  18 http://www.youtube.com/
  18 http://www.winamp.com/player/download/en-us
  18 http://www.narutowire.com/
  17 http://www.pingtest.net/
  17 http://stagevu.com/videos/Films_and_Movies
  17 http://home.myspace.com/index.cfm?fuseaction=user
  16 http://www.renren.com/SysHome.do
  16 http://www.baixaki.com.br/site/dwnld37053.htm
  16 http://m.www.yahoo.com/
  16 http://apps.facebook.com/yoville/index.php?src=xpromo&aff=zbar&crt=mafiawars
  16 http://apps.facebook.com/cafeworld/?ref=bookmark
  16 http://abc.go.com/watch/v/240273/241139/there-is-no-normal-anymore
  15 http://www.chip.de/
  14 http://www.boygeniusreport.com/
  14 http://thereifixedit.com/
  14 http://abc.go.com/watch/v/240273/241852/a-bright-new-day
  13 http://www.ustream.tv/discovery/live/gaming
  13 http://www.ulla.com/u/CHAT/webstart.php
  13 http://www.touslesdrivers.com/index.php?v_page=30&v_forum=0
Flags: blocking1.9.2? → blocking1.9.2+
Blocks: 532114
for some reason this signature has really jumped on 3.6b4.   in previous beta's it was making up around 1-2% of all crashes, but now its making up about 4% of all crashes.

here is a snapshot of 2009-11-30

release total-crashes  
                NPSWF32.dll@0x136a29-crashes  
                        pct.

total	233706	822	0.00351724
fx3015	50334	1	1.98673e-05
fx355	122547	3	2.44804e-05
*fx36b4*16576	695	*0.0419281*
fx36b3	2703	59	0.0218276
fx36b2	1193	18	0.015088
fx36b1	2776	39	0.014049
Is crash-per-user up, or did this get higher because we fixed other crashes so it became a greater percentage without changing frequency?
I can reproduce this 100% on my Win Vista Lab machine by loading http://www.ulla.com/u/CHAT/webstart.php and clicking around a few times. I am using Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2b5pre) Gecko/20091201 Namoroka/3.6b5pre (.NET CLR 3.5.30729).
jst asked me to test this using Beta 2 to see if I was able to reproduce the crash - using Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2b2) Gecko/20091108 Firefox/3.6b2 (.NET CLR 3.5.30729) I crash loading the site in Comment 16, but the crash report shows something different at the top of the stack: [@RtlDeleteCriticalSection ]

http://crash-stats.mozilla.com/report/index/bp-fcca692c-5ad2-41ab-baae-8a06d2091201 is my report.

http://tinyurl.com/y8dfe72 is the link to the crashes in [@RtlDeleteCriticalSection ] in beta 4.
Just to clarify you don't have actually do anything but load the site and wait - no clicking is required to reproduce the bug. I also forgot to add the lab machine is running 64 bit Win Vista.

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2b1) Gecko/20091019 Firefox/3.6b1 (.NET CLR 3.5.30729) crashes in the same stack as Comment 17 loading the site: http://crash-stats.mozilla.com/report/index/2d088844-9b86-474e-8e79-ca4ea2091201 is the report.

I also tested this on Windows 7 64 bit and it crashes using the latest Namoroka nightly, but with the latest version of flash on the labs.adobe.com site -> 10.1.51.45.
Using the same machine as in Comments 17 and 18, I installed the Alpha 1 - Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2a1) Gecko/20090806 Namoroka/3.6a1 (.NET CLR 3.5.30729). The site does not crash using Alpha 1, so I guess that narrows the window down to between Alpha and Beta 1.
(In reply to comment #15)
> Is crash-per-user up, or did this get higher because we fixed other crashes so
> it became a greater percentage without changing frequency?

overall crash volume, and crashes per 100 users is up on beta4
https://wiki.mozilla.org/CrashKill/Crashr#Release_3.6b4

trying to sort out possible reasons for that in bug 532114

looks like this bug could be a contributor, along with a changing dynamic in the make up of the beta tester pool and content that they might be hitting.
Sorry, yeah, I meant "is this-crash-times-per-user up?"
I worked on trying to isolate the regression range:

20091005->Works
20091006->Crash

Bug 520295 seems to be the only bug in that range that involved plugins in some way.

chofmann wanted me to do a screencast of the site, but I think if you load http://www.ulla.com/u/CHAT/webstart.php you can see that the page continually keeps reloading, and the crash happens after about the 4th reload.
(In reply to comment #22)
> 20091005->Works
> 20091006->Crash

Are those mozilla-central builds or mozilla-1.9.2 builds?

(Getting a regression range for both mozilla-central and mozilla-1.9.2 might narrow things down further.)

Shortly, I'll see if I can repro using your steps.
(In reply to comment #17)
> jst asked me to test this using Beta 2 to see if I was able to reproduce the
> crash - using Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2b2)
> Gecko/20091108 Firefox/3.6b2 (.NET CLR 3.5.30729) I crash loading the site in
> Comment 16, but the crash report shows something different at the top of the
> stack: [@RtlDeleteCriticalSection ]
> 
> http://crash-stats.mozilla.com/report/index/bp-fcca692c-5ad2-41ab-baae-8a06d2091201
> is my report.

Based on that, it looks like you're hitting bug 520639 then, which we also needed steps to reproduce for! :)
I was using 1.9.2 builds for the regression range in Comment 22. So it sounds as if I am hitting a different bug then according to Comment 24?
(In reply to comment #22)
> I worked on trying to isolate the regression range:
> 
> 20091005->Works
> 20091006->Crash
> 
> Bug 520295 seems to be the only bug in that range that involved plugins in some
> way.

I'm fairly sure that the checkin in that date range on 1.9.2 that "caused" the bug that Marcia hit here was the one for bug 514327, not 520295. In reality this problem has been around for a long time, it was just made significantly worse by bug 514327.
jst suggested to me that the fix for bug 520639 may have fixed this, and we haven't seen any of these crashes yet in today's builds, so it may well have.
I am no longer able to reproduce this using Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b5pre) Gecko/20091204 Namoroka/3.6b5pre and loading the site in Comment 16. To be thorough I also checked the Win Vista and Win XP latest nightlies and received the same result. Adding verified 1.9.2 keyword to this bug and removing qawanted.
Keywords: qawanted → verified1.9.2
Marking fixed (fixed by the fix for bug 520639).
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
I am using the latest Shiretoko release: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7pre) Gecko/20091204 Shiretoko/3.5.7pre (.NET CLR 3.5.30729) and appears I am still being affected by this bug.
But no automatic bug report is sent as the browser is not completely crashing, but just using a lot of CPU(40%) and becomes unresponsive on any page which contains a SWF file including youtube.com
By disabling the SWF plugin, the browser works fine. Strangly I noticed that Safe-Mode isn't disabling the plugins?! Even if I start in safemode the browser will still become unresponsive with SWF files unless I then go to Addons and disable it manually.
Using Shockwave Flash plugin version: 10.0.32.18
During the unresponsive time, examining the Firefox threads with ProcessExplorer shows:
NPSWF32.dll+0x14fdda containing NPSWF32.dll+0x14fcfb
I don't see why you think your problem is *this* bug; this bug is about a crash at a particular address in the Flash library (NPSWF32.DLL).  It sounds like you may also be having a problem that is related to Flash, but not this problem.
Crash Signature: [@ NPSWF32.dll@0x136a29]
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.