558647 - [meta] [OOPP] hang (crash report) in [@ KiFastSystemCallRet] [@mozilla::plugins::PluginModuleParent::ShouldContinueFromReplyTimeout ]

Status: RESOLVED INCOMPLETE

(Core Graveyard :: Plug-ins, defect)

Description

[Maini Marco]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a5pre) Gecko/20100410 Minefield/3.7a5pre (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a5pre) Gecko/20100410 Minefield/3.7a5pre (.NET CLR 3.5.30729)

Now it is ranked as #4 in crash stats for Firefox 3.7prea5.
Probably related with plugin such as Acrobat Reader and Flash
http://crash-stats.mozilla.com/report/index/bp-52386e46-faab-40fb-a188-1d27d2100411

Reproducible: Sometimes

Comment 1

[Maini Marco]

Link to crash report bp-6ad971f6-d2fd-4dad-836f-574f42100410

Comment 2

[Benjamin Smedberg]

This is the signature we get from the hang detector. We're going to need to do some more extensive analysis to break this down into individual bugs.

Comment 3

[juan becerra [:juanb]]

This is the #1 crasher in 3.6.3plugin1 and 3.6.4pre and the #4 on 3.7a5pre. No reproducible steps yet, but I will investigate tomorrow as I have seen this a couple of times in a VM.

Comment 5

[juan becerra [:juanb]]

One way I have been able to see related crashes on my vm is by:

1. Fresh install 3.6.4pre
2. Install Flash 10.r45
3. Install Quicktime (latest)
4. Restart
5. Play a youtube video
6. Kill the plugin-container process as the video is playing.

It will kill the whole browser. I have seen crashers like the following, but the signatures are not consistent: http://crash-stats.mozilla.com/report/index/138f4897-491f-4bdb-881c-1b6f42100413

Comment 6

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to comment #5)
> http://crash-stats.mozilla.com/report/index/138f4897-491f-4bdb-881c-1b6f42100413

This crash appears to be a separate, weird problem.  Summary of IRC discussion
 - the backtrace shows
     (1) |msg = new PBrowserStream::Msg_Write(offset, data, newlength);| threw bad_alloc (these could be (nothrow/fallible), separate bug)
     (2) the plugin host caught bad_alloc and blamed it on the plugin
     (3) the plugin host tried to show a prompt, but |new nsPrompt(aParent)| threw bad_alloc again
     (4) this unhandled exception terminated firefox-bin
 - juanb could easily repro in his VM "a" bad_alloc (not always (1)'s) after terminating plugin-container even though the VM wasn't under any apparent memory pressure.  Doubling the VM's allocated RAM lessened the frequency but didn't eliminate.
 - bsmedberg couldn't repro on Win7 in a recording VM with half the (original) allocated RAM of juanb's

I don't even have a WAG about the |bad_alloc|s juanb sees.  If we don't see them in the wild on real HW, I'm inclined to hand-wavingly blame them on a seam in The Matrix of Mac/VMWare Fusion 3.0.1/Windows 7.

Comment 7

[alex_mayorga]

FWIW I ended up here from about:crashes I use nightly build in real hardware and got these crashes:
http://crash-stats.mozilla.com/report/index/bp-96a9b9bf-9053-4a07-bd19-b79c92100412
http://crash-stats.mozilla.com/report/index/bp-96a9b9bf-9053-4a07-bd19-b79c92100412

I might try to reproduce again using comment #5 but I don't quite get what I need to do for step 6

Let me know if there's anything I might chip-in.

Comment 8

[alex_mayorga]

Crashing this morning too with Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.3a5pre) Gecko/20100414 Minefield/3.7a5pre ID:20100414050851
http://crash-stats.mozilla.com/report/index/bp-60a461b9-3006-4393-8055-b02872100415
I think these are related to opening PDF files from http://www.grsproadsafety.org/page-good_practice-347.html?gp_id=3 using Acrobat Reader 8.0 but the crashes are indeed pretty random and have only taken Minefield down completely a couple of times.

Comment 9

[Benjamin Smedberg]

Alex, this is a meta-bug for all hangs. If you are experiencing a crash that you can reproduce or kinda-reproduce, please file a bug listing what you were doing.

Also, if it "took Minefield down completely", that is a different bug that we should look at separately.

Comment 10

[IU]

(In reply to comment #9)
> Also, if it "took Minefield down completely", that is a different bug that we
> should look at separately.

Not really.  I've had the [@ KiFastSystemCallRet] crash take Minefield down completely, but it's hard to intentionally reproduce.

Comment 11

[Benjamin Smedberg]

Then it's a *different* and *more important* bug. *This* bug is about the hang detector, which will produce a crash reason of EXCEPTION_BREAKPOINT and should not take down the browser. If you have reports which took down the browser, please file them separately, even if they have the same signature listed here.

Comment 12

[IU]

Not to be a prick, but "EXCEPTION_BREAKPOINT" is the exact reason I got.  See here: http://crash-stats.mozilla.com/report/index/bp-459cfdf2-1b52-45db-aafd-47f0a2100408

Comment 13

[Benjamin Smedberg]

That's a plugin-side hang report. When the browser crashed did the standalone crash reporter appear?

In any case, we want steps to reproduce these hang reports: just not in this bug.

Comment 14

[IU]

(In reply to comment #13)
> When the browser crashed did the standalone crash reporter appear?

I don't remember (don't see one listed).  But if I crash again, I'll file a different bug (if I can find steps to reproduce).

Comment 15

[Gert-Paul van der Beek]

On http://www.rtl.nl/components/actueel/rtlnieuws/weer/miMedia/215962/216070.public.29597918.RTL_Weer_s1_a108.xml Firefox 3.6.4 is crashing all the time. Maybe this can help to find the cause of this bug.

It is a player for missed TV programs. The controls don't do anything, volume, full screen, etc.. After trying some control, waiting a few seconds, maybe right clicking and switching to another window, there always is a crash (but only in the plugin-part).

UA: Mozilla/5.0 (Windows; U; Windows NT 6.1; nl; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4

Comment 16

[Benjamin Smedberg]

Gert-Paul and anyone else who encounters this bug: if you have steps to reproduce a hang, please file another bug, and include the steps to reproduce as well as crash report IDs from about:crashes. We can't use this meta-bug to track each individual kind of hang report that comes in.

Comment 17

[chris hofmann]

from a sample of crashes from yesterday here are the top source lines of the stack for all the crashes that have this signature.  maybe this can help to sort out the possible frequency of the various stacks that are showing up.

172:crashdata chofmann$ awk -F\t '$8 ~ /3.6.4/ {print $20,$1}' 20100419* | sort | uniq -c | sort -nr | grep KiFastSystem
4034 hg:hg.mozilla.org/releases/mozilla-1.9.2:toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:7ef41676c6e2 KiFastSystemCallRet
 868  KiFastSystemCallRet
 247 hg:hg.mozilla.org/releases/mozilla-1.9.2:ipc/chromium/src/base/message_pump_win.cc:7ef41676c6e2 KiFastSystemCallRet
 128 hg:hg.mozilla.org/releases/mozilla-1.9.2:ipc/glue/WindowsMessageLoop.cpp:7ef41676c6e2 KiFastSystemCallRet
  35 hg:hg.mozilla.org/releases/mozilla-1.9.2:db/sqlite3/src/sqlite3.c:7ef41676c6e2 KiFastSystemCallRet
  30 hg:hg.mozilla.org/releases/mozilla-1.9.2:dom/plugins/PluginModuleChild.h:7ef41676c6e2 KiFastSystemCallRet
  16 hg:hg.mozilla.org/releases/mozilla-1.9.2:dom/plugins/PluginInstanceChild.cpp:7ef41676c6e2 KiFastSystemCallRet
  14 hg:hg.mozilla.org/releases/mozilla-1.9.2:ipc/chromium/src/base/platform_thread_win.cc:7ef41676c6e2 KiFastSystemCallRet
  13 hg:hg.mozilla.org/releases/mozilla-1.9.2:dom/plugins/PluginScriptableObjectChild.cpp:7ef41676c6e2 KiFastSystemCallRet
   7 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/libpkix/pkix/top/pkix_build.c:7ef41676c6e2 KiFastSystemCallRet
   4 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/libpkix/pkix/checker/pkix_policychecker.c:7ef41676c6e2 KiFastSystemCallRet
   3 hg:hg.mozilla.org/releases/mozilla-1.9.2:toolkit/crashreporter/google-breakpad/src/client/windows/crash_generation/crash_generation_client.cc:7ef41676c6e2 KiFastSystemCallRet
   3 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/pk11wrap/pk11pars.c:7ef41676c6e2 KiFastSystemCallRet
   3 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/libpkix/pkix_pl_nss/pki/pkix_pl_cert.c:7ef41676c6e2 KiFastSystemCallRet
   3 hg:hg.mozilla.org/releases/mozilla-1.9.2:nsprpub/pr/src/misc/prtrace.c:7ef41676c6e2 KiFastSystemCallRet
   3 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/jsapi.cpp:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:xpcom/string/src/nsTSubstring.cpp:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/ssl/ssl3con.c:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/pk11wrap/pk11pqg.c:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/pk11wrap/pk11cert.c:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/pk11wrap/pk11akey.c:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/certdb/secname.c:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:obj-firefox/dist/private/nss/pk11pars.h:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:nsprpub/lib/libc/src/plerror.c:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:netwerk/streamconv/converters/nsGopherDirListingConv.cpp:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:ipc/chromium/src/base/message_pump_win.h:7ef41676c6e2 KiFastSystemCallRet
   2 hg:hg.mozilla.org/releases/mozilla-1.9.2:dom/plugins/BrowserStreamChild.cpp:7ef41676c6e2 KiFastSystemCallRet
   2 f:\\dd\\vctools\\crt_bld\\self_x86\\crt\\src\\free.c KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:toolkit/crashreporter/google-breakpad/src/client/windows/handler/exception_handler.cc:f9ba867bf971 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/util/secasn1d.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/util/nssb64d.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/ssl/sslsecur.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/smime/cmsutil.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/smime/cmsarray.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/pki/trustdomain.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/pki/pkibase.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/pk11wrap/pk11skey.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/pk11wrap/pk11auth.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/nss/nssinit.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/libpkix/pkix_pl_nss/pki/pkix_pl_nameconstraints.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/libpkix/pkix_pl_nss/module/pkix_pl_httpdefaultclient.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/libpkix/pkix_pl_nss/module/pkix_pl_httpcertstore.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/libpkix/pkix_pl_nss/module/pkix_pl_aiamgr.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/libpkix/pkix/util/pkix_tools.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/cryptohi/seckey.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/certhigh/ocsp.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/certhigh/certhigh.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:security/nss/lib/certdb/stanpcertdb.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:obj-firefox/js/src/jsautokw.h:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:obj-firefox/ipc/ipdl/PPluginInstanceChild.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:nsprpub/pr/src/misc/prtpool.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:nsprpub/pr/src/misc/prtime.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:modules/libpref/src/prefapi.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:layout/style/nsStyleSet.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:layout/style/nsRuleNode.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/prmjtime.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/nanojit/Nativei386.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/nanojit/LIR.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/jsxml.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/jsvector.h:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/jstracer.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/jsregexp.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/json.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/jsfun.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:js/src/dtoa.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:jpeg/jmemmgr.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:ipc/chromium/src/base/atomic_ref_count.h:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:gfx/cairo/libpixman/src/pixman-mmx.c:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:dom/plugins/PluginModuleChild.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:content/events/src/nsEventStateManager.cpp:7ef41676c6e2 KiFastSystemCallRet
   1 hg:hg.mozilla.org/releases/mozilla-1.9.2:content/events/src/nsDOMEvent.cpp:7ef41676c6e2 KiFastSystemCallRet

Comment 18

[chris hofmann]

oops. should have made that an attachment.

Comment 19

[Mike Beltzner [:beltzner, not reading bugmail]]

We can't block on the meta bug, so once we've worked out some of the primary causes, those should be nominated as blockers.

Comment 20

[Michael Kraft [:morac]]

I got this on 3.6.4b1.  A flash video hung so I manually killed the plugin-container.exe process from the Windows XP SP3 task manager.  Then any time a video tried to play on http://cnettv.cnet.com/cloak-dagger-iphone-4g/9742-1_53-50086632.html it crashed with this stack:

http://crash-stats.mozilla.com/report/index/6b4e8233-163c-42eb-8d80-b6e872100422

Comment 21

[Johnny Stenback (:jst)]

Not blocking 1.9.3 on this meta bug. Nominate specific bugs as they're filed if they're worthy of blocking a release.

Comment 22

[Michael Kraft [:morac]]

Since upgrading to Flash 10.1, frequently Firefox 3.5.4 tells me Flash has crashed.  I've been running the Firefox 3.5.4 release candidate (which is the same as the release) for weeks so the only change was upgrading from Flash 9 to Flash 10.1.  It tends to happen when other processes are using a lot of CPU time (starving Firefox).

Here's a bunch of crash logs from Firefox:

http://crash-stats.mozilla.com/report/index/7ddc0c26-65e4-4430-9fc5-4371e2100624
http://crash-stats.mozilla.com/report/index/521ced93-86e9-4c43-ace8-8116e2100622
http://crash-stats.mozilla.com/report/index/5c78da0a-5c94-44c4-92c2-faf642100622

Comment 23

[(not currently active) Ted Mielczarek]

Those are hang reports. If your system is starved for CPU, then it's possible that Flash isn't responding to messages in time, so the browser is killing it because it thinks it's hung. Your Flash upgrade might have made this start happening. Perhaps Flash 10.1 uses more CPU for certain things.

Comment 24

[Michael Kraft [:morac]]

Is there a way to tell Firefox to give the plugin more time to complete its task?  

I'm getting this crash on hulu.com all the time now because the plugin process uses 99% of the CPU for a few seconds while the page loads and then Firefox kills it before it can finish.

Comment 25

[Michael Kraft [:morac]]

Never mind I found that I can change the setting with dom.ipc.plugins.timeoutSecs

Doesn't seem to make a difference though.  I managed to get Hulu to load in Safari (for Windows), but it took about 30 seconds and used 1 GB (yes GB) of RAM to load the page.  Looks like Flash 10.1 is just buggy.

Comment 26

[timeless]

michael: please file your own bug report, since the problem seems to be w/ the flash plugin, please use plugins:adobe flash for it. provide the incident ids and steps to reproduce.

Comment 27

[Benjamin Smedberg]

Resolving old bugs which are likely not relevant any more, since NPAPI plugins are deprecated.