596441 - Loading site causes CPU and Kernel Utilization to explode stalling the Firefox GUI almost completely

Status: RESOLVED WORKSFORME

(Core Graveyard :: Plug-ins, defect, P1)

Description

[CruNcher]

User-Agent:       Mozilla/5.0 (Windows NT 5.1; rv:2.0b7pre) Gecko/20100914 Firefox/4.0b7pre
Build Identifier: Mozilla/5.0 (Windows NT 5.1; rv:2.0b7pre) Gecko/20100914 Firefox/4.0b7pre

http://img801.imageshack.us/img801/566/namorokafine.png    (Namoroka Nightly)
http://img831.imageshack.us/img831/6941/minefieldprob.png (Minefield Nightly and starting with Beta 2)

http://forums.mozillazine.org/viewtopic.php?f=23&t=1990331

Reproducible: Always

Steps to Reproduce:
1. Turn of OOPP (to only need to measure the main Firefox Process)
2. Load the site http://hcm.24h.com.vn/
3. Watch what happens in the CPU utilization and Kernel times of Firefox.exe
Actual Results:  
Windows CPU and Kernel Utilization explode causing the Firefox GUI to almost become unresponsive to any inputs

Expected Results:  
CPU Utilization should be High (due to the lot of flash adds) but not constantly fully utilizing 1 core as the repainting causes it to spike here and there, driven by a Javascript.  

Could be that the Gif Animation CPU Utilization plays overall into the CPU Utilization part https://bugzilla.mozilla.org/process_bug.cgi

Comment 1

[CruNcher]

Attachment: CPU and Kernel Explosion

Showing what happens after site load starting with Firefox 4.0 Beta 2

Comment 2

[Timothy Nikkel (:tnikkel)]

There is flash on that page with wmode=transparent, so this might be a dupe of bug 586162.

Comment 3

[CruNcher]

Jep this and the Gif Animation Utilization Problem seem to play a major role here :) https://bugzilla.mozilla.org/show_bug.cgi?id=595671

Comment 4

[CruNcher]

Jep Timothy i isolated all the stuff and it seems the GIF Problem plays no role here it is the wmode=transparent problem only most probably

Comment 5

[CruNcher]

Ehh great they fixed the page so the bug is obviously not there anymore though i have made a local testcase for this :) i exactly pinpointed which flash was causing this also the overlaying wmode=transparent pavilion light flash animation on the left and right side layer, that is now white instead.

Comment 6

[CruNcher]

http://roosterteeth.com/archive/?id=1431 in bug https://bugzilla.mozilla.org/show_bug.cgi?id=586162 is also a nice testcase for it not as heavy but it shows the same problems.

Comment 7

[Scoobidiver (away)]

Cruncher,
Can I have the graphic info in the "about:support" page ?

Comment 8

[CruNcher]

Graphics
      
Adapter Description: NVIDIA GeForce 9800 GT  
Vendor ID: 10de
Device ID: 0614
Adapter RAM: Unknown
Adapter Drivers: nv4_disp
Driver Version: 6.14.12.6063
Driver Date: 9-10-2010
Direct2D Enabled: false
DirectWrite Enabled: true
GPU Accelerated Windows: 1/1 Direct3D 9

also gonna upload the testcase

Comment 9

[CruNcher]

Attachment: Causes Extreme Strees on Minefield GUI becomes unresponsive

Testcase Semi Local (dynamically loads flash adds via javascript from the server)

Comment 10

[Scoobidiver (away)]

With the testcase, here below is the CPU usage on my computer under Windows 7 with a T4300 processor :
* 65 %, whatever the HW accel, the dom.ipc.plugins.enabled value, the windows 7 theme (aero or classic)
* 30% in FF 3.6.10

I suspect something is wrong in Core / Layout.
The issue appeared between 4.0 beta 1 and beta 2.

Comment 11

[Timothy Nikkel (:tnikkel)]

I think this is the known issue with wmode=transparent flash. It should be much improved when bug 596451 is fixed.

Comment 12

[Benjamin Smedberg]

I think that depends on whether we are going to have to keep doing alpha
recovery for transparent plugins. I haven't finished that research yet.

Comment 13

[Jim Mathies [:jimm]]

This appears to be addressed by async plugin painting. Please reopen if you're still seeing the problem in a nightly.

Comment 14

[CruNcher]

Nope the Performance issues aren't fixed yet the difference between Namoroka and Minefield still exists Minefield is extremely overloading the CPU what Namoroka doesn't not sure though if this is mainly a Async Plugin issue only it seems many factors come together on this testcase.

Comment 15

[CruNcher]

and by the things internally exploding inside Minefield java, css or i-frame or a combination of those the GUI response get heavily affected to the point where interaction is hardly possible anymore.

Comment 16

[Ed Morley [:emorley]]

Using the testcase from comment 9, Win7 with Aero enabled, clean profile with hardware accel both on (D3D9, no D2D) and off, have generated regression range...

Last good nightly: 2010-08-27 First bad nightly: 2010-08-28

Pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=e1d55bbd1d1d&tochange=6e3f6d18c124

Perhaps related to the two GIF bugs, given the similar regression range? (Bug 615063, Bug 595671)

Comment 17

[Timothy Nikkel (:tnikkel)]

Ed, what happens if you test without aero glass?

Comment 18

[Ed Morley [:emorley]]

Using:
- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b8pre) Gecko/20101128 Firefox/4.0b8pre
- H/w acceleration turned off in prefs.
- Testcase from comment 9
- OOPP left on (unlike the OP's report)
- My normal firefox profile (not clean, but adblock was turned off, otherwise blocked half the flash on the testcase)

CPU usage as reported by task manager.

With Win7 aero transparencies enabled:
- firefox.exe = ~35-45%
- plugin-container.exe = ~20-25%

With "Win7 Basic" theme, ie: aero transparencies disabled:
- firefox.exe = ~20-30%
- plugin-container.exe = ~20-25%

CPU usage varied greatly second by second, depending on:
- What part of the page was currently shown
- Whether scrolling had just taken place
- Whether any of the flash adverts had just been rolled over with the cursor

Comment 19

[Scoobidiver (away)]

CPU usage higher with aero theme than with basic theme is bug 598584.
I think it is a dupe.

Comment 20

[Timothy Nikkel (:tnikkel)]

Ed, do both of those cases (aero transparent and not transparent) have the same regression range for you?

Comment 21

[Ed Morley [:emorley]]

The regression range in comment 16 (Aug 27th-28th) was with aero enabled.

Using "Windows 7 Basic" theme (so no transparency), the testcase from comment 9 and layers.accelerate.none = true...
2009-12-16: 20-30% (no OOPP)
2010-06-08: Fx 6-13%, plugin: 14-30% = total ~20-40%
2010-07-21: Fx 13-25%, plugin: 13-30% = total ~25-55%
2010-08-12: Fx 14-28%, plugin: 15-40% = total ~25-70%
2010-08-23: Fx 16-29%, plugin: 14-40% = total ~30-70%
2010-08-25: Fx 17-30%, plugin: 15-40% = total ~30-70%
2010-08-26: Fx 15-31%, plugin: 14-42% = total ~30-75%
2010-08-27: Fx 17-31%, plugin: 13-40% = total ~30-70%
2010-08-28: Fx 18-30%, plugin: 18-40% = total ~35-70% (GUI sluggish)
2010-09-03: Fx 20-35%, plugin: 25-40% = total ~40-75% (GUI sluggish)

Going by purely whether the GUI is responsive or not, the regression range appears to be the same with both aero on and off.

As for general CPU usage, it seems as clear as mud. There are several points at which the usage jumps in the dates above, but it's hard to tell whether this is just due to the inaccuracy of the readings or an actual issue.

I'm not entirely convinced this is an ideal test-case, since page includes animated GIFs as well as lots of flash, so the GIF issues may be covering up underlying problems with flash plugins and/or OOPP. It may be best to isolate just the flash on that page (or say just use a YouTube video) and see how that affects things.

Also, I wasn't sure whether I should have been testing with OOPP on or off, since whilst comment 1 says off, it makes the results harder to interpret & the GUI stalling problem appears to exist the same regardless of OOPP state.

Comment 22

[Scoobidiver (away)]

So this bug and bug 598584 have the same regression range.
In this range, there are a lot of Graphics and Layout patches.

Comment 23

[CruNcher]

Attachment: Testcase no Gif loading

Hi added cleaned up testcase, to give better cleaner results avoiding bug https://bugzilla.mozilla.org/show_bug.cgi?id=595671
though i guess it doesn't play much of a overhead role here most gif where non animated.

Comment 24

[Benjamin Smedberg]

Testing with OOPP off is worthless, really. The codepaths are different and we're not going to ship in that configuration.

Comment 25

[Benjamin Smedberg]

Bug 596451 landed on 10-November, and I don't see any real GUI lockups/slowdowns with the zipped testcase in a nightly. Can somebody confirm?

Comment 26

[CruNcher]

Sorry for the late reply but work is killing me soon vacation though yeah :)
i can confirm that finally the problem is gone Kernel Utilization is back in Namoroka regions :D Now the Gif issue and 2 major performance issues are covered then some Flash IPC Performance issues (frame skips on Silverlight and Flash Video) and Firefox 4.0 is back @ least Performance wise :)

Comment 27

[CruNcher]

Everyone who worked on fixing this great work it took long but now its over from beta2 to beta9 :) i guess it was now a major performance drawback.

Comment 28

[CruNcher]

And yes the problem is also gone with IPC off and still i don't understand why testing with OFF in this case here should have been a problem, because the heart of this issue was a general performance issue that is now fixed and had nothing really todo with IPC though there are now still IPC Performance issues on the list that need to get fixed.

Comment 29

[Emanuel Hoogeveen [:ehoogeveen]]

Well, the non-IPC path still doesn't have asynchronous layer-based plugin painting as far as I know (added for IPC on Windows in bug 596451), so disabling OOPP might actually make things worse in theory - in practice perhaps the remaining issues on either side end up compensating each other, I don't really know.

Comment 30

[Joshua Mitchell (Inactive)]

Issue is resolved - clearing old keywords - qa-wanted clean-up