Last Comment Bug 561818 - [OOPP] IPC severely degrades mouse responsiveness with windowless Flash video
: [OOPP] IPC severely degrades mouse responsiveness with windowless Flash video
Status: NEW
: perf
Product: Core
Classification: Components
Component: Plug-ins (show other bugs)
: Trunk
: x86 Windows XP
: -- normal with 15 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
: Benjamin Smedberg [:bsmedberg]
Mentors:
http://abcnews.go.com/Entertainment/v...
Depends on:
Blocks: slowui OOPP 558629
  Show dependency treegraph
 
Reported: 2010-04-26 11:31 PDT by IU
Modified: 2012-09-22 22:38 PDT (History)
27 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
-
needed
wanted


Attachments
windowless youtube test case (889 bytes, text/html)
2010-04-26 14:34 PDT, Jim Mathies [:jimm]
no flags Details
windowed youtube test case (1.12 KB, text/html)
2010-04-27 12:43 PDT, Jim Mathies [:jimm]
no flags Details

Description IU 2010-04-26 11:31:45 PDT
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a5pre) Gecko/20100426 Minefield/3.7a5pre (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a5pre) Gecko/20100426 Minefield/3.7a5pre

With IPC plugins enabled, responsiveness to mouse input degrades severely for windowless Flash video.  The patch for bug 558629 fixed the windowed case but left the windowless case unresolved.

With Flash Player 10.1 installed, this degradation in responsiveness can also
be observed when trying to pause a video and having it take several seconds
before the video is paused.  My STR focuses on the scroll wheel, but the issues
should have the same origin.

To demonstrate the issue, my STR requires installation of a program called
Volumouse.  It allows for audio volume control using shortcut key(s) + the
mouse wheel.  This is for illustration purposes only, as I know of no other
easy way to show the performance issue. Note: I'm on a 933 MHz P3; thus, the
relative impact of the degradation is really bad.


Reproducible: Always

Steps to Reproduce:
1. Download and install Volumouse (use the self-install executable):
http://www.nirsoft.net/utils/volumouse.html
2. Start Volumouse and configure it as follows:
a) Use the wheel when: Ctrl+Shift are down
d) Component: Play Control; Channels: All Channels; Steps: 2000
c) Set all the other "Use the wheel when:" options to "Disabled"
d) Run Volumouse application in high priority
e) OK
3. Create a new profile and launch Minefield with it
4. Disable IPC plugins and restart
5. Visit the linked URL
6. Once the advertisement finishes playing and the "View" video is playing, use
the hotkey from Step 2a and your mouse wheel to scroll volume up and down. 
Take note of the responsiveness
7. Close the tab with the video loaded or click the home button
8. Reenable IPC plugins and restart
9. Visit the linked URL
10. Again, once the advertisement finishes playing and the video is playing,
use the hotkey from Step 2a and your mouse wheel to change volume up and down. 
Note the relative degradation in responsiveness.
Comment 1 Benjamin Smedberg [:bsmedberg] 2010-04-26 12:19:59 PDT
What build did you test with? The second set of patches in the original bug should have fixed windowless.
Comment 2 IU 2010-04-26 12:31:57 PDT
I tested with this build: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32/1272300941/

...which has the patch.  It doesn't solve the problem for that ABC News video link.
Comment 3 IU 2010-04-26 12:36:15 PDT
What are some other windowless video links I can test with (I'm not sure how to tell the difference)?  Maybe there's a quirk unique to ABC News video?
Comment 4 IU 2010-04-26 14:07:30 PDT
Also, msnbc videos have responsiveness issues when I click trying to pause a video.
Comment 5 Benjamin Smedberg [:bsmedberg] 2010-04-26 14:14:21 PDT
Hulu is windowless.
Comment 6 IU 2010-04-26 14:19:15 PDT
Unfortunately, since I'm in Canada, Hulu is blocked for me.
Comment 7 Benjamin Smedberg [:bsmedberg] 2010-04-26 14:32:15 PDT
IU, do you see this problem when the video is paused also, or only when it is playing? And do you see it only when using volumouse hotkeys, or also when clicking on video controls?
Comment 8 Jim Mathies [:jimm] 2010-04-26 14:34:39 PDT
Created attachment 441607 [details]
windowless youtube test case
Comment 9 IU 2010-04-26 14:50:25 PDT
(In reply to comment #7)
> IU, do you see this problem when the video is paused also, or only when it is
> playing? And do you see it only when using volumouse hotkeys, or also when
> clicking on video controls?

Not a problem is video is paused, because CPU is is not being choked to death.

I see it not just with Volumouse.  As mentioned in comment 4, I also experience it when trying to pause videos on msnbc.
Comment 10 IU 2010-04-26 14:56:21 PDT
(In reply to comment #8)
> Created an attachment (id=441607) [details]
> windowless youtube test case

Issue is reproducible with the the attached test case.
Comment 11 Jim Mathies [:jimm] 2010-04-27 07:40:43 PDT
(In reply to comment #10)
> (In reply to comment #8)
> > Created an attachment (id=441607) [details] [details]
> > windowless youtube test case
> 
> Issue is reproducible with the the attached test case.

UI, would you mind taking this try build for a spin and see if it improves the issue? 

https://build.mozilla.org/tryserver-builds/jmathies@mozilla.com-testing/

You shouldn't keep this installed, it's setup to severely limit Flash's frame rate.
Comment 12 IU 2010-04-27 10:55:58 PDT
No. It doesn't improve the issue.

This hidden window that windowless flash uses, is it a hidden Firefox window or a hidden Adobe flash window?
Comment 13 Jim Mathies [:jimm] 2010-04-27 11:31:47 PDT
(In reply to comment #12)
> No. It doesn't improve the issue.
> 
> This hidden window that windowless flash uses, is it a hidden Firefox window or
> a hidden Adobe flash window?

It's a hidden flash window with a class name of 'SWFlash_PlaceholderX'. But that's not the problem here, that last try build brought the frequency of wm-user heart beat events down to around 20/sec.. Whatever you're running into it doesn't appear to have anything to do with that.
Comment 14 IU 2010-04-27 12:30:32 PDT
I just noticed something when comparing playback, of the video you attached, between when it's played windowless (per your attachment) and windowed (when I click the video to take me to the Youtube page to play it there)

In the windowless case, the video playback is quite jerky, as opposed to the windowed case where it is smoother.
Comment 15 Jim Mathies [:jimm] 2010-04-27 12:42:30 PDT
(In reply to comment #14)
> I just noticed something when comparing playback, of the video you attached,
> between when it's played windowless (per your attachment) and windowed (when I
> click the video to take me to the Youtube page to play it there)
> 
> In the windowless case, the video playback is quite jerky, as opposed to the
> windowed case where it is smoother.

Are you using that build I posted a couple hours ago? If so, that's expected. Uninstall that if you're using it and re-install a regular nightly.
Comment 16 Jim Mathies [:jimm] 2010-04-27 12:43:07 PDT
Created attachment 441897 [details]
windowed youtube test case
Comment 17 IU 2010-04-27 12:47:05 PDT
(In reply to comment #15)
> Are you using that build I posted a couple hours ago? If so, that's expected.
> Uninstall that if you're using it and re-install a regular nightly.

No.  I was using today's nightly build when I did that particular test.
Comment 18 Jim Mathies [:jimm] 2010-04-27 12:53:31 PDT
(In reply to comment #17)
> (In reply to comment #15)
> > Are you using that build I posted a couple hours ago? If so, that's expected.
> > Uninstall that if you're using it and re-install a regular nightly.
> 
> No.  I was using today's nightly build when I did that particular test.

So that sounds like bottlenecks in messaging between the two processes, or the overhead of shared surface rendering, or both.
Comment 19 IU 2010-04-27 13:03:05 PDT
The interesting thing is: the difference in video performance between the two test cases is not huge.  It is perceptible on my system, being slow enough, but one could be forgiven for not noticing without a back-to-back comparison.  In contrast, the mouse wheel with Volumouse STR difference is huge.  The slow-down is practically like night and day.
Comment 20 Mike Beltzner [:beltzner, not reading bugmail] 2010-04-28 12:43:35 PDT
This blocks bug 558629, which is a blocker, so this blocks. However, bsmedberg feels like it's not a common enough case to hard block 3.6.4, and I'm inclined to agree. So needed.

Please renominate if we get more data that changes our understanding of the severity.
Comment 21 IU 2010-04-30 09:56:51 PDT
This also severely affects interaction with the msnbc video player.  Videos take a while to start playing and, after you've scrolled the list of videos to the end and try clicking on the player controls, change video categories and continue scrolling, clicking, responsiveness gets worse and worse -- the player taking tens of seconds to respond to any mouse input.
Comment 22 Mike Beltzner [:beltzner, not reading bugmail] 2010-05-08 16:39:45 PDT
IU: is that using the latest beta build from http://www.mozilla.com/firefox/all-beta.html? (release May 4 2010)
Comment 23 IU 2010-05-08 17:42:45 PDT
The nightly trunk builds.
Comment 24 IU 2010-05-08 17:58:18 PDT
The beta you linked to is, however, also affected.  I have verified this.
Comment 25 christian 2010-05-10 10:26:28 PDT
How common is windowless Flash vs windowed Flash?
Comment 26 Jim Mathies [:jimm] 2010-05-10 10:31:25 PDT
(In reply to comment #25)
> How common is windowless Flash vs windowed Flash?

50/50 split between major video sites. (example: youtube: windowed, hulu: windowless)
Comment 27 juan becerra [:juanb] 2010-05-10 14:41:04 PDT
UI, step 2d says "Component: Play Control" but "Play Control" is not an option I see in the Volumouse configuration window, so I tried "Component: Volume Control."

I haven't been able to reproduce this yet using the test cases mentioned here, on a VM. I'll try on hardware next.
Comment 28 IU 2010-05-10 14:59:00 PDT
(In reply to comment #27)
> UI, step 2d says "Component: Play Control" but "Play Control" is not an option
> I see in the Volumouse configuration window, so I tried "Component: Volume
> Control."

It's probably operating system or sound card driver dependent, so doesn't matter.

> 
> I haven't been able to reproduce this yet using the test cases mentioned here,
> on a VM. I'll try on hardware next.

Real hardware is better -- preferably single core.  If you have multiple cores and you can disable the other cores, that would be good.  Benjamin Smedberg also had trouble reproducing on a VM.
Comment 29 juan becerra [:juanb] 2010-05-10 16:11:03 PDT
I've now tried on a couple of older, single core, machines (Pentium 1.6ghz), and I can see a response delay in the volume control scrollbar which, depending on how many times I scroll up and down rapidly - especially in full screen mode, will stay up scrolling up and down. During this time you cannot pause the video. You really have to go crazy on the scroll up/down in those machines, though. We don't have older machines off-hand to check if this is more pronounced in older hardware.
Comment 30 IU 2010-05-10 20:58:39 PDT
Try the MSNBC video player too, if you haven't tried (but not with the STR I provided above).  It's hard to come up with an STR for it, but when it goes unresponsive, there's usually nothing that can be done until the immediate video stops playing.
Comment 31 juan becerra [:juanb] 2010-05-13 17:23:10 PDT
I got a hold of a Pentium 2 500mhz machine with less thatn 400mb of memory, and I compared 3.6.3 and 3.6.4 with Flash 10.1 (rc4) and it is hard to tell the difference, because in both instances the browser becomes unresponsive to mouse clicks, very shortly after starting to play the video. I did not install Volumouse.

I can't scroll the page or switch tabs. It's possible that in 3.6.4 the browser became sluggish sooner into the video playback, but it's hard to tell the difference. Maybe I now need to get a hold of a slightly faster machine similar to the reporter's.
Comment 32 christian 2010-05-17 10:10:34 PDT
I don't think we can hard block on this based on the STR. Removing blocking flag.
Comment 33 IU 2010-05-17 21:00:45 PDT
(In reply to comment #32)
> I don't think we can hard block on this based on the STR. Removing blocking
> flag.

The STR is meant to provide an easy way of observing the impact of this bug.  It is not meant to convey a general use case.  It demonstrates the impact of this bug on mouse input and browser responsiveness.  There are many other scenarios for observing this bug (e.g. my comment 21 and 30), but they are not as easy to illustrate by STR as with comment 0.
Comment 34 christian 2010-05-18 17:15:46 PDT
At some point we need to decide if this is a blocker or not as it isn't getting traction. There are other issues that are more easily reproduced and need attention. I'm not saying this bug is unimportant, just that it wouldn't block a release based on the information we have.
Comment 35 [not reading bugmail] 2010-06-07 21:43:50 PDT
(In reply to comment #9)
> (In reply to comment #7)
> > IU, do you see this problem when the video is paused also, or only when it is
> > playing? And do you see it only when using volumouse hotkeys, or also when
> > clicking on video controls?
> 
> Not a problem is video is paused, because CPU is is not being choked to death.
> 

I noticed if I changed the quality level of the flash object I was seeing better response by firefox.  I wonder if this has anything to do with high bandwidth usage and per tab network prioritization bug 514490 code.
Comment 36 Aaron Kelley 2010-06-23 18:10:45 PDT
My wife plays the "Happy Pets" game on Facebook all the time.  When 3.6.4 came along, she noticed a serious drop in mouse responsiveness when there is a lot of activity on screen.  Once I disabled OOPP for Flash, it returned to normal.  May be related to this bug (someone left a note here: http://etherpad.mozilla.com:9000/ep/pad/export/video-sites/latest?format=html).  Just wanted to bring it up, it may affect other Flash games or applications.
Comment 37 Alex Macfarlane Smith 2010-06-24 00:23:52 PDT
I was amused to see in the release notes it now says "Some slower machines may see degraded mouse responsiveness when viewing certain Flash videos". I'm not sure I'd consider my 2x2.4GHz machine to be slow :)
Comment 38 Alex Macfarlane Smith 2010-06-24 00:42:02 PDT
...having said that, it appears fine in the latest version of Firefox for me (evidently the issue I had has been fixed).
Comment 39 mshtei 2010-06-26 16:02:49 PDT
Hi, I've got a comment on the relelase note's item for this bug. There is a typo in Russian text for it . Where should I report it? Thanks
Comment 40 christian 2010-06-26 19:23:44 PDT
I believe the proper component is Websites/www.mozilla-europe.org
Comment 41 mshtei 2010-06-26 19:53:01 PDT
Thanks
Comment 42 Mike 2010-09-02 11:43:04 PDT
Linux affected too
I run Flash music player in background tab, and FF responds slowly to mouse movement in menus (sort of "delayed repaint")
Mozilla/5.0 (X11; Linux i686; rv:2.0b6pre) Gecko/20100902 Firefox/4.0b6pre
Comment 43 Oskar Ivanić (:icecold) 2010-09-16 18:50:15 PDT
Flash based games or videos or anything that is on flash got worse with beta 5.
If you visit this website http://www.infinitystamp.net/m-light/ , and scroll a little bit and then try to switch to another tab, you can see a delay. With latest trunk, I get worse delay than on beta 5. I saw on Firefox Test Drivers group (Facebook), how people started to complain about how beta 5 got slower for them and when they play games it's horrible. On latest trunk I disabled OOPP because it fixes temporary my problem, but CPU usage is still higher than 3.6.9 or other browsers.
See this bug https://bugzilla.mozilla.org/show_bug.cgi?id=594959 and comment #3 for more info.
Comment 44 Antti Tervasmäki 2010-09-22 06:29:05 PDT
Local ISP has a good test page: http://www.elisa.fi/viihde/
(firefox.exe and plugin-container.exe have high CPU)
It takes Minefield seconds to notice if you wont go somewhere..

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b7pre) Gecko/20100922 Firefox/4.0b7pre ID:20100922041040
Comment 45 Jim Mathies [:jimm] 2010-09-22 06:41:02 PDT
(In reply to comment #44)
> Local ISP has a good test page: http://www.elisa.fi/viihde/
> (firefox.exe and plugin-container.exe have high CPU)
> It takes Minefield seconds to notice if you wont go somewhere..
> 
> Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b7pre) Gecko/20100922
> Firefox/4.0b7pre ID:20100922041040

What type of system are you using?

On my system, with ipc off, fx varies between 2% and 32%. With ipc enabled, fx sits at around 0% - 2%, with pc varying between 2% - 32%.
Comment 46 Jim Mathies [:jimm] 2010-09-22 06:43:01 PDT
For reference, IE9 does a little better, with a single process running between 2% - 15%.
Comment 47 Antti Tervasmäki 2010-09-22 07:07:37 PDT
My system: AMD Phenom II X4 940 3,2 MHz, 8G, Radeon 5870, Win7 Ultimate 64bit.
Minefield in default configs so all Direct2/3 etc are on...

IPC makes a difference, though all those layer things do most load...
Comment 48 IU 2010-09-22 09:08:00 PDT
(In reply to comment #46)
> For reference, IE9 does a little better, with a single process running between
> 2% - 15%.

Jim, is it possible what you're actually seeing is this regression?: bug 595671 comment 13
Comment 49 Jim Mathies [:jimm] 2010-09-22 09:17:43 PDT
(In reply to comment #48)
> (In reply to comment #46)
> > For reference, IE9 does a little better, with a single process running between
> > 2% - 15%.
> 
> Jim, is it possible what you're actually seeing is this regression?: bug 595671
> comment 13

Might be although that site looks to be mostly flash based.
Comment 50 elim.css 2011-03-02 15:42:20 PST
I have problems with the whole browser tab freezing up and mouse movement very jerky/delayed when playing a YouTube video or any flash video on another site for that matter. It just stops responding until the video is buffered and makes the mouse unresponsive. Then you get another 2-3 second delay if you try and hit the pause button. The problem occurs with Chrome 9 as well, but not as bad - there's perhaps a 1sec delay and minor jerky mouse movement for a second. So that makes me think the problem is partly to do with Flash itself but it's definitely a lot worse on Firefox.

I would say my computer was up to the task of playing videos perfectly a few versions of Firefox/Flash ago, but something changed recently has made it worse. Using an Intel dual core 1.7Ghz with 3GB RAM, Radeon X1600 128MB, Firefox 3.6.14, Flash Player 10.2 and Win XP Pro SP3.

Note You need to log in before you can comment on or make changes to this bug.