Closed Bug 514848 Opened 15 years ago Closed 14 years ago

audio and video are choppy with 100% cpu during playback

Categories

(Core :: Audio/Video, defect)

Product:
Core
Component:
Audio/Video
Platform:
All
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: blizzard, Unassigned)

References

(
URL
)

Details

Attachments

(1 file)

Samples
1.70 KB, application/zip
Details 
See the URL above and conversation here:

http://identi.ca/conversation/9437690

Appears to play back smoothly on Chromium on Linux.

That site might have something strange going on in the rendering pipeline that's worth looking at.

Tried in both 3.5 and 3.6 nightlies, same result.
Plays fine on 3.5 in Windows. Possibly something to do with the sound issues we've been having. What sound system are you using? Pulse? I'll test on Linux when I'm at my Linux box next.
The main issue here is painting performance; the video is being scaled up from its native size (which is expensive) and we're repainting around 2x more than necessary due to the custom controls.  Playing the video and then switching to another tab is a good way to test whether painting performance is involved.  Digging the video URL out of the page and playing it back in its own tab is another way to test this.  Doing this, I see CPU use about half of what it is when playing the video embedded in the DailyMotion page.

The other issue is that Firefox's CPU is spiking very high for short periods of time during playback (for no more than 3-5 seconds).  I'm not sure what's causing this.  CPU was rising above 100% of one core, so suspect the decoder/playback thread may be doing something strange during this time.
This is not a Linux issue, this hits Windows hard as well. I searched for other related bugs; there are some but I'm still not sure Mozilla is well aware of the performance problems <video> has, they are serious (serious enough for me to always try to serve video in Flash, with <video> as a fallback).

I've seen some justifications such as "we can't take advantage of hardware acceleration" and "we have to composite the video with other stuff", but this is far from justifying the current levels of performance, as you'll see below.

I just did a test against Flash (10 r32). Latest Firefox nightly (Gecko/20090922 Minefield/3.7a1pre). Machine has a Core 2 Duo at 2GHz (T7250) and is running Windows 7 RC. Do note that I see this behavior on several machines and OSes; if anything, it gets worse on older CPUs (specially Pentium 4) - by worse I mean worse in comparison to Flash.

Video was 450x360 pixels in size. The Flash version was encoded in H.264, which is more expensive to decode (not that this matters much, as 90% of the CPU is wasted elsewhere). I checked CPU usage while playback, as well as how much it increased when drawing some stuff over the video (in the case of <video>, the built-in controls; for Flash, some transparent controls comparable in size). For <video>, the video was opened on its own (not inside a web page).

100% CPU usage means both cores used fully. CPU was placed in the highest power state possible all the time (otherwise low CPU usages would get inflated).

zoom_level      <video> <video>+controls Flash Flash+controls
100% (450x360)  12%     16%               8%    9%
111% (500x400)  24%     29%              11%   13%
162% (730x584)  46%*    54%**            18%   20%

* Some frames were dropped
** A lot of frames were dropped

Note that Flash was running in software mode (no hardware acceleration at all). Not that its results are strange: this is the kind of CPU usage I'd expect based on my experience with similar software video pipelines. I can get similar (slightly better) results with standalone players (such as mplayer), even when forcing them to do software color conversion, software scaling, software composition of subtitles, and forcing them to draw to the screen in the slowest possible way (no DirectX/OpenGL, no XV, no overlays, no nothing).

Resize quality was similar in both cases (I didn't check it thoroughly, but it looks like standard bilinear in both cases - definitely not nearest neighborhood).

So yeah, it's slow. A lot. If I had to guess, the worst offender would be the resizer (which is very noticeable too when resizing static images, making full page zoom a but of a hassle performance-wise), but in general everything seems a lot slower than it should.

I understand the specific difficulties that a browser might have, but I'd expect at least the same performance Flash has right now, as I don't see how Flash would have it any easier (if anything, being a plugin would make it slower).
Was the video you were testing with being scaled while playing? Do you have the video you used available so I can test with it?
Attached file Samples — 
(In reply to comment #4)
> Was the video you were testing with being scaled while playing? Do you have the
> video you used available so I can test with it?

In all 3 cases the same video was used. The video was 450x360, so it wasn't scaled on the first case and it was scaled in the other two cases.

This happens with any video. I'm attaching a simple testcase just in case - it fetches the same video from TinyVid and YouTube. It's not exactly the same test but should be pretty close. Results are more or less the same, for example on a Prescott P4 at 3GHz (no HT):

zoom            <video> Flash 
100% (450x360)  48%     30%
111% (500x400)  76%     42%
162% (730x584)  100%*   56%

* A few frames lost here and there

(This was against an older build of Firefox, so I'd expect it to perform a bit better now)
Summary: audio and video are choppy with 100% cpu during playback on linux → audio and video are choppy with 100% cpu during playback
Video is absolutely not working, even with Firefox 3.6.
(I use windows XP, Centrino 1.4 Ghz, Firefox 3.6)

The worst problem is that audio is choppy if the CPU is at 100%. Choppy video is less annoying than choppy audio! The most important thing is to give audio somehow a higher priority (compared to video) so it never skips!
I think some commenters here are reporting different issues. The Linux-specific issue is that playing video on sites such as openvideo.dailymotion.com renders Firefox basically unusable, where video takes 100% CPU, but video rendering is too slow to produce much of anything on the screen. I have verified that this problem does not exist (at least, not nearly as severely) on a different machine running OSX.

Judging by the few comments here relating to Windows I would suggest that the issue in those comments is distinct from what is described by the bug reporter.
I was trying to duplicate this but daily motion seems to require downloading the entire video before it allows playing it. I can't press 'Play' to stop the buffering. Chris Blizzard, who can we get in touch with at Daily Motion to look into that?
A surprising observation: when I play a video on openvideo.dailymotion.com, and I hit pause, the CPU is still busied, until I switch to a different tab.
Here's another link with super distorted audio playback: http://www.spreadfirefox.com/5years/en-US/
Significantly, the problem occurs also when playing WebM videos on YouTube's HTML5 demo with today's trunk nightly build.
Bastiaan, which problem? The one described in comment 10, comment 14 or comment 15?
(In reply to comment #17)
> Bastiaan, which problem? The one described in comment 10, comment 14 or comment
> 15?

The problem described in comment 1 (audio and video are choppy with 100% cpu during playback).
Er, that should have been comment 0.
Bastiaan, what OS are you using and what is the architecture and speed of your CPU?
1. Fedora 13.
2. i686, Intel(R) Core(TM)2 CPU T7400 @ 2.16GHz

FWIW, I can "play" the same YouTube page using Chromium fluently (like comment 0), which registers as using approx. 20% of one of the CPU cores per WebM video.
> FWIW, I can "play" the same YouTube page using Chromium fluently (like comment
> 0), which registers as using approx. 20% of one of the CPU cores per WebM
> video.

Which YouTube page specifically? I'd like to test against it since I have a machine with similar specs to yours. What size video too. 360p? Are you trying full screen or non-full screen?
One more thing I just noticed: only video is choppy, not audio. (This contrasts with many commenters who reported choppy audio as well.)
(In reply to comment #22)
> Which YouTube page specifically? I'd like to test against it since I have a
> machine with similar specs to yours. What size video too. 360p? Are you trying
> full screen or non-full screen?

I haven't found a WebM video on YouTube that plays fluently. I'm playing 360p (which seems to be the default) non-full screen.

For example:

http://www.youtube.com/watch?v=4KvOXs1YfwQ
Bastiaan, can you please try the steps in bug 571386 comment 3 and report the results?
Matthew, the video is quite fluent when following said instructions.
Bastiaan, Based on comment 26 it looks like Bug 577843 which has a fix in progress.
Chris, I'm not sure about that, because I'm unable to reproduce the issue on a Mac OSX machine or a Windows machine (both of which are slower than my GNU/Linux machine), and bug 577843 is filed against Windows XP. Mind you, on my GNU/Linux system, (scaled) video rendering is so slow Firefox doesn't have time to respond to user input. I would suspect that sharpening the conversion routines wouldn't cause such a drastic reduction in processing time required to fix the issue on my machine.
If it helps, by the way, I had a similar performance problem with CSS transition effects, but this was fixed by roc's layers work.
I have had this issue playing audio, but only when running from localhost. I am unable to replicate on a remote server.
My test package is here: http://versionindustries.com/remote/FF-AUDIO.zip

When running this locally (http://localhost/FF-AUDIO/) I get the following results:
- on page load, normal CPU usage (10-15%)
- on clicking PLAY, CPU goes up to 80 - 90% (combined usage for both cores) within 2 seconds
- audio occasionally chops to start with, then plays smoothly
- CPU usage remains at a combined figure of 80-90%.
- If the file is in the browser cache, this appears to mitigate the effect.

When running the package on a remote server, the max CPU usage I saw was about 30%, and only for a moment.

Macbook Pro 2.4 Ghz Core 2 Duo, 4GB RAM, Mac OS X 10.6.2, Firefox 3.6.10.
(In reply to comment #27)
> Bastiaan, Based on comment 26 it looks like Bug 577843 which has a fix in
> progress.

It turns out you were right. Today's nightly build (which includes the patch for 577843) fixes this bug for me. Video is played smoothly and audio is uninterrupted.

It should be noted that CPU usage is still high (when compared to MPlayer with X11 rendering, for example), but that is a relatively minor problem.

Thanks!
(In reply to comment #31)
> Today's nightly build (which includes the patch
> for 577843) fixes this bug for me. Video is played smoothly and audio is
> uninterrupted.

Victory then!
Status: NEW → RESOLVED
Closed: 14 years ago
Depends on: 577843
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: