Closed Bug 894372 Opened 11 years ago Closed 9 years ago

Gstreamer backend don't use hardware accelerate (VAAPI)

Categories

(Core :: Audio/Video: Playback, defect, P5)

All
Linux
defect

Tracking

()

RESOLVED WONTFIX

People

(Reporter: vova.kravets, Unassigned)

References

Details

(Whiteboard: [Power:P3])

Attachments

(1 file)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20130715 Firefox/25.0 (Nightly/Aurora) Build ID: 20130715100109 Steps to reproduce: 1. Install Gstreamer 0.10 2. Install Gstreamer VAAPI plugin 3. Install Firefox 25.0a1 4. Play any HTML5 video 1080p/720p 5. Look at CPU usage Actual results: CPU usage is big. It seems VAAPI is not used. Note: If I use below command I see that VAAPI is used. gst-launch playbin2 <url> Expected results: VAAPI should use and Hardware Accelerate should use during rendering also.
Depends on: 799318
No longer depends on: 799318
Severity: normal → major
Dont you think FF uses an internal copy of GST?
I haven't looked into this, but it's expected that it doesn't work yet. It requires some work in the backend and in the firefox gfx bits to pass around VAAPI backed textures, instead of decoding frames into main memory.
With the majority of linux drivers now supporting VA-API (both directly and using mesa vdpau drivers throught vdpau-va-driver) and with HTML5 standard being used more widely (thank God), this should really be addressed ASAP, since the CPU usage of firefox when decoding and rendering videos is quite a lot (even compared to the "evil" flash player, with hw capabilities).
Blocks: 422540
Status: UNCONFIRMED → NEW
Component: Untriaged → Video/Audio
Ever confirmed: true
Product: Firefox → Core
Hardware: x86_64 → All
Version: 25 Branch → Trunk
No longer blocks: 422540
Depends on: 422540
(In reply to vova.kravets from comment #0) > User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20130715 > Firefox/25.0 (Nightly/Aurora) > Build ID: 20130715100109 > > Steps to reproduce: > > 1. Install Gstreamer 0.10 > 2. Install Gstreamer VAAPI plugin > [...] > > VAAPI should use and Hardware Accelerate should use during rendering also. If i am not wrong, hardware acceleration need 0.5.7 et 0.5.8 of gstreamer-vaapi and Gtsreamer 1.x http://lists.freedesktop.org/archives/gstreamer-devel/2014-May/047657.html > Support for hardware codecs and special memory types has been > improved with bugfixes and feature additions in various plugins > and base classes.
(In reply to vulcain from comment #4) > (In reply to vova.kravets from comment #0) > > User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20130715 > > Firefox/25.0 (Nightly/Aurora) > > Build ID: 20130715100109 > > > > Steps to reproduce: > > > > 1. Install Gstreamer 0.10 > > 2. Install Gstreamer VAAPI plugin > > [...] Since now Firefox uses Gstreamer 1.0, adding video hardware decoding for HTML5 should be a priority. > > > > VAAPI should use and Hardware Accelerate should use during rendering also. > > If i am not wrong, hardware acceleration need 0.5.7 et 0.5.8 of > gstreamer-vaapi and Gtsreamer 1.x > http://lists.freedesktop.org/archives/gstreamer-devel/2014-May/047657.html > > Support for hardware codecs and special memory types has been > > improved with bugfixes and feature additions in various plugins > > and base classes.
That's true, however I don't think it's much something that is related to firefox anymore. In fact using the latest firefox release, together with the current gstreamer1.0-vaapi git¹, as soon as I open a youtube html5 video or other html5 videos around², I get in my console: libva info: VA-API version 0.35.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_35 libva info: va_openDriver() returns 0 Which is actually a syntom that things are correctly played with my intel vaapi driver. I don't know exactly what gstreamer-vaapi revision introduced the changes needed to make firefox to be aware of the vaapi acceleration: for sure it didn't work with 0.5.7, but I didn't try 0.5.8. Anyway, it seems that most of the work to get the hw acceleartion has to be done outside firefox, as this new gstreamer1.0 support already gives us the framework needed. [1] For ubuntu amd64 you might try these debs if you want http://people.ubuntu.com/~3v1n0/gstreamer-vaapi/ [2] That said, youtube works, vimeo also, but other videos were not playing video or with some glitches, while in other cases, I got a crash³, but as bonus the CPU usage is really low. [3] What said is valid only for intel in my tests, because using radeon throught the vdpau-va-driver this always crashes (and the same does totem by using that version of gstreamer-vaapi with any mp4 video). I still had no time to debug it and to check weather the problem is in the radeon vdpau driver or somewhere else; input from nvidia and nouveau users would be welcome.
The GStreamer 1.0 backend does autoplug vaapidecode if it's available, however CPU usage of Firefox is much higher here when it is: ~120% with vaapi vs. ~30% without, for Youtube 720p.
gstreamer-vaapi 5.9 has been released and in NEWS file we can read among others: * Fix support for Firefox >= 30 when built with GStreamer 1.0 APIs And it indeed now works without crashing Firefox, but apparently it's not optimal and requires improvements on Firefox's side. Here is a comment by Gwenole Beauchesne who works on gstreamer-vaapi: Firefox will not use hardware accelerated rendering and requires the decoded frames to be transferred to their own buffers. i.e. reading every single frame from GPU memory is necessary. If you want to compare Firefox behaviour's to the "optimal" one, you could probably use gst-launch-1.0 uri=file:///path/to/Video.mp4 video-sink="ximagesink" (similar to firefox behaviour), vs. gst-launch-1.0 uri=file:///path/to/Video/mp4 video-sink="vaapisink" (the default). Yes, we have NV12 -> I420 (HW accelerated through GPU memory) -> read back to I420 buffer -> I420 to RGB32 conversion through SSE2 code it seems. Not the fastest pipeline, but patches are welcome for Firefox. source:http://www.phoronix.com/forums/showthread.php?104498-GStreamer-VA-API-Plug-In-Update-Adds-New-Features&p=431726#post431726 So, could Firefox devs work on this? Because right now there is no noticeable drop of CPU usage.
(In reply to Marco Trevisan (Treviño) from comment #6) > .... input from nvidia and nouveau users would be > welcome. Here are my findings with a Nvidia GT 430 graphics card and a lowly Pentium 4 3Ghz and the following software versions: Ubuntu 14.04.1 (AMD64) Firefox 31.0 gStreamer-vaapi 5.9 (from http://people.ubuntu.com/~3v1n0/gstreamer-vaapi/ ) Nvidia driver 331.38 youtube and vimeo work. I tested some other HTML5 videos on other sites which also worked, but most of the times crashed when changeing resolution or pausing/playing the video (this also happens with youtube). For this hardware setup the difference between version 5.9 and 5.7 is that HD720 and HD1080p video's are now playable! While on the old (5.7) version of the gstreamer-vaapi, playing these video's in Firefox was impossible (i.e. a dia show and 100% cpu). I also tested Totem with mp4 content. It crashed within 2 seconds with version 5.9 as you already found out.
According to comment 4, this needs GStreamer 1.0, so it should depend on bug 947287.
Depends on: 947287
Doesn't work with gstreamer-vaapi 0.5.10 and 0.6.0 (green background instead of video). (firefox:31337): GStreamer-CRITICAL **: gst_mini_object_lock: assertion 'object != NULL' failed (firefox:31337): GStreamer-CRITICAL **: gst_memory_unmap: assertion 'mem != NULL' failed (firefox:31337): GStreamer-CRITICAL **: gst_memory_get_sizes: assertion 'mem != NULL' failed (firefox:31337): GStreamer-CRITICAL **: gst_mini_object_unlock: assertion 'object != NULL' failed (firefox:31337): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed gst-play-1.0 command works with hardware acceleration without any issues. Mesa 11.0.0_rc1 with Gallium3D VAAPI state tracker, radeonsi.
(In reply to Mathew Hodson from comment #10) > According to comment 4, this needs GStreamer 1.0, so it should depend on bug > 947287. Correct.
Installed Firefox 40.0 from OpenSUSE 13.2 repositories (32bit) When I visit facebook.com, after a few scrolls on the page, Firefox crashes with only output VAAPI and GStreamer related messages. libva info: VA-API version 0.34.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_34 libva info: va_openDriver() returns 0 (firefox:3891): GStreamer-CRITICAL **: gst_mini_object_lock: assertion 'object != NULL' failed (firefox:3891): GStreamer-CRITICAL **: gst_memory_unmap: assertion 'mem != NULL' failed (firefox:3891): GStreamer-CRITICAL **: gst_memory_get_sizes: assertion 'mem != NULL' failed
This is the crashreport that gets generated after I visit facebook.com using: MozillaFirefox-40.0-38.1.i586 Mesa-10.3.7-91.6.1.i586 GStreamer-1.4.3-1.4.i586 libgstvaapi-1_4-2-0.6.0-9.2.i586 libgstvaapi-drm-1_4-2-0.6.0-9.2.i586 libgstvaapi-glx-1_4-0-0.5.9-4.1.i586 libgstvaapi-x11-1_4-0-0.5.9-4.1.i586 libgstvaapi-x11-1_4-2-0.6.0-9.2.i586
(In reply to glunardi from comment #13) > When I visit facebook.com, after a few scrolls on the page, Firefox crashes > with only output VAAPI and GStreamer related messages. Can you file a new bug?
(In reply to Marco Castelluccio [:marco] from comment #15) > Can you file a new bug? Created https://bugzilla.mozilla.org/show_bug.cgi?id=1202140 Thanks!
Shouldn't this bug have "[Power]" added to the bug whiteboard? https://wiki.mozilla.org/Performance/Project_Candle
Whiteboard: [Power]
Component: Audio/Video → Audio/Video: Playback
(In reply to Mariusz Libera from comment #8) > gstreamer-vaapi 5.9 has been released and in NEWS file we can read among > others: > * Fix support for Firefox >= 30 when built with GStreamer 1.0 APIs > And it indeed now works without crashing Firefox, but apparently it's not > optimal and requires improvements on Firefox's side. Here is a comment by > Gwenole Beauchesne who works on gstreamer-vaapi: [Snip] > So, could Firefox devs work on this? Because right now there is no > noticeable drop of CPU usage. Agreed. I don't even know whether it's using acceleration or not because the CPU usage is so high. Linux desperately needs a mainstream browser to support efficient HTML5 video playback. Chrome/Chromium devs aren't interested in supporting Linux either.
Whiteboard: [Power] → [Power:P3]
Hello, I guess now that GStreamer will be deprecated and FFmpeg/Libav will be used directly: https://bugzilla.mozilla.org/show_bug.cgi?id=1207429 the proper bug to discuss hardware accelerated video playback now would be: https://bugzilla.mozilla.org/show_bug.cgi?id=563206 ? Regards
gstreamer is going in bug 1234092
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: