Gstreamer backend don't use hardware accelerate (VAAPI)

RESOLVED WONTFIX

Status

()

P5
major
RESOLVED WONTFIX
5 years ago
3 years ago

People

(Reporter: vova.kravets, Unassigned)

Tracking

Trunk
All
Linux
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [Power:P3])

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20130715 Firefox/25.0 (Nightly/Aurora)
Build ID: 20130715100109

Steps to reproduce:

1. Install Gstreamer 0.10
2. Install Gstreamer VAAPI plugin
3. Install Firefox 25.0a1
4. Play any HTML5 video 1080p/720p
5. Look at CPU usage



Actual results:

CPU usage is big. It seems VAAPI is not used.

Note: If I use below command I see that VAAPI is used.

gst-launch playbin2 <url>


Expected results:

VAAPI should use and Hardware Accelerate should use during rendering also.
(Reporter)

Updated

5 years ago
Depends on: 799318
(Reporter)

Updated

5 years ago
No longer depends on: 799318
(Reporter)

Updated

5 years ago
Severity: normal → major

Comment 1

5 years ago
Dont you think FF uses an internal copy of GST?

Comment 2

5 years ago
I haven't looked into this, but it's expected that it doesn't work yet. It requires some work in the backend and in the firefox gfx bits to pass around VAAPI backed textures, instead of decoding frames into main memory.
With the majority of linux drivers now supporting VA-API (both directly and using mesa vdpau drivers throught vdpau-va-driver) and with HTML5 standard being used more widely (thank God), this should really be addressed ASAP, since the CPU usage of firefox when decoding and rendering videos is quite a lot (even compared to the "evil" flash player, with hw capabilities).
Blocks: 422540
Status: UNCONFIRMED → NEW
Component: Untriaged → Video/Audio
Ever confirmed: true
Product: Firefox → Core
Hardware: x86_64 → All
Version: 25 Branch → Trunk
No longer blocks: 422540
Depends on: 422540

Comment 4

5 years ago
(In reply to vova.kravets from comment #0)
> User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20130715
> Firefox/25.0 (Nightly/Aurora)
> Build ID: 20130715100109
> 
> Steps to reproduce:
> 
> 1. Install Gstreamer 0.10
> 2. Install Gstreamer VAAPI plugin 
> [...]
> 
> VAAPI should use and Hardware Accelerate should use during rendering also.

If i am not wrong, hardware acceleration need 0.5.7 et 0.5.8 of gstreamer-vaapi and Gtsreamer 1.x
http://lists.freedesktop.org/archives/gstreamer-devel/2014-May/047657.html
> Support for hardware codecs and special memory types has been
>      improved with bugfixes and feature additions in various plugins
>      and base classes.
(In reply to vulcain from comment #4)
> (In reply to vova.kravets from comment #0)
> > User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20130715
> > Firefox/25.0 (Nightly/Aurora)
> > Build ID: 20130715100109
> > 
> > Steps to reproduce:
> > 
> > 1. Install Gstreamer 0.10
> > 2. Install Gstreamer VAAPI plugin 
> > [...]
Since now Firefox uses Gstreamer 1.0, adding video hardware decoding for HTML5 should be a priority.


> > 
> > VAAPI should use and Hardware Accelerate should use during rendering also.
> 
> If i am not wrong, hardware acceleration need 0.5.7 et 0.5.8 of
> gstreamer-vaapi and Gtsreamer 1.x
> http://lists.freedesktop.org/archives/gstreamer-devel/2014-May/047657.html
> > Support for hardware codecs and special memory types has been
> >      improved with bugfixes and feature additions in various plugins
> >      and base classes.
That's true, however I don't think it's much something that is related to firefox anymore.

In fact using the latest firefox release, together with the current gstreamer1.0-vaapi git¹, as soon as I open a youtube html5 video or other html5 videos around², I get in my console:

libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_35
libva info: va_openDriver() returns 0

Which is actually a syntom that things are correctly played with my intel vaapi driver.

I don't know exactly what gstreamer-vaapi revision introduced the changes needed to make firefox to be aware of the vaapi acceleration: for sure it didn't work with 0.5.7, but I didn't try 0.5.8.

Anyway, it seems that most of the work to get the hw acceleartion has to be done outside firefox, as this new gstreamer1.0 support already gives us the framework needed.

[1] For ubuntu amd64 you might try these debs if you want http://people.ubuntu.com/~3v1n0/gstreamer-vaapi/
[2] That said, youtube works, vimeo also, but other videos were not playing video or with some glitches, while in other cases, I got a crash³, but as bonus the CPU usage is really low.
[3] What said is valid only for intel in my tests, because using radeon throught the vdpau-va-driver this always crashes (and the same does totem by using that version of gstreamer-vaapi with any mp4 video). I still had no time to debug it and to check weather the problem is in the radeon vdpau driver or somewhere else; input from nvidia and nouveau users would be welcome.
The GStreamer 1.0 backend does autoplug vaapidecode if it's available, however CPU usage of Firefox is much higher here when it is: ~120% with vaapi vs. ~30% without, for Youtube 720p.

Comment 8

4 years ago
gstreamer-vaapi 5.9 has been released and in NEWS file we can read among others:
* Fix support for Firefox >= 30 when built with GStreamer 1.0 APIs
And it indeed now works without crashing Firefox, but apparently it's not optimal and requires improvements on Firefox's side. Here is a comment by Gwenole Beauchesne who works on gstreamer-vaapi:

Firefox will not use hardware accelerated rendering and requires the decoded frames to be transferred to their own buffers. i.e. reading every single frame from GPU memory is necessary. If you want to compare Firefox behaviour's to the "optimal" one, you could probably use gst-launch-1.0 uri=file:///path/to/Video.mp4 video-sink="ximagesink" (similar to firefox behaviour), vs. gst-launch-1.0 uri=file:///path/to/Video/mp4 video-sink="vaapisink" (the default).

Yes, we have NV12 -> I420 (HW accelerated through GPU memory) -> read back to I420 buffer -> I420 to RGB32 conversion through SSE2 code it seems. Not the fastest pipeline, but patches are welcome for Firefox. 

source:http://www.phoronix.com/forums/showthread.php?104498-GStreamer-VA-API-Plug-In-Update-Adds-New-Features&p=431726#post431726

So, could Firefox devs work on this? Because right now there is no noticeable drop of CPU usage.

Comment 9

4 years ago
(In reply to Marco Trevisan (Treviño) from comment #6)
> .... input from nvidia and nouveau users would be
> welcome.
Here are my findings with a Nvidia GT 430 graphics card and a lowly Pentium 4 3Ghz and the following software versions:
Ubuntu 14.04.1 (AMD64)
Firefox 31.0
gStreamer-vaapi 5.9 (from http://people.ubuntu.com/~3v1n0/gstreamer-vaapi/ )
Nvidia driver 331.38

youtube and vimeo work. I tested some other HTML5 videos on other sites which also worked, but most of the times crashed when changeing resolution or pausing/playing the video (this also happens with youtube). For this hardware setup the difference between version 5.9 and 5.7 is that HD720 and HD1080p video's are now playable! While on the old (5.7) version of the gstreamer-vaapi, playing these video's in Firefox was impossible (i.e. a dia show and 100% cpu).
I also tested Totem with mp4 content. It crashed within 2 seconds with version 5.9 as you already found out.

Comment 10

3 years ago
According to comment 4, this needs GStreamer 1.0, so it should depend on bug 947287.

Updated

3 years ago
Depends on: 947287

Comment 11

3 years ago
Doesn't work with gstreamer-vaapi 0.5.10 and 0.6.0 (green background instead of video).

(firefox:31337): GStreamer-CRITICAL **: gst_mini_object_lock: assertion 'object != NULL' failed
(firefox:31337): GStreamer-CRITICAL **: gst_memory_unmap: assertion 'mem != NULL' failed
(firefox:31337): GStreamer-CRITICAL **: gst_memory_get_sizes: assertion 'mem != NULL' failed
(firefox:31337): GStreamer-CRITICAL **: gst_mini_object_unlock: assertion 'object != NULL' failed
(firefox:31337): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

gst-play-1.0 command works with hardware acceleration without any issues.

Mesa 11.0.0_rc1 with Gallium3D VAAPI state tracker, radeonsi.

Comment 12

3 years ago
(In reply to Mathew Hodson from comment #10)
> According to comment 4, this needs GStreamer 1.0, so it should depend on bug
> 947287.

Correct.

Comment 13

3 years ago
Installed Firefox 40.0 from OpenSUSE 13.2 repositories (32bit)

When I visit facebook.com, after a few scrolls on the page, Firefox crashes with only output VAAPI and GStreamer related messages.

libva info: VA-API version 0.34.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_34
libva info: va_openDriver() returns 0
(firefox:3891): GStreamer-CRITICAL **: gst_mini_object_lock: assertion 'object != NULL' failed
(firefox:3891): GStreamer-CRITICAL **: gst_memory_unmap: assertion 'mem != NULL' failed
(firefox:3891): GStreamer-CRITICAL **: gst_memory_get_sizes: assertion 'mem != NULL' failed

Comment 14

3 years ago
Created attachment 8657393 [details]
glunardi-bug-894372-crashreport.txt

This is the crashreport that gets generated after I visit facebook.com using:
 MozillaFirefox-40.0-38.1.i586
 Mesa-10.3.7-91.6.1.i586
 GStreamer-1.4.3-1.4.i586
 libgstvaapi-1_4-2-0.6.0-9.2.i586
 libgstvaapi-drm-1_4-2-0.6.0-9.2.i586
 libgstvaapi-glx-1_4-0-0.5.9-4.1.i586
 libgstvaapi-x11-1_4-0-0.5.9-4.1.i586
 libgstvaapi-x11-1_4-2-0.6.0-9.2.i586
(In reply to glunardi from comment #13)
> When I visit facebook.com, after a few scrolls on the page, Firefox crashes
> with only output VAAPI and GStreamer related messages.

Can you file a new bug?

Comment 16

3 years ago
(In reply to Marco Castelluccio [:marco] from comment #15)
> Can you file a new bug?
Created https://bugzilla.mozilla.org/show_bug.cgi?id=1202140

Thanks!

Comment 17

3 years ago
Shouldn't this bug have "[Power]" added to the bug whiteboard?
https://wiki.mozilla.org/Performance/Project_Candle
Whiteboard: [Power]
Component: Audio/Video → Audio/Video: Playback

Comment 18

3 years ago
(In reply to Mariusz Libera from comment #8)
> gstreamer-vaapi 5.9 has been released and in NEWS file we can read among
> others:
> * Fix support for Firefox >= 30 when built with GStreamer 1.0 APIs
> And it indeed now works without crashing Firefox, but apparently it's not
> optimal and requires improvements on Firefox's side. Here is a comment by
> Gwenole Beauchesne who works on gstreamer-vaapi:

[Snip]

> So, could Firefox devs work on this? Because right now there is no
> noticeable drop of CPU usage.

Agreed. I don't even know whether it's using acceleration or not because the CPU usage is so high. Linux desperately needs a mainstream browser to support efficient HTML5 video playback. Chrome/Chromium devs aren't interested in supporting Linux either.
Priority: -- → P5
Whiteboard: [Power] → [Power:P3]

Comment 19

3 years ago
Hello,

I guess now that GStreamer will be deprecated and FFmpeg/Libav will be used directly:

https://bugzilla.mozilla.org/show_bug.cgi?id=1207429

the proper bug to discuss hardware accelerated video playback now would be:

https://bugzilla.mozilla.org/show_bug.cgi?id=563206

?

Regards
gstreamer is going in bug 1234092
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.