Closed Bug 1660336 Opened 4 years ago Closed 3 years ago

Add VA-API decode path to bundled ffvpx

Categories

(Core :: Audio/Video: Playback, enhancement)

79 Branch
Desktop
Linux
enhancement

Tracking

()

RESOLVED FIXED
85 Branch
Tracking Status
firefox85 --- fixed

People

(Reporter: pmenzel+bugzilla.mozilla.org, Assigned: stransky)

References

(Blocks 1 open bug)

Details

Attachments

(6 files, 1 obsolete file)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:79.0) Gecko/20100101 Firefox/79.0

Steps to reproduce:

Try to use hardware accelerated WebRTC VP8 decode.

Actual results:

The fan started spinning (bug #1660195).

Expected results:

It’d be great if libvpx shipped by Firefox supported VA-API decode path.

From Martin’s article:

Mozilla binaries perform VP8/VP9 decoding by bundled libvpx library which is missing VA-API decode path. If your hardware supports it and you want to use VA-API for VP8/VP9 decoding, you need to disable bundled libvpx and force external ffmpeg. Go to about:config and set media.ffvpx.enabled to false. Fedora sets that by default when VA-API is enabled.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: Untriaged → WebRTC: Networking
Product: Firefox → Core
Status: UNCONFIRMED → NEW
Component: WebRTC: Networking → Audio/Video: Playback
Ever confirmed: true
OS: Unspecified → Linux
Hardware: Unspecified → Desktop

AFAI understand libvpx is a pure software decoder and it doesn't make sense to "add VA-API" to it. What we actually want is for Firefox to try the external FFmpeg's hardware decoding first, then the internal libvpx (if enabled) and then the external FFmpeg's software decoding.

(In reply to Jan Alexander Steffens [:heftig] from comment #2)

AFAI understand libvpx is a pure software decoder and it doesn't make sense to "add VA-API" to it. What we actually want is for Firefox to try the external FFmpeg's hardware decoding first, then the internal libvpx (if enabled) and then the external FFmpeg's software decoding.

Only how it's used for video playback.
We have multiple copies unfortunately (one for webrtc, one for video playback (unused) and one for webp).

The one shipping with webrtc is used for encoding.

We're trying to remove the need for libvpx altogether

In any case, I will close this as won't fix.
With webrtc today, libvpx is used for encoding, ffvp8 is used for decoding and it can already be hardware decoded.

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX

I think the confusion may come from the original blog entry that incorrectly refers to libvpx.

It's incorrect, we use ffvpx.

I do see an interest in shipping VA-API aware code with our own version of ffvpx on linux.

Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
Summary: Add VA-API decode path to bundled libvpx → Add VA-API decode path to bundled ffvpx
  • Update in-tree ffvpx library with VP8/VP9 VAAPI HW decode code from FFmpeg 4.2.
  • Enable VP8/VP9 VAAPI HW decode on MOZ_WAYLAND target.
Assignee: nobody → stransky

Implemented DMABufSurfaceWrapper and VAAPIDisplayHolder as a versioned class templates
as they are going to be used by both system ffmpeg and bundled ffvpx decoders.

Depends on D90554

Depends on: 1665639
  • Ship libva headers to allow build ffvpx with VA-API support
  • Link libva runtime by library wrapper

Updated the patches with libva headers and run-time libva linker. It allows to build/run on systems without system libva library.
I know about the gfxPlatformGtk::GetPlaform() / RDD issue and I'd like to address that in a separate patch for both ffmpeg/ffvpx.
Thanks.

When https://phabricator.services.mozilla.com/D94650 is checked in we can remove runtime va* functions loading from FFmpegLibWrapper which may also help to address an issue with gfxPlatform in RDD process.

Attachment #9183555 - Attachment description: Bug 1660336 Link libva run-time to allow build and run Firefox on systems without libva installed, r?jya → Bug 1660336 Provide libva wrapper to run Firefox on systems without libva installed, r?jya
Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7694d9e2424b
Add VP8/VP9 VAAPI HW decode code to bundled ffvpx and build it with MOZ_WAYLAND target, r=jya
https://hg.mozilla.org/integration/autoland/rev/450b4f240ff5
Implement DMABufSurfaceWrapper and VAAPIDisplayHolder as templates, r=jya
https://hg.mozilla.org/integration/autoland/rev/f53408903b5e
Implement FFmpegLibWrapper::LinkVAAPILibs() to link VAAPI libraries and use at at FFmpegRuntimeLinker/FFVPXRuntimeLinker, r=jya

Backed out for causing build bustages in vaapi

Backout link: https://hg.mozilla.org/integration/autoland/rev/6af9cd7b395cde46d062ccf76713b74e8cf08038

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cusercancel%2Crunning%2Cpending%2Crunnable&revision=f53408903b5e1e6fb34bc6ac96bc239a3484bb82

Failure log: https://treeherder.mozilla.org/logviewer?job_id=320800064&repo=autoland&lineNumber=26061

"WARNING - /builds/worker/checkouts/gecko/media/ffvpx/config_unix64.h:235:9: warning: 'HAVE_VALGRIND_VALGRIND_H' macro redefined [-Wmacro-redefined]
[task 2020-11-05T08:13:40.759Z] 08:13:40 INFO - #define HAVE_VALGRIND_VALGRIND_H 0
[task 2020-11-05T08:13:40.759Z] 08:13:40 INFO - ^
[task 2020-11-05T08:13:40.759Z] 08:13:40 INFO - /builds/worker/workspace/obj-build/mozilla-config.h:106:9: note: previous definition is here
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - #define HAVE_VALGRIND_VALGRIND_H 1
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - ^
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - In file included from /builds/worker/checkouts/gecko/media/ffvpx/libavutil/hwcontext_vaapi.c:49:
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - /builds/worker/checkouts/gecko/media/ffvpx/libavutil/hwcontext_vaapi.h:22:10: fatal error: 'va/va.h' file not found
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - #include <va/va.h>
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - ^~~~~~~~~
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - 2 warnings and 1 error generated.
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - /builds/worker/checkouts/gecko/config/rules.mk:596: recipe for target 'hwcontext_vaapi.o' failed
[task 2020-11-05T08:13:40.760Z] 08:13:40 ERROR - make[4]: *** [hwcontext_vaapi.o] Error 1
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - make[4]: Leaving directory '/builds/worker/workspace/obj-build/media/ffvpx/libavutil'
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - make[4]: *** Waiting for unfinished jobs....
[task 2020-11-05T08:13:40.760Z] 08:13:40 INFO - make[4]: Entering directory '/builds/worker/workspace/obj-build/modules/libmar/tool'"

Flags: needinfo?(stransky)

Thanks. We need libva 2.0 headers in build roots. Re-opened Bug 1665639 for it.

Flags: needinfo?(stransky)
Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1d4f4423422a
Add VP8/VP9 VAAPI HW decode code to bundled ffvpx and build it with MOZ_WAYLAND target, r=jya
https://hg.mozilla.org/integration/autoland/rev/a1fce14ac3a8
Implement DMABufSurfaceWrapper and VAAPIDisplayHolder as templates, r=jya
https://hg.mozilla.org/integration/autoland/rev/bc879f517930
Implement FFmpegLibWrapper::LinkVAAPILibs() to link VAAPI libraries and use at at FFmpegRuntimeLinker/FFVPXRuntimeLinker, r=jya
Backout by csabou@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1ca5b384f447
Backed out 3 changesets for bustages on wcontext_vaapi.h.
Pushed by rmaries@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a7089f76921f
Add VP8/VP9 VAAPI HW decode code to bundled ffvpx and build it with MOZ_WAYLAND target, r=jya
https://hg.mozilla.org/integration/autoland/rev/1e094ee6647f
Implement DMABufSurfaceWrapper and VAAPIDisplayHolder as templates, r=jya
Flags: needinfo?(stransky)

Wouldn't it be easier to swap order in which ffvpx- and ffmpeg-based decoders are registered? That will make Firefox prefer OS-provided ffmpeg for everything, including VP9.

(In reply to Rinat from comment #24)

Wouldn't it be easier to swap order in which ffvpx- and ffmpeg-based decoders are registered? That will make Firefox prefer OS-provided ffmpeg for everything, including VP9.

We still want to have bundled ffvpx with va-api support, especially for webrtc and distros without default system ffmpeg (fedora/suse...).

Pushed by nerli@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e6f396b25887
Add VP8/VP9 VAAPI HW decode code to bundled ffvpx and build it with MOZ_WAYLAND target, r=jya
https://hg.mozilla.org/integration/autoland/rev/5ccda5ab6563
Implement DMABufSurfaceWrapper and VAAPIDisplayHolder as templates, r=jya
https://hg.mozilla.org/integration/autoland/rev/373a658bb281
Implement FFmpegLibWrapper::LinkVAAPILibs() to link VAAPI libraries and use at at FFmpegRuntimeLinker/FFVPXRuntimeLinker, r=jya
https://hg.mozilla.org/integration/autoland/rev/bdf9d27bd4d2
Build ffvpx FFmpegVideoDecoder module with VAAPI support, r=jya
https://hg.mozilla.org/integration/autoland/rev/f1dadb052d3a
Provide libva wrapper to run Firefox on systems without libva installed, r=jya
https://hg.mozilla.org/integration/autoland/rev/1aa6c9d65403
Provide libva headers to build Firefox without libva-devel installed, r=jya

Interesting. I did local builds on Ubuntu 18.04 and I don't see such errors there...will look at it.

Flags: needinfo?(stransky)
Pushed by ccoroiu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/04056b5c72d9
Add VP8/VP9 VAAPI HW decode code to bundled ffvpx and build it with MOZ_WAYLAND target, r=jya
https://hg.mozilla.org/integration/autoland/rev/6fb988919317
Implement DMABufSurfaceWrapper and VAAPIDisplayHolder as templates, r=jya
https://hg.mozilla.org/integration/autoland/rev/5575575cc0c4
Implement FFmpegLibWrapper::LinkVAAPILibs() to link VAAPI libraries and use at at FFmpegRuntimeLinker/FFVPXRuntimeLinker, r=jya
https://hg.mozilla.org/integration/autoland/rev/6c74e8eef334
Build ffvpx FFmpegVideoDecoder module with VAAPI support, r=jya,glandium,jgilbert
Pushed by ccoroiu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b43bed57327d
Provide libva wrapper to run Firefox on systems without libva installed, r=jya
https://hg.mozilla.org/integration/autoland/rev/0f7b1edcd999
Provide libva headers to build Firefox without libva-devel installed, r=jya

I was checking why some recent landings of patches broke try-comm-central task job, and saw this post by chance.

I have been using the following patch to avoid the macro redefinition error in my local build of C-C TB
for the last few years.

# HG changeset patch
# User ISHIKAWA, Chiaki <ishikawa@yk.rim.or.jp>
# Parent  270e38bde0892e4b8eff0afc3ffe07ed96a7a72f
Avoid conflict of HAVE_VALGRIND_VALGRIND_H definition with upper-level definition.

diff --git a/media/ffvpx/config_unix64.h b/media/ffvpx/config_unix64.h
--- a/media/ffvpx/config_unix64.h
+++ b/media/ffvpx/config_unix64.h
@@ -237,17 +237,17 @@
 #define HAVE_SYS_SELECT_H 1
 #define HAVE_SYS_SOUNDCARD_H 1
 #define HAVE_SYS_TIME_H 1
 #define HAVE_SYS_UN_H 1
 #define HAVE_SYS_VIDEOIO_H 0
 #define HAVE_TERMIOS_H 1
 #define HAVE_UDPLITE_H 0
 #define HAVE_UNISTD_H 1
-#define HAVE_VALGRIND_VALGRIND_H 0
+// avoid conflict: #define HAVE_VALGRIND_VALGRIND_H 0
 #define HAVE_WINDOWS_H 0
 #define HAVE_WINSOCK2_H 0
 #define HAVE_INTRINSICS_NEON 0
 #define HAVE_ATANF 1
 #define HAVE_ATAN2F 1
 #define HAVE_CBRT 1
 #define HAVE_CBRTF 1
 #define HAVE_COPYSIGN 1

(I enforce the "treat warning as errors" for local build and hit upon the redefinition error some years ago.)

Just FYI.

Flags: needinfo?(stransky)
Pushed by malexandru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/708c5d069936
Provide libva wrapper to run Firefox on systems without libva installed, r=jya
https://hg.mozilla.org/integration/autoland/rev/9d4893cc2f99
Provide libva headers to build Firefox without libva-devel installed, r=jya
Status: REOPENED → RESOLVED
Closed: 4 years ago3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 85 Branch

Reopened as three patches are missing:

D90554 Bug 1660336 Add VP8/VP9 VAAPI HW decode code to bundled ffvpx and build it with MOZ_WAYLAND target
D90556 Bug 1660336 Implement FFmpegLibWrapper::LinkVAAPILibs() to link VAAPI libraries
D90557 Bug 1660336 Build ffvpx FFmpegVideoDecoder module with VAAPI support, r?jya

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → NEW

Patches failed to land with message:
"Revisions: D90554 diff 373469 ← D90555 diff 372311 ← D90556 diff 372312 ← D90557 diff 373470

            Details: We're sorry, Lando could not rebase your commits for you automatically. Please manually rebase your commits and try again.

hg error in cmd: hg import --no-commit -s 95 /tmp/tmpagy1zcb1: applying /tmp/tmpagy1zcb1

patching file media/ffvpx/libavutil/moz.build
Hunk #2 FAILED at 62
1 out of 2 hunks FAILED -- saving rejects to file media/ffvpx/libavutil/moz.build.rej
patching file media/ffvpx/config_common.h
Hunk #1 FAILED at 18
1 out of 1 hunks FAILED -- saving rejects to file media/ffvpx/config_common.h.rej
abort: patch failed to apply"

Flags: needinfo?(stransky)

Martin, please re-add the check-in needed tag after rebase.

I see, Thanks.

Flags: needinfo?(stransky)
Pushed by stransky@redhat.com:
https://hg.mozilla.org/integration/autoland/rev/fb8fee4471de
Add VP8/VP9 VAAPI HW decode code to bundled ffvpx and build it with MOZ_WAYLAND target, r=jya
https://hg.mozilla.org/integration/autoland/rev/b6ca98c5a3cc
Implement DMABufSurfaceWrapper and VAAPIDisplayHolder as templates, r=jya
https://hg.mozilla.org/integration/autoland/rev/7315e7b327f7
Implement FFmpegLibWrapper::LinkVAAPILibs() to link VAAPI libraries and use at at FFmpegRuntimeLinker/FFVPXRuntimeLinker, r=jya
https://hg.mozilla.org/integration/autoland/rev/13936ff79b4c
Build ffvpx FFmpegVideoDecoder module with VAAPI support, r=jya,glandium,jgilbert
Flags: needinfo?(stransky)

Hm, the error is on opt/debug Linux x64 build-linux64-base-toolchains-clang Bbc (seems to be clang 5?) which I'm not able to reproduce. The error is:

/builds/worker/fetches/clang/bin/ld.lld: error: can't create dynamic relocation R_X86_64_PC32 against symbol: ff_cos_32 in readonly segment
[task 2020-12-05T21:52:13.384Z] 21:52:13 INFO - >>> defined in fft_float.o
[task 2020-12-05T21:52:13.384Z] 21:52:13 INFO - >>> referenced by fft.asm:394 (/builds/worker/checkouts/gecko/media/ffvpx/libavcodec/x86/fft.asm:394)

Mike, can you help me understand what's going on and how I can reproduce it? Which distro is that? Can I reproduce it locally somehow?
Thanks.

Flags: needinfo?(mh+mozilla)

Looks like it's caused by c++/c code linking together, when I move mozva.cpp to mozva.c the test passes.

Pushed by stransky@redhat.com:
https://hg.mozilla.org/integration/autoland/rev/321cbfd8cbb8
Add VP8/VP9 VAAPI HW decode code to bundled ffvpx and build it with MOZ_WAYLAND target, r=jya
https://hg.mozilla.org/integration/autoland/rev/84ea73688876
Implement DMABufSurfaceWrapper and VAAPIDisplayHolder as templates, r=jya
https://hg.mozilla.org/integration/autoland/rev/c5097975c9c6
Implement FFmpegLibWrapper::LinkVAAPILibs() to link VAAPI libraries and use at at FFmpegRuntimeLinker/FFVPXRuntimeLinker, r=jya
https://hg.mozilla.org/integration/autoland/rev/adbd3858fa50
Build ffvpx FFmpegVideoDecoder module with VAAPI support, r=jya,glandium,jgilbert
Attachment #9191569 - Attachment is obsolete: true
Flags: needinfo?(mh+mozilla)

Hi,
Does that fix also fixes VAAPI HW decode for Flatpak Firefox ?
Thanks

(In reply to antistress from comment #49)

Hi,
Does that fix also fixes VAAPI HW decode for Flatpak Firefox ?
Thanks

It may fix VP8/9 VA-API playback in Flatpak if you have libva and other libraries (intel/amd drivers) installed.

(In reply to Martin Stránský [:stransky] from comment #50)

Thanks !

See Also: → 1737116
See Also: → 1652958
You need to log in before you can comment on or make changes to this bug.