Open Bug 1646329 Opened 8 months ago Updated 2 months ago

Use VA-API decoder with WebRTC

Categories

(Core :: Audio/Video: Playback, enhancement)

enhancement

Tracking

()

People

(Reporter: stransky, Assigned: stransky)

References

(Depends on 1 open bug, Blocks 1 open bug)

Details

Let's use VA-API decoder with WebRTC, from my experience it's VP8 streams which can be processed by ffmpeg.

Just a note, that Chromium, last time I looked, has VP8 GPU decoding disabled.

(In reply to Paul Menzel from comment #1)

Just a note, that Chromium, last time I looked, has VP8 GPU decoding disabled.

On ChromeOS? (Chrome uses VAAPI only on ChromeOS.)

from my experience it's VP8 streams which can be processed by ffmpeg.

Jitsi also supports x264 and vp9 (not to mention upcoming AV1 support at some point) - the default is still vp8 though (see e.g. 1). Just as a note that we won't be stuck with vp8 forever and therefore it's great to have VAAPI support :)

1: https://community.jitsi.org/t/flagship-vp9-support-notes-and-discussion/33921/53

Thanks for planning it.

For the reference, see comments form libwebrtc project here: https://bugs.chromium.org/p/webrtc/issues/detail?id=11296

Chromium supposedly already supports, but outside the library.

What about encoder though? That might have even bigger value for WebRTC.

Agree, low-end tablets, laptops and Compute Sticks based on Intel Atom (BayTrail and CherryTrail generations) would greatly benefit from hardware-based encoding.

See Also: → consolidation
Depends on: 1655747

The WebRTC decoding should work now in latest nightly, you just need to set media.ffmpeg.low-latency.enabled to true at about:config.

Depends on: 1657845

(In reply to Shmerl from comment #5)

What about encoder though? That might have even bigger value for WebRTC.

I don't think so. On a regular call you typically need to decode many streams for various participant but encode only one from you.
I'd love to see any testing/numbers if the vaapi decoding here helps or not.

VA-API based encoding will need WebRTC hacking to get images from camera exported as dmabuf surfaces and encoded directly in GPU memory, WebRTC uses kNative surface type for that.

Assignee: nobody → stransky

I don't think so. On a regular call you typically need to decode many streams for various participant but encode only one from you.

My assumption was simple. Encoding is always more intense, so even if it's a simple 1:1 call, encoding will load the CPU significantly. For a laptop, it drains the battery. Some testing and comparison of course would be good, but I'd assume it can also depend on the used codec and actual encoder implementation.

(In reply to Shmerl from comment #9)

I don't think so. On a regular call you typically need to decode many streams for various participant but encode only one from you.

My assumption was simple. Encoding is always more intense, so even if it's a simple 1:1 call, encoding will load the CPU significantly. For a laptop, it drains the battery. Some testing and comparison of course would be good, but I'd assume it can also depend on the used codec and actual encoder implementation.

That could be, but this issue is about decoding. Please create a separate issue.

This can be used now with both Wayland and X11 EGL backends.

Summary: [Wayland] Use VA-API decoder with WebRTC → Use VA-API decoder with WebRTC
Depends on: 1659054

btw. it also needs enabled media.navigator.mediadatadecoder_vpx_enabled in release.
Also that may work from FF81 as WebRTC/VP8 is sometimes encoded to YV12 which is supported by Bug 1655747.

i read that zoom uses parts of webrtc protocol, does that mean it could use vaapi decoder?

Depends on: 1665324
Depends on: 1665329
See Also: → 1656252
You need to log in before you can comment on or make changes to this bug.