1833354 - [Linux] Implement V4L2-M2M hardware-accelerated video decode

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, enhancement)

Description

[David Turner]

Steps to reproduce:

Play back h.264 video in Firefox on a Raspberry Pi 4.

Actual results:

Firefox video playback on the Raspberry Pi currently uses software decode only, and does not use the available decode hardware accelerators. This results in high CPU usage and stuttering/dropped frames at higher resolutions, especially for 60fps video.

Expected results:

Currently hardware-accelerated video decode in Firefox uses VA-API (via FFmpeg). This works for desktop platforms (e.g. nvidia/amd/intel) but is not supported on many arm64 platforms like the Raspberry Pi.

Raspberry Pi exposes the video decode accelerators using the Video4Linux Memory-to-memory (V4L2-M2M) API, the Linux kernel API for video encode/decode acceleration. (The OMX and MMAL APIs are deprecated).

I propose to add support to Firefox for using V4L2-M2M video decode accelerators using the FFmpeg platform, in a manner similar to the existing VA-API support.

Comment 1

[David Turner]

I already have a working implementation of this and would like to submit these changes, I have created this bug for tracking and discussion

Comment 2

[David Turner]

Attachment: Bug 1833354 - Import hwcontext_drm.h header from FFmpeg. r?#media-playback-reviewers

Import the hwcontext_drm.h header from FFmpeg/libavutil. This will be
used by the V4L2-M2M decode accelerator which uses DRM-PRIME buffer
handles for zero-copy decoding.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dbf06230b9c3
Import hwcontext_drm.h header from FFmpeg. r=media-playback-reviewers,padenot DONTBUILD

Comment 4

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/dbf06230b9c3

Comment 5

[David Turner]

Attachment: Bug 1833354 - Fix chroma subsampling in DMABufSurface. r?#media-playback-reviewers

In the ImportPRIMESurfaceDescriptor method we set the sizes of the
buffers for each plane. This was done by successively halving the
buffer width and height for each successive plane. This works for
2-planar formats with 4:2:0 chroma sub-sampling (e.g. NV12) but not for
3-planar 4:2:0 formats (e.g. YUV420). Update this method to work with
all 4:2:0 formats regardless of number of planes.

Comment 6

[David Turner]

Attachment: Bug 1833354 - Add sandbox rules for V4L2 in RDD. r?#media-playback-reviewers

Adding V4L2 hardware acceleration support for video playback requires
the RDD to be allowed to:

• Look in /dev to enumerate available V4L2 devices
• Make sockets to connect to the wayland server
• Use VIDEO-type ioctls

Comment 7

[Martin Stránský [:stransky] (ni? me)]

David, I have some ffmpeg/dmabuf debugging patches available so feel free to ping me if you need anything (like dump decoded dmabuf surfaces to png files or so).

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0f37afc972af
Fix chroma subsampling in DMABufSurface. r=media-playback-reviewers,padenot,stransky

Comment 9

[Darkspirit]

(I'm just a user/tester.)
Thanks for implementing this!
If I understand correctly, bug 1748460 comment 1 wanted to avoid socket in the RDD sandbox.
If possible, please attach a diff of all your changes for an overview.
Maybe there is an alternative, or, if needed, socket could be restricted to ARM64 builds.

Comment 10

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/0f37afc972af

Comment 11

[David Turner]

Attachment: Bug 1833354 - Extend FFmpeg video frame pool to support V4L2. r?stransky,#media-playback-reviewers

Currently the video frame pool used to manage hardware-accelerated
buffers for FFmpeg only supports VAAPI. Extend it to support V4L2 by
adding a new overload of GetVideoFrameSurface which takes an
AVDRMFrameDescriptor (which wraps a DRM-PRIME buffer handle). The
AVDRMFrameDescriptor is converted to a VADRMPRIMESurfaceDescriptor so we
can use the existing machinery in DMABufSurface and avoid duplication
there.

This depends on the drm_fourcc.h header from libdrm, I use the vendored
version of libdrm instead of adding a system dependency.

Comment 12

[David Turner]

Attachment: Bug 1833354 - Add NV12 support for V4L2. r?stransky,#media-playback-reviewers

Add support for the NV12 pixel format to the V4L2 buffer handling code
in FFmpegVideoFramePool.

Depends on D178349

Comment 13

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1833354 [Linux] Add surface dump and YUV plane clear debug code to DMABufSurfaceYUV r?emilio

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/463c2366cbd3
Extend FFmpeg video frame pool to support V4L2. r=stransky,media-playback-reviewers
https://hg.mozilla.org/integration/autoland/rev/29bcf118021e
Add NV12 support for V4L2. r=stransky,media-playback-reviewers

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a6d962be323a
[Linux] Add surface dump and YUV plane clear debug code to DMABufSurfaceYUV r=emilio DONTBUILD

Comment 16

[David Turner]

Attachment: Bug 1833354 - Add V4L2 hw-acceleration to FFmpeg video decoder. r?stransky,#media-playback-reviewers

Extend the FFmpeg video decoder's hardware acceleration support to
include V4L2 in addition to VAAPI. If hardware acceleration is enabled
and VAAPI is not available then attempt to initialise a V4L2-wrapper
codec (by name, because libavcodec does not correctly prioritise these
codecs as hardware-accelerated). If V4L2 is not available then this
step will fail and we fall back to software decoding.

DRM-PRIME buffer format is used for zero-copy decoding in a similar
manner to the VAAPI decoder, except that FFmpeg implements V4L2 using
hardware wrapper decoders instead of using a hwaccel so we don't store a
hwcontext.

This adds a use of the libavcodec avcodec_find_decoder_by_name function,
so I add this to the ffmpeg library wrapper.

Comment 17

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/463c2366cbd3
https://hg.mozilla.org/mozilla-central/rev/29bcf118021e
https://hg.mozilla.org/mozilla-central/rev/a6d962be323a

Comment 18

[David Turner]

Attachment: Bug 1833354 - Add V4L2 hw decode probing to GfxInfo. r?#media-playback-reviewers

In GfxInfo, probe all V4L2 devices to see which of them can do M2M
hardware decode acceleration. Set the hardware decode support features
appropriately depending on what we find.

Comment 19

[Martin Stránský [:stransky] (ni? me)]

I think we need to add a flag to build VA-API support or V4L2 support (or both).

It's set in toolkit/moz.configure file: https://searchfox.org/mozilla-central/source/toolkit/moz.configure

I suggest to add --enable-vaapi and --enable-v4l2 flags here (make it dependable on Linux/Gtk).

Let me know if you have any trouble with moz.configure that so I can help or create the config patch.

Thanks.

Comment 20

[David Turner]

Attachment: Bug 1833354 - Add build options for VAAPI and V4L2. r?#media-playback-reviewers

Add separate options to control whether we build VA-API and/or V4L2
hardware video decode. These options default to enabled on the
platforms where these decoders are used.

The downstream MOZ_WAYLAND_USE_HWDECODE build option is used within the
FFmpeg platform and is only set when building against a supported
version of the FFmpeg libraries (i.e. not for ffmpeg57).

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/12536127a826
Add build options for VAAPI and V4L2. r=media-playback-reviewers,glandium,stransky

Comment 22

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/12536127a826

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c1bf8fa33c09
Add sandbox rules for V4L2 in RDD. r=jld
https://hg.mozilla.org/integration/autoland/rev/599392144b13
Add V4L2 hw-acceleration to FFmpeg video decoder. r=stransky,media-playback-reviewers
https://hg.mozilla.org/integration/autoland/rev/e1fb66fe89b0
Add V4L2 hw decode probing to GfxInfo. r=media-playback-reviewers,stransky

Comment 24

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/c1bf8fa33c09
https://hg.mozilla.org/mozilla-central/rev/599392144b13
https://hg.mozilla.org/mozilla-central/rev/e1fb66fe89b0

Comment 25

[Martin Stránský [:stransky] (ni? me)]

So is that working now? To support VP8/9/AV1 HW decode you'd need to update bundled ffvpx (or disable bundled ffvpx and use system ffmpeg only).

Comment 26

[David Turner]

Yes, h.264 v4l2 is now all finished and should be working on nightly. It would be nice to support more codecs but I don't have any hardware which can do vp8/vp9/av1 decode and I'd rather not push totally untested code.

Let me know if there was any particular neatening/refactoring changes you'd like me to do before I close this bug.

Comment 27

[Robert Mader [:rmader]]

(In reply to David Turner from comment #26)

Let me know if there was any particular neatening/refactoring changes you'd like me to do before I close this bug.

There's one little request from my side: the warning V4L2 device %s does not support M2M modes can be quite noisy on devices with complex cameras. I'd suggest to silence/remove it completely.

Comment 28

[wiagn233]

(In reply to David Turner from comment #26)

Yes, h.264 v4l2 is now all finished and should be working on nightly. It would be nice to support more codecs but I don't have any hardware which can do vp8/vp9/av1 decode and I'd rather not push totally untested code.

Let me know if there was any particular neatening/refactoring changes you'd like me to do before I close this bug.

Could you push other formats' support first? I think I could test that on TDA4VM.

Comment 29

[David Turner]

(In reply to wiagn233 from comment #28)

Could you push other formats' support first? I think I could test that on TDA4VM.

I just had a quick look at the TDA4VM out of curiosity and it looks like it only supports H.264 and H.265 decode. The former is already landed on nightly, it would be good to hear if that works for you. H.265 would be quite a big task because Firefox doesn't support it at all, and support in other browsers (and use on the web) is basically nonexistant, so we decided not to implement it for now.

(In reply to Robert Mader [:rmader] from comment #27)

There's one little request from my side: the warning V4L2 device %s does not support M2M modes can be quite noisy on devices with complex cameras. I'd suggest to silence/remove it completely.

Sure, I'll push another diff to do that. It's also not ideal to have extraneous warnings/errors in about:support which can cause confusion.

Comment 30

[blacknova]

I can test vp8/vp9(but not av1), but I am unsure where to get the nightly build for aarch64 architecture. Would I have to compile it myself?

Comment 31

[Robert Mader [:rmader]]

(In reply to blacknova from comment #30)

I can test vp8/vp9(but not av1), but I am unsure where to get the nightly build for aarch64 architecture. Would I have to compile it myself?

Thanks! Unfortunately we don't have aarch64 nightly builds yet (see bug 1677963) :/ You can build it yourself or use the try server if you have an account with permissions (usually for regular contributors). I triggered a fresh build here: https://treeherder.mozilla.org/jobs?repo=try&revision=27c1a447eaa72a7804461a1996868aabc4e7b078, the file will be at "Linux AArch64 opt"->"B"->"Artifacts and Debugging Tools"->"target.tar.bz2
"

Comment 32

[Martin Stránský [:stransky] (ni? me)]

To test vp8/vp9/av1 decode on aarch64 you need to set media.ffvpx.enabled to false at about:config and restart browser.
It's because bundled ffvpx library (used for vp8/vp9/av1 decode) doesn't have backported V4L2 acceleration from ffmpeg so you need to use system ffmpeg for it.

Comment 33

[wiagn233]

How to check this series is working? The build above says
HARDWARE_VIDEO_DECODING:
default: available,
runtime: unavailable, Force disabled by gfxInfo, Blocklisted; failure code FEATURE_FAILURE_VIDEO_DECODING_TEST_FAILED
at about:support.

Comment 34

[Martin Stránský [:stransky] (ni? me)]

Run Firefox as:

MOZ_GFX_DEBUG=1 MOZ_LOG="PlatformDecoderModule:5" firefox

That logs gfx probe and following playback.

Comment 35

[blacknova]

I have it installed but I am on holiday at the minute. Initial testing on vp9 seems to be very succesful on the pinephone pro

Comment 36

[wiagn233]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #34)

Run Firefox as:

MOZ_GFX_DEBUG=1 MOZ_LOG="PlatformDecoderModule:5" firefox

That logs gfx probe and following playback.

Seems the log doesn't have anything indicating whether using HW or SW for decoding?

Comment 37

[blacknova]

Attachment: about:support for test on pinephone pro with glxinfo error

I got a warning that:
cannot access /sys/bus/pci

Do I need to force enable hardware decoding? Also will be attaching test with vp9. I have tests with vp8 and av1 as well so let me know if you need those.

Comment 38

[blacknova]

Attachment: stderr and stdout for vp9 playback test

Comment 39

[Robert Mader [:rmader]]

Thanks for testing!

(In reply to blacknova from comment #37)

Do I need to force enable hardware decoding?

I think for the moment you'll need media.hardware-video-decoding.force-enabled

Some comments:

(#0) Error glxtest: cannot access /sys/bus/pci

We should probably silence this warning, it's not an error.

(#1) Error glxtest: DRM render node not clearly detectable. Falling back to using the only one that was found.

Same here. It will go away once https://gitlab.freedesktop.org/mesa/mesa/-/issues/5591 or something similar lands, but the warning is only relevant on very niche setups. IIRC both warnings were introduced by me :)

(#2) Error vaapitest: ERROR
(#3) Error vaapitest: VA-API test failed: failed to initialise VAAPI connection.

IMO we should silence these warnings as well and simply report VA-API as being unavailable, maybe with some hint. AFAICS we only print them when other software using VA-API would fail as well, including vainfo.

(#4) Error v4l2test: V4L2 device /dev/video11 does not support M2M modes
(#5) Error v4l2test: V4L2 device /dev/video10 does not support M2M modes
(#6) Error v4l2test: V4L2 device /dev/video9 does not support M2M modes
(#7) Error v4l2test: V4L2 device /dev/video8 does not support M2M modes
(#8) Error v4l2test: V4L2 device /dev/video7 does not support M2M modes
(#9) Error v4l2test: V4L2 device /dev/video6 does not support M2M modes
(#10) Error v4l2test: V4L2 device /dev/video5 does not support M2M modes
(#11) Error v4l2test: V4L2 device /dev/video4 does not support M2M modes

As mentioned above I think we should silence these warnings - they are about camera (sub)devices which of course don't support hardware video decoding.

HARDWARE_VIDEO_DECODING
default available
runtime unavailable Force disabled by gfxInfo Blocklisted; failure code FEATURE_FAILURE_VIDEO_DECODING_TEST_FAILED

The "TEST_FAILED" part seems wrong and is probably caused by the failing VA-API test. Instead IMO we should block it for stable/late beta by a blocklist entry and be allow it for nigthly/early beta.

v4l2test probing device '/dev/video3'
v4l2test driver rockchip-rga card rockchip-rga bus_info platform:rga version 393991
V4L2_SPLANE
TRUE
V4L2_MPLANE
FALSE
V4L2_CAPTURE_FMTS
BA24 BX24 AR24 XR24 RGB3 BGR3 AR12 AR15 RGBP NV21 NV61 NV12 NV16 YU12 422P YV12
V4L2_OUTPUT_FMTS
BA24 BX24 AR24 XR24 RGB3 BGR3 AR12 AR15 RGBP NV21 NV61 NV12 NV16 YU12 422P YV12
V4L2_SUPPORTED
TRUE
v4l2test probing device '/dev/video2'
v4l2test driver hantro-vpu card rockchip,rk3399-vpu-dec bus_info platform:ff650000.video-codec version 393991
V4L2_SPLANE
FALSE
V4L2_MPLANE
TRUE
V4L2_CAPTURE_FMTS
NV12
V4L2_OUTPUT_FMTS
MG2S VP8F
V4L2_SUPPORTED
TRUE
v4l2test probing device '/dev/video1'
v4l2test driver hantro-vpu card rockchip,rk3399-vpu-enc bus_info platform:ff650000.video-codec version 393991
V4L2_SPLANE
FALSE
V4L2_MPLANE
TRUE
V4L2_CAPTURE_FMTS
JPEG
V4L2_OUTPUT_FMTS
YM12 NM12 YUYV UYVY
V4L2_SUPPORTED
TRUE
v4l2test probing device '/dev/video0'
v4l2test driver rkvdec card rkvdec bus_info platform:rkvdec version 393991
V4L2_SPLANE
FALSE
V4L2_MPLANE
TRUE
V4L2_CAPTURE_FMTS
NV12
V4L2_OUTPUT_FMTS
S264 VP9F
V4L2_SUPPORTED
TRUE

Looks good to me.

Comment 40

[Martin Stránský [:stransky] (ni? me)]

I wonder why we see:

GLX_TEST: childgltest finished
[GFX1-]: glxtest: cannot access /sys/bus/pci
[GFX1-]: glxtest: DRM render node not clearly detectable. Falling back to using the only one that was found.
vaapitest start, device /dev/dri/renderD128
vaInitialize failed -1
ERROR

I think it's supposed to have VA-API disabled and V4L2 enabled, right?

Comment 41

[David Turner]

I think it's supposed to have VA-API disabled and V4L2 enabled, right?

arm64 builds have both vaapi and v4l2 enabled at compile-time because I seem to recall seeing some niche embedded platforms which implemented vaapi drivers.

The logs all look as expected given I haven't implemented VP8/9 support yet. The probing works correctly, it doesn't mark VP8/9 as a hw-accelerated format, and so it uses software decode. I'll code up a patch to add VP8/9 hardware-accelerated decode and if @blacknova can test it then we can land it based on that.

Comment 42

[wiagn233]

(In reply to Robert Mader [:rmader] from comment #39)

Thanks for testing!

(In reply to blacknova from comment #37)

Do I need to force enable hardware decoding?

I think for the moment you'll need media.hardware-video-decoding.force-enabled

Some comments:

(#0) Error glxtest: cannot access /sys/bus/pci

We should probably silence this warning, it's not an error.

(#1) Error glxtest: DRM render node not clearly detectable. Falling back to using the only one that was found.

Same here. It will go away once https://gitlab.freedesktop.org/mesa/mesa/-/issues/5591 or something similar lands, but the warning is only relevant on very niche setups. IIRC both warnings were introduced by me :)

(#2) Error vaapitest: ERROR
(#3) Error vaapitest: VA-API test failed: failed to initialise VAAPI connection.

IMO we should silence these warnings as well and simply report VA-API as being unavailable, maybe with some hint. AFAICS we only print them when other software using VA-API would fail as well, including vainfo.

(#4) Error v4l2test: V4L2 device /dev/video11 does not support M2M modes
(#5) Error v4l2test: V4L2 device /dev/video10 does not support M2M modes
(#6) Error v4l2test: V4L2 device /dev/video9 does not support M2M modes
(#7) Error v4l2test: V4L2 device /dev/video8 does not support M2M modes
(#8) Error v4l2test: V4L2 device /dev/video7 does not support M2M modes
(#9) Error v4l2test: V4L2 device /dev/video6 does not support M2M modes
(#10) Error v4l2test: V4L2 device /dev/video5 does not support M2M modes
(#11) Error v4l2test: V4L2 device /dev/video4 does not support M2M modes

As mentioned above I think we should silence these warnings - they are about camera (sub)devices which of course don't support hardware video decoding.

HARDWARE_VIDEO_DECODING
default available
runtime unavailable Force disabled by gfxInfo Blocklisted; failure code FEATURE_FAILURE_VIDEO_DECODING_TEST_FAILED

The "TEST_FAILED" part seems wrong and is probably caused by the failing VA-API test. Instead IMO we should block it for stable/late beta by a blocklist entry and be allow it for nigthly/early beta.

v4l2test probing device '/dev/video3'
v4l2test driver rockchip-rga card rockchip-rga bus_info platform:rga version 393991
V4L2_SPLANE
TRUE
V4L2_MPLANE
FALSE
V4L2_CAPTURE_FMTS
BA24 BX24 AR24 XR24 RGB3 BGR3 AR12 AR15 RGBP NV21 NV61 NV12 NV16 YU12 422P YV12
V4L2_OUTPUT_FMTS
BA24 BX24 AR24 XR24 RGB3 BGR3 AR12 AR15 RGBP NV21 NV61 NV12 NV16 YU12 422P YV12
V4L2_SUPPORTED
TRUE
v4l2test probing device '/dev/video2'
v4l2test driver hantro-vpu card rockchip,rk3399-vpu-dec bus_info platform:ff650000.video-codec version 393991
V4L2_SPLANE
FALSE
V4L2_MPLANE
TRUE
V4L2_CAPTURE_FMTS
NV12
V4L2_OUTPUT_FMTS
MG2S VP8F
V4L2_SUPPORTED
TRUE
v4l2test probing device '/dev/video1'
v4l2test driver hantro-vpu card rockchip,rk3399-vpu-enc bus_info platform:ff650000.video-codec version 393991
V4L2_SPLANE
FALSE
V4L2_MPLANE
TRUE
V4L2_CAPTURE_FMTS
JPEG
V4L2_OUTPUT_FMTS
YM12 NM12 YUYV UYVY
V4L2_SUPPORTED
TRUE
v4l2test probing device '/dev/video0'
v4l2test driver rkvdec card rkvdec bus_info platform:rkvdec version 393991
V4L2_SPLANE
FALSE
V4L2_MPLANE
TRUE
V4L2_CAPTURE_FMTS
NV12
V4L2_OUTPUT_FMTS
S264 VP9F
V4L2_SUPPORTED
TRUE

Looks good to me.

Nope, rockchip devices are using v4l2-request rather than v4l2-m2m for decoders, only RGA is using v4l2-m2m, so should actually using software decoder. I think we need some output indicating whether actually using hardware decoders.

Comment 43

[David Turner]

Attachment: Bug 1833354 - Quieten V4L2 support warnings. r?#media-playback-reviewers,rmader

v4l2test emits errors/warnings for every device which isn't an M2M
decoder, which can be quite a lot of devices. So only emit these as
warnings instead of critical errors. To see the output from v4l2test,
set gfx.logging.level = 5 in about:config.

Comment 44

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b8d3a882e4cb
Quieten V4L2 support warnings. r=rmader

Comment 45

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/b8d3a882e4cb

Comment 46

[blacknova]

(In reply to David Turner from comment #41)

I think it's supposed to have VA-API disabled and V4L2 enabled, right?

arm64 builds have both vaapi and v4l2 enabled at compile-time because I seem to recall seeing some niche embedded platforms which implemented vaapi drivers.

The logs all look as expected given I haven't implemented VP8/9 support yet. The probing works correctly, it doesn't mark VP8/9 as a hw-accelerated format, and so it uses software decode. I'll code up a patch to add VP8/9 hardware-accelerated decode and if @blacknova can test it then we can land it based on that.

Yep I am still here and ready to test.

(In reply to wiagn233 from comment #42)

Nope, rockchip devices are using v4l2-request rather than v4l2-m2m for decoders, only RGA is using v4l2-m2m, so should actually using software decoder. I think we need some output indicating whether actually using hardware decoders.

Is that going to be a problem? Is there possibility for v4l2-request support in firefox? Or is that going to have to be v4l2-m2m support for rockchips?

Comment 47

[David Turner]

(In reply to blacknova from comment #46)

Is that going to be a problem? Is there possibility for v4l2-request support in firefox? Or is that going to have to be v4l2-m2m support for rockchips?

The problem is that ffmpeg doesn't support v4l2-requests codecs as far as I know, and firefox depends on ffmpeg for its hardware-accelerated codec support on Linux.

Comment 48

[pbrobinson]

arm64 builds have both vaapi and v4l2 enabled at compile-time because I seem to recall seeing some niche embedded platforms which implemented vaapi drivers.

There are devices which use the discrete AMD/NVIDIA GPUs too on aarch64.

Comment 49

[ramanarvind]

(In reply to Robert Mader [:rmader] from comment #31)

(In reply to blacknova from comment #30)

I can test vp8/vp9(but not av1), but I am unsure where to get the nightly build for aarch64 architecture. Would I have to compile it myself?

Thanks! Unfortunately we don't have aarch64 nightly builds yet (see bug 1677963) :/ You can build it yourself or use the try server if you have an account with permissions (usually for regular contributors). I triggered a fresh build here: https://treeherder.mozilla.org/jobs?repo=try&revision=27c1a447eaa72a7804461a1996868aabc4e7b078, the file will be at "Linux AArch64 opt"->"B"->"Artifacts and Debugging Tools"->"target.tar.bz2
"

I tried this build and there was a clear difference in the CPU cycles consumed while doing H.264 decode. Tried decoding https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/720/Big_Buck_Bunny_720_10s_1MB.mp4 with the build shared above and the stock Firefox available with Ubuntu and I could notice that the CPU cycles were lesser using the build shared above.

I am trying to build a WebRTC application on a Raspberry Pi 4B. While the above reasonably solves the video decode problem I could see from the Firefox logs that it continues to use OpenH264 for H.264 video encode. Anything that can be done to get Firefox to use H.264 accelerated encode?

Thanks and Best regards
Arvind

Comment 50

[Robert Mader [:rmader]]

Anything that can be done to get Firefox to use H.264 accelerated encode?

For encode we have tracker issue bug 1658900 (should be expanded to V4L2 now) but I don't think anyone has been working on it yet.

Comment 51

[Robert Mader [:rmader]]

Btw., David, I think we should close this bug now and create new ones for follow-ups - is that ok with you?

Comment 52

[David Turner]

That sounds good to me, I didn't have anything else planned for V4L2 decode specifically.