1805102 - [AMD] AV1 VAAPI fails Frame 0: size 852x480 upscaled 852 render 852x480 subsample 2x2 bitdepth 8 tiles 1x1. Update ffvpx to ffmpeg 6.1

Status: RESOLVED WORKSFORME

(Core :: Audio/Video: Playback, enhancement)

Description

[Ye Jingchen]

Attachment: firefox-vaapi.log

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0

Steps to reproduce:

My hardware is HP Elitebook G9 running Arch Linux, with an "AMD Ryzen 7 PRO 6850HS with Radeon Graphics" CPU.

0. Open Firefox with environment variable MOZ_LOG="PlatformDecoderModule:5"
1. Open this video: https://www.bilibili.com/video/BV1EW411X7uA/
2. Switch to 480P, which is available without logging in
3. Right click inside video player, select the last item ("Video statistics")
4. Check that video codecs is indeed "av01"
5. Start playing

Actual results:

The videos plays fine, but looking at the logs, after "FFMPEG: VA-API FFmpeg init successful" it says:

[av1 @ 0x7f01a901f000] Failed to read unit 0 (type 5).
[av1 @ 0x7f01a901f000] Failed to read packet.

which might indicate a failure to decode the video; but later Firefox seems to decide to disable hardware decoding and use dav1d instead:

FFMPEG: VAAPI is disabled by parent decoder module.
FFMPEG:   codec libdav1d : dav1d AV1 decoder by VideoLAN

And dav1d plays the video successfully.

Expected results:

The AV1 video should utilize hardware decoding as it should be.

Note that AV1 hardware decoding works with some other AV1 videos on Bilibili, for example https://www.bilibili.com/video/BV1tD4y1y7Z5, and YouTube AV1 videos (haven't met one where AV1 hwdec fails on YT).

Comment 1

[Ye Jingchen]

Attachment: My about:support info

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[Darkspirit]

Thanks for the report!
Does this problem still exist with latest https://nightly.mozilla.org?

Comment 4

[Ye Jingchen]

Oh, I forgot to mention that I set media.ffvpx.enabled = false when first reported this issue, so it was using the system ffmpeg 2:5.1.2-1 I assume. But flipping it on and off doesn't seem to change the behavior: they both failed to hwdec the first video, and succeeded on the second one. Just that in the logs are FFMPEG and FFVPX respectively.

Retested with Arch package firefox 108.0.1-1 build id 20221216153905. Will try nightly.

Comment 5

[Ye Jingchen]

Tried nightly 110.0a1 20221220093956 with a fresh new profile, and the problem still exists: failed to hwdec first video, success on the second one.

Logs are nearly identical too: first comes FFVPX: VA-API FFmpeg init successful, then

[av1 @ 0x7ff99ce29c00] Failed to read unit 0 (type 5).
[av1 @ 0x7ff99ce29c00] Failed to read packet.

and finally

FFVPX: VAAPI is disabled by parent decoder module.
FFVPX:   codec libdav1d : dav1d AV1 decoder by VideoLAN

See the full nightly log.

Comment 6

[Ye Jingchen]

Attachment: firefox nightly failed to hwdec av1

Comment 7

[Darkspirit]

(Ye Jingchen from comment 6)

[RDD 50516: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: Supported accelerated formats:
[RDD 50516: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: h264
[RDD 50516: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: vp9
[RDD 50516: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: av1
[RDD 50516: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: VA-API FFmpeg init successful
[RDD 50516: MediaPDecoder #1]: V/PlatformDecoderModule ProcessDecode: mDuration=33000µs ; mTime=0µs ; mTimecode=0µs
[av1_frame_split @ 0x7ff970f5e280] Frame 0: size 852x480 upscaled 852 render 852x480 subsample 2x2 bitdepth 8 tiles 1x1.
[av1 @ 0x7ff99ce29c00] Failed to read unit 0 (type 5).
[av1 @ 0x7ff99ce29c00] Failed to read packet.
[RDD 50516: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: avcodec_send_packet error: Not yet implemented in FFmpeg, patches welcome
[RDD 50516: MediaPDecoder #2]: D/PlatformDecoderModule FFVPX: ProcessFlush()
[Child 50412: MediaSupervisor #2]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type video/av1
[Child 50412: MediaSupervisor #2]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type video/av1
[RDD 50516: MediaSupervisor #1]: D/PlatformDecoderModule FFmpeg decoder supports requested type 'video/av1'
[RDD 50516: MediaSupervisor #1]: D/PlatformDecoderModule FFmpeg decoder supports requested type 'video/av1'
[RDD 50516: MediaSupervisor #1]: D/PlatformDecoderModule FFVPX: FFmpegVideoDecoder::FFmpegVideoDecoder MIME video/av1 Codec ID 226
[RDD 50516: MediaSupervisor #1]: D/PlatformDecoderModule FFVPX: VAAPI is disabled by parent decoder module.
[RDD 50516: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: Initialising FFmpeg decoder
[RDD 50516: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: codec libdav1d : dav1d AV1 decoder by VideoLAN

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

Comment 9

[Ye Jingchen]

I wondered whether this problem lies within Firefox or FFMPEG, so I tried using ffplay[2] to test a relatively small video[1]:

1. Download it in AV1 format using yt-dlp: yt-dlp -S vcodec:av01 -o bilibili-av1-test.mp4 'https://www.bilibili.com/video/BV1Gv4y1e79H' .
2. Try ffplay without options ffplay bilibili-av1-test.mp4: video plays fine using libdav1d; some Unknown Metadata OBU type 6 are printed but doesn't seem to affect anything.
3. Try forcing VAAPI ffplay -vcodec av1 bilibili-av1-test.mp4: video window only shows a frequency spectrum graph? Familiar error messages are printed in terminal:

[av1 @ 0x7f8480030a40] Failed to read unit 1 (type 5).
[av1 @ 0x7f8480030a40] Failed to read extradata.

So it seems upstream FFMPEG is to be blamed. Should I report this issue to them?

[1] https://www.bilibili.com/video/BV1Gv4y1e79H, 3 minutes long, 9.3MB (480p AV1)
[2] Arch Linux ffmpeg 2:6.0-8 https://archlinux.org/packages/extra/x86_64/ffmpeg/

Comment 10

[Ye Jingchen]

Attachment: ffplay.log

ffplay log when trying VAAPI

Comment 11

[Martin Stránský [:stransky] (ni? me)]

Can you try mpv player? as

mpv --hwdec=vaapi test_clip

on terminal.
Thanks.

Comment 12

[Ye Jingchen]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #11)

Can you try mpv player? as

mpv --hwdec=vaapi test_clip

on terminal.
Thanks.

Of course:

% mpv --hwdec=vaapi bilibili-av1-test.mp4 
 (+) Video --vid=1 (*) (av1 854x466 30.000fps)
 (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
File tags:
 Description: Bilibili VXCode Swarm Transcoder v0.7.52
[ffmpeg/video] av1: Failed to read unit 1 (type 5).
[ffmpeg/video] av1: Failed to read extradata.
Could not open codec.
[ffmpeg/video] libdav1d: Unknown Metadata OBU type 6
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [gpu-next] 854x466 yuv420p
AV: 00:00:04 / 00:03:11 (2%) A-V:  0.000 Dropped: 1
[ffmpeg/video] libdav1d: Unknown Metadata OBU type 6
(Paused) AV: 00:00:05 / 00:03:11 (3%) A-V:  0.000 Dropped: 1

Exiting... (Quit)
% mpv --version
mpv 0.35.1-dirty Copyright © 2000-2023 mpv/MPlayer/mplayer2 projects
 built on Fri Apr 21 07:11:25 2023
FFmpeg library versions:
   libavutil       58.2.100
   libavcodec      60.3.100
   libavformat     60.3.100
   libswscale      7.1.100
   libavfilter     9.3.100
   libswresample   4.10.100
FFmpeg version: n6.0

Comment 13

[Ye Jingchen]

(In reply to Ye Jingchen from comment #12)

(In reply to Martin Stránský [:stransky] (ni? me) from comment #11)

Can you try mpv player? as

mpv --hwdec=vaapi test_clip

on terminal.
Thanks.

MPV also plays fine after fallback to software decoding.

Comment 14

[Martin Stránský [:stransky] (ni? me)]

Please report at Mesa bug tracker as it looks like a bug in gfx drivers: https://gitlab.freedesktop.org/mesa/mesa/-/issues
Thanks.

Comment 15

[Martin Stránský [:stransky] (ni? me)]

Can you please download the clip and attach it here? I'm unable to reproduce but I'm not sure I have correct environment. Also local file is better for reliable debugging.
Thanks.

Comment 16

[Ye Jingchen]

Attachment: bilibili-av1-test.mp4

Comment 17

[Martin Stránský [:stransky] (ni? me)]

Yes, I do see that too:

[av1_frame_split @ 0x7fc31a5e5600] Frame 0:  size 854x466  upscaled 854  render 854x466  subsample 2x2  bitdepth 8  tiles 1x1.
[av1 @ 0x7fc314169000] Failed to read unit 0 (type 5).
[av1 @ 0x7fc314169000] Failed to read packet.
[RDD 217991: MediaPDecoder #1]: D/PlatformDecoderModule FFVPX: avcodec_send_packet error: Not yet implemented in FFmpeg, patches welcome

Comment 18

[Martin Stránský [:stransky] (ni? me)]

We correctly fallback to SW decode. So there's nothing actionable on Firefox side, it's a bug in ffmpeg itself.

Comment 19

[Ye Jingchen]

I tried using ffmpeg-git 6.1.r111115.g8e21f32fe7-1 from AUR, and it can use VAAPI to successfully decode Bilibili's AV1 video, both in mpv and Firefox, so I assume this is also fixed upstream :). ffmpeg 6.1 is planned to be released in July, so once that happens (and Firefox updates to that version?), AV1 videos in bilibili.com can be hardware decoded by default.

Comment 20

[Martin Stránský [:stransky] (ni? me)]

(In reply to Ye Jingchen from comment #19)

I tried using ffmpeg-git 6.1.r111115.g8e21f32fe7-1 from AUR, and it can use VAAPI to successfully decode Bilibili's AV1 video, both in mpv and Firefox, so I assume this is also fixed upstream :). ffmpeg 6.1 is planned to be released in July, so once that happens (and Firefox updates to that version?), AV1 videos in bilibili.com can be hardware decoded by default.

That would need bundled ffvpx update in Firefox then.

Comment 21

[Ye Jingchen]

Just tried this again. FFvpx bundled with Firefox 119 is able to utilize VAAPI for Bilibili's AV1 videos, so this bug can probably be marked as resolved...?

BTW, the newly released ffmpeg 6.1 (by setting media.ffvpx.enabled = false) also works in this case.