1925446 - AV1 8K 60 fps stop working

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P4)

Description

[Roberto Aldà]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36

Steps to reproduce:

Hi,

my system is:
Ubuntu Desktop 24.04.01 LTS
FIrefox Nightly 132.0a1 (2024-09-22) (64-bit)
Firefox flag:media.av1.enabled=true
Nvidia RTX 4000 SFF Ada Generation 20Gb (supports AV1 HW decoding)
Nvidia Driver Version: 550.107.02
elFarto/nvidia-vaapi-driver correctly installed and working(https://github.com/elFarto/nvidia-vaapi-driver)

when i play video 4k/8k at 30 or 60fps H264,VP9 i have no problem at all, nvidia hw acceleration works well,an video are smooth.
Instead when i play AV1 files they works well at 4k 30/60fps and 8k 30fps,
but with AV1 8K 60 fps after few seconds the nvidia hw acceleration stops working and the load move to the CPU causing stuttering.

The sample video used has 77Mb/s of video bitrate , color space YUV 420.
I've already report this issue to nvidia-vaapi-driver prject issues:https://github.com/elFarto/nvidia-vaapi-driver/issues/333

Is an nvidia-vaapi-driver issue or a FIrefox issue?

Thank you.

Actual results:

When i play AV1 8K 60 fps after few seconds the nvidia hw acceleration stops working and the load move to the CPU causing stuttering.

Expected results:

Video AV1 8K 60 fps can play smoothly

Comment 1

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Paul Adenot (:padenot)]

Reporter, can you follow the following steps:

• Go to about:support and copy-paste its contents here (raw data)
• Please create a log and share it here. Steps are:
  • In Firefox, ideally freshly started with no other tabs, go to about:logging in a new tab
  • Select the Media preset and click Start Logging
  • Reproduce the bug
  • Back on about:logging, click Stop Logging
  • In the new tab that appears with the Firefox Profiler web application, in the top right click the button to upload the profile
  • Make sure hidden threads are included and upload, then share the link here or send privately to a Mozilla developer
  • A video that demonstrates these steps : https://paul.cx/public/about-logging-presentation.webm

This will tell use what part of the entire thing is slow and if we're effectively falling back to CPU decoding.

Comment 3

[Roberto Aldà]

Hi,

here you can download about:support RAW data:
https://drive.google.com/file/d/13ogPqfkAdBdafFCXbP261Tk9eRR0-Rsc/view?usp=sharing

You can download Firefox logs from here:
https://drive.google.com/file/d/1RR76B7lRf-vYD8WG4MbzbRB1vlPOB4Mr/view?usp=sharing

Thank you.

Comment 4

[Paul Adenot (:padenot)]

We should be a bit more lenient here, or special case enormous videos. Martin, do you feel this to be risky? Maybe having a threshold instead of a count?

Comment 5

[Martin Stránský [:stransky] (ni? me)]

The main idea with HW decode is to check if it works and have smooth transition to SW decode otherwise. We enabled HW decode by default only because we have the smooth and quick fallback. I think we may add a pref to force HW decode regardless of performance but IMHO we should not keep the broken HW decode running by default.

Comment 6

[Paul Adenot (:padenot)]

In this case, it's not broken and is better than the sw decoder.

Comment 7

[Martin Stránský [:stransky] (ni? me)]

(In reply to Paul Adenot (:padenot) from comment #6)

In this case, it's not broken and is better than the sw decoder.

Do you know exact reason why it switched to SW? I don't see it in the log. AFAIK we switch to SW if HW it not able to keep video fps. We can't say if the video is just large or there's a driver bug or anything else and we assume SW always works and it's safe fallback.

Better decision would need to compare SW and HW decode time and choose the best variant but that may delay the decode start. Also decode time usually depends on position in the clip according to actual content.

The case here is a bit extreme and corner one, for instance my hardware (AMD Radeon RX 6600 XT) also fails to decode such video in HW while SW decoder fails too (Threadripper workstation with 64 cores and 200GB ram). I don't think we should adjust the decoder for such extreme scenario. Especially when we support VA-API on Intel/AMD only, nvidia-vaapi-driver is a bridge for NVIDIA NVDEC interface and needs extra tweaks on user side.

Comment 8

[Martin Stránský [:stransky] (ni? me)]

But we can add a pref to force HW decode and not fallback to SW. That may solve it here. VA-API is disable by default for NVIDIA and you need to flip prefs anyway to get it working.

Comment 9

[Paul Adenot (:padenot)]

Let's add a pref.

Comment 10

[Paul Adenot (:padenot)]

Attachment: Bug 1925446 - Add a pref to disable the software fallback logic when decoding in hardware with ffmpeg on Linux. r?#media-playback-reviewers

Comment 11

[Paul Adenot (:padenot)]

Roberto, you'll be soon able to disable this fall back, using media.ffmpeg.disable-software-fallback, thanks for filing the bug.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/719a63004f3d
Add a pref to disable the software fallback logic when decoding in hardware with ffmpeg on Linux. r=media-playback-reviewers,aosmond

Comment 13

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/719a63004f3d

Comment 14

[Roberto Aldà]

Hi,
thank you very much for the new pref media.ffmpeg.disable-software-fallback !
It will be possible to have all the preferences which enable video hw decoding in Firefox release stable?
Pres are:

gfx.webrender.all
media.ffmpeg.vaapi.enabled
media.hardware-video-decoding.enabled

If so, how long will it take approximately?

Thank you.