1894818 - Support HEVC playback on Linux

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, enhancement, P2)

Description

[Zipdox]

Firefox for Android

Steps to reproduce:

Use Firefox on Linux with a VA-API capable GPU with HEVC decoding support.
Set media.ffmpeg.vaapi.enabled in about:config to true.
Try https://tests.caniuse.com/?feat=hevc

Actual results:

The video doesn't play and displays "No video with supported format and MIME type".

Expected results:

The video plays using VA-API for decoding.

Windows recently gained HEVC support using MFT, I believe Linux should have it too with VA-API.
https://bugzilla.mozilla.org/show_bug.cgi?id=1853448

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Darkspirit]

(I'm a user.)
Mozilla shouldn't do it because it would have to pay patent license fees.
Please encode to AV1 or VP9 video with opus audio instead. AVIF is also a powerful replacement for JPEG.
The dav1d software decoder is being ported/transpiled to Rust: https://www.memorysafety.org/initiative/av1/
Hardware decoding is supported since bug 1745285.

Comment 3

[Zipdox]

(In reply to Darkspirit from comment #2)

(I'm a user.)
Mozilla shouldn't do it because it would have to pay patent license fees.
Please encode to AV1 or VP9 video with opus audio instead. AVIF is also a powerful replacement for JPEG.
The dav1d software decoder is being ported/transpiled to Rust: https://www.memorysafety.org/initiative/av1/
Hardware decoding is supported since bug 1745285.

No they don't. Using VA-API offloads decoding to the GPU, whose vendor already paid the licensing fees. Like I said, HEVC is already available on Windows with MFT.

Comment 4

[Darkspirit]

GPU vendors don't pay all necessary codec patent license fees. HEVC is only available on Windows if the user buys HEVC video extensions.

Comment 5

[Zipdox]

Firefox is not implementing HEVC. Firefox leaves it up to the OS. VA-API would do the same thing. Most distros ship with HEVC enabled, but Fedora disabled it. You can get the HEVC extension for free on Windows as well. But your comment prompted me to email AMD about licensing of codecs. I'll comment here when I get a response.

Comment 6

[Dominik Mierzejewski]

(In reply to Zipdox from comment #5)

Firefox is not implementing HEVC. Firefox leaves it up to the OS. VA-API would do the same thing. Most distros ship with HEVC enabled, but Fedora disabled it.

No, Firefox doesn't support it even if the OS (VA-API) does. On my machine:

$ vainfo |grep HEVC
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointFEI
      VAProfileHEVCMain               :	VAEntrypointEncSliceLP
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointEncSliceLP
      VAProfileHEVCMain12             :	VAEntrypointVLD
      VAProfileHEVCMain12             :	VAEntrypointEncSlice
      VAProfileHEVCMain422_10         :	VAEntrypointVLD
      VAProfileHEVCMain422_10         :	VAEntrypointEncSlice
      VAProfileHEVCMain422_12         :	VAEntrypointVLD
      VAProfileHEVCMain422_12         :	VAEntrypointEncSlice
      VAProfileHEVCMain444            :	VAEntrypointVLD
      VAProfileHEVCMain444            :	VAEntrypointEncSliceLP
      VAProfileHEVCMain444_10         :	VAEntrypointVLD
      VAProfileHEVCMain444_10         :	VAEntrypointEncSliceLP
      VAProfileHEVCMain444_12         :	VAEntrypointVLD
      VAProfileHEVCSccMain            :	VAEntrypointVLD
      VAProfileHEVCSccMain            :	VAEntrypointEncSliceLP
      VAProfileHEVCSccMain10          :	VAEntrypointVLD
      VAProfileHEVCSccMain10          :	VAEntrypointEncSliceLP
      VAProfileHEVCSccMain444         :	VAEntrypointVLD
      VAProfileHEVCSccMain444         :	VAEntrypointEncSliceLP
      VAProfileHEVCSccMain444_10      :	VAEntrypointVLD
      VAProfileHEVCSccMain444_10      :	VAEntrypointEncSliceLP
...

However, Firefox (132.0.1) Fedora 41 build still doesn't play HEVC-encoded movies.

Comment 7

[Zipdox]

(In reply to Dominik Mierzejewski from comment #6)

(In reply to Zipdox from comment #5)

Firefox is not implementing HEVC. Firefox leaves it up to the OS. VA-API would do the same thing. Most distros ship with HEVC enabled, but Fedora disabled it.

No, Firefox doesn't support it even if the OS (VA-API) does. On my machine:

$ vainfo |grep HEVC
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointFEI
      VAProfileHEVCMain               :	VAEntrypointEncSliceLP
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointEncSliceLP
      VAProfileHEVCMain12             :	VAEntrypointVLD
      VAProfileHEVCMain12             :	VAEntrypointEncSlice
      VAProfileHEVCMain422_10         :	VAEntrypointVLD
      VAProfileHEVCMain422_10         :	VAEntrypointEncSlice
      VAProfileHEVCMain422_12         :	VAEntrypointVLD
      VAProfileHEVCMain422_12         :	VAEntrypointEncSlice
      VAProfileHEVCMain444            :	VAEntrypointVLD
      VAProfileHEVCMain444            :	VAEntrypointEncSliceLP
      VAProfileHEVCMain444_10         :	VAEntrypointVLD
      VAProfileHEVCMain444_10         :	VAEntrypointEncSliceLP
      VAProfileHEVCMain444_12         :	VAEntrypointVLD
      VAProfileHEVCSccMain            :	VAEntrypointVLD
      VAProfileHEVCSccMain            :	VAEntrypointEncSliceLP
      VAProfileHEVCSccMain10          :	VAEntrypointVLD
      VAProfileHEVCSccMain10          :	VAEntrypointEncSliceLP
      VAProfileHEVCSccMain444         :	VAEntrypointVLD
      VAProfileHEVCSccMain444         :	VAEntrypointEncSliceLP
      VAProfileHEVCSccMain444_10      :	VAEntrypointVLD
      VAProfileHEVCSccMain444_10      :	VAEntrypointEncSliceLP
...

However, Firefox (132.0.1) Fedora 41 build still doesn't play HEVC-encoded movies.

I'm talking about how HEVC works on Windows. I consider MFT part of the OS.

Comment 8

[Atmospheric Ignition]

I'm also unable to play HEVC video with Firefox on Arch Linux, despite having x265, and other libraries necessary for decoding HEVC installed, and being able to play HEVC with other video players.

Since Firefox now supports HEVC playback through hardware decoding on macOS, I don't really see a reason the same approach of offloading the decoding to VA-API wouldn't be possible on Linux. That is the same approach Chromium is taking to support HEVC on Linux.
Firefox for Android also seems to support HEVC decoding using this same trick.

vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: VA-API NVDEC driver [direct backend]
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileAV1Profile0            :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain12             :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD
      VAProfileHEVCMain444            :	VAEntrypointVLD
      VAProfileHEVCMain444_10         :	VAEntrypointVLD
      VAProfileHEVCMain444_12         :	VAEntrypointVLD

Comment 9

[Alastor Wu [:alwu]]

This is already on our roadmap, we will implement HEVC support on Linux as well.

Comment 10

[Alastor Wu [:alwu]]

I'm going to work on this, currently all other platforms are already supporting HEVC. Linux is the last platform, I'm aiming to finish this on either Fx137 or Fx138.

Comment 11

[Alastor Wu [:alwu]]

Attachment: Bug 1894818 - part1 : support HEVC via VAAPI.

Comment 12

[Alastor Wu [:alwu]]

Attachment: Bug 1894818 - part2 : add gfx feature for hevc.

Comment 13

[Alastor Wu [:alwu]]

Attachment: Bug 1894818 - part3 : support HEVC for ffmpeg video decoder.

Comment 14

[Alastor Wu [:alwu]]

Attachment: Bug 1894818 - part4 : enable hevc check on Linux.

Comment 15

[Alastor Wu [:alwu]]

Attachment: Bug 1894818 - part5 : append SEI and ensure the NALUs are appended in the order of nalu's type.

Comment 16

[Alastor Wu [:alwu]]

Attachment: Bug 1894818 - part6 : properly report the codec support for FFmpegDecoderModule.

Comment 17

[Zipdox]

Glad to see someone's finally working on this.

Comment 19

[Alastor Wu [:alwu]]

Attachment: Bug 1894818 - part7 : guard HEVC support by the pref.

Comment 20

[Martin Stránský [:stransky] (ni? me)]

Do we really support HEVC/H.265 on Linux? AFAIK SW decoding is disabled for patent reasons, has it been enabled recently?

Comment 21

[[e-mail]]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #20)

Do we really support HEVC/H.265 on Linux? AFAIK SW decoding is disabled for patent reasons, has it been enabled recently?

VA-API is HW decoding, not SW

Comment 22

[Zipdox]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #20)

Do we really support HEVC/H.265 on Linux? AFAIK SW decoding is disabled for patent reasons, has it been enabled recently?

No mainstream browser except Safari supports HEVC software decoding, indeed due to patent licensing reasons (Apple is part of the patent pool). They all offload it to hardware, as does Firefox. Although in my opinion we should just give Via-LA the finger and implement software decoding anyway, and put it behind a compiler flag. Most distros ship with HEVC-enabled software anyway.

Comment 23

[Maximilian Böhm]

While I’m not categorically against such decoding support for HEVC via VA-API, I’m actually confused. Didn’t we want to push royalty-free web codecs such as VP9 and AV1 for a patent-free multi-platform future? Making H.265 available everywhere will lead to lots of sites abandoning the royality-free web codecs, making support for H.265 mandatory for every FLOSS browser on every platform. In my opinion, this is a disservice to the ideas of the open web.
It seems this has been already implemented in the Windows version without a debate about the implications, I find this really irritating: https://bugzilla.mozilla.org/show_bug.cgi?id=1853448

Comment 24

[Atmospheric Ignition]

While I understand the frustration about patent impeded codecs, Firefox for Windows already supports HEVC playback through hardware decoding. Firefox for macOS has HEVC hardware decoding riding the trains, and it looks like it will land in Firefox 136. If HEVC was not going to be supported to encourage the use of patent-free codecs, that conversation should have happened earlier, because it looks like the ship has already sailed. And I don't see a reason for HEVC playback through hardware acceleration not to be supported exclusively on Linux.

As a user, I'm increasingly running into issues with video playback on sites like GitHub, which do no transcode uploaded videos from their original codecs, and with many modern phones shooting HEVC video by default, which means that I always have to have Chromium open to the side just to view these videos.

This is also a pain for self-hosted photo/video backup solutions like Immich. Because Firefox doesn't support HEVC playback, and most of my videos are encoded as HEVC (default for many modern phones), I have to transcode all my videos to VP9, AV1, or H264. Transcoding that many videos took a week on my home server, and was done with the sole purpose of being able to watch my own videos specifically in Firefox.

While there is definitely an argument to be made about encouraging the use of royalty-free codecs, all other browsers already support HEVC in some way, and I'm not about to go ask random people uploading to GitHub issues to "Please use a royalty-free video codec for the videos you attach to issues."

Most normal users do not know what a "royalty-free codec" and do not care about anything other than their websites working properly, and if Firefox fails to accommodate that, then it's just another feature Firefox is missing. The patents on HEVC are also very dependent on jurisdiction, with my jurisdiction having no enforceable patents on HEVC.

Comment 25

[Zipdox]

(In reply to mabo from comment #23)

While I’m not categorically against such decoding support for HEVC via VA-API, I’m actually confused. Didn’t we want to push royalty-free web codecs such as VP9 and AV1 for a patent-free multi-platform future? Making H.265 available everywhere will lead to lots of sites abandoning the royality-free web codecs, making support for H.265 mandatory for every FLOSS browser on every platform. In my opinion, this is a disservice to the ideas of the open web.
It seems this has been already implemented in the Windows version without a debate about the implications, I find this really irritating: https://bugzilla.mozilla.org/show_bug.cgi?id=1853448

HEVC isn't fully supported outside of Safari. Like I said, there's no software support for it in any other browser. You also forgot that service providers have to pay royalties to offer HEVC encoded content. Not supporting it in Firefox just leads to Firefox being left behind. Firefox' market share is already so low that many services don't care to support/test for it, so not implementing HEVC is a disservice to both the users and to Firefox itself (since it risks losing more market share).

Besides, most content providers and large companies are already on board with AV1.

Comment 26

[Atmospheric Ignition]

I should also mention that support for HW accelerated HEVC is much more common than HW accelerated AV1, especially in anything more than 3 years old. Despite software-based AV1 decoders being really power efficient in my experience, they still cannot match the power efficiency of HW accelerated HEVC.

Comment 27

[Alastor Wu [:alwu]]

Mozilla is definitely not promoting the use of proprietary codecs, but that doesn't mean we should avoid supporting HEVC altogether. As others have pointed out, Firefox is the only major browser lacking HEVC support, leading to significant compatibility issues for users. Poor video compatibility is a major factor in users switching to other browsers, and we don’t want to risk losing more users due to this limitation.

That said, we will not include a HEVC decoder in Firefox. Instead, we will rely on the system ffmpeg to determine whether HEVC decoding is available, following the same approach we use for H.264.

Comment 28

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5937811c915c
part1 : support HEVC via VAAPI. r=stransky
https://hg.mozilla.org/integration/autoland/rev/74c1ba3a1ca8
part2 : add gfx feature for hevc. r=stransky
https://hg.mozilla.org/integration/autoland/rev/2677fb479185
part3 : support HEVC for ffmpeg video decoder. r=stransky
https://hg.mozilla.org/integration/autoland/rev/2eac4d91c956
part4 : enable hevc check on Linux. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/3ed08095d11c
part5 : append SEI and ensure the NALUs are appended in the order of nalu's type. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/3f32cc520dbf
part6 : properly report the codec support for FFmpegDecoderModule. r=stransky
https://hg.mozilla.org/integration/autoland/rev/b32d076dc9dd
part7 : guard HEVC support by the pref. r=media-playback-reviewers,stransky

Comment 29

[Cristina Horotan [:chorotan]]

Backed out for causing build bustages and mda perma failure

Backout link

Push with failures

Failure log build bustage
Failure log mda

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/10a7adf51b3e
part1 : support HEVC via VAAPI. r=stransky
https://hg.mozilla.org/integration/autoland/rev/0f1981a34a9f
part2 : add gfx feature for hevc. r=stransky
https://hg.mozilla.org/integration/autoland/rev/ef50ff463b34
part3 : support HEVC for ffmpeg video decoder. r=stransky
https://hg.mozilla.org/integration/autoland/rev/631490ea5485
part4 : enable hevc check on Linux. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/ec1384ecdb3b
part5 : append SEI and ensure the NALUs are appended in the order of nalu's type. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/07a1ce16f451
part6 : properly report the codec support for FFmpegDecoderModule. r=stransky
https://hg.mozilla.org/integration/autoland/rev/2cf227217636
part7 : guard HEVC support by the pref. r=media-playback-reviewers,stransky

Comment 31

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/10a7adf51b3e
https://hg.mozilla.org/mozilla-central/rev/0f1981a34a9f
https://hg.mozilla.org/mozilla-central/rev/ef50ff463b34
https://hg.mozilla.org/mozilla-central/rev/631490ea5485
https://hg.mozilla.org/mozilla-central/rev/ec1384ecdb3b
https://hg.mozilla.org/mozilla-central/rev/07a1ce16f451
https://hg.mozilla.org/mozilla-central/rev/2cf227217636

Comment 32

[Pascal Chevrel:pascalc]

Is that worth mentioning in our release notes or not?

Comment 33

[Alastor Wu [:alwu]]

Release Note Request (optional, but appreciated)
[Why is this notable]: Support HEVC playback on Linux
[Affects Firefox for Android]: No
[Suggested wording]: : Support HEVC playback on Linux
[Links (documentation, blog post, etc)]: No

Comment 34

[Martin Stránský [:stransky] (ni? me)]

(In reply to Alastor Wu [:alwu] from comment #27)

That said, we will not include a HEVC decoder in Firefox. Instead, we will rely on the system ffmpeg to determine whether HEVC decoding is available, following the same approach we use for H.264.

Note that right now (with the patches here) only HW decoding is supported while H.264 is supported by both SW and HW decoding by system ffmpeg.

Comment 35

[Alastor Wu [:alwu]]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #34)

Note that right now (with the patches here) only HW decoding is supported while H.264 is supported by both SW and HW decoding by system ffmpeg.

No, the current implementation has the same capabilities as H.264. SW decoding is supported if the system's ffmpeg supports it.

Comment 36

[Pascal Chevrel:pascalc]

Relnote tracking moved to bug 1950032