1711882 - Firefox doesn't play xHE-AAC content (mp4a.40.42)

Status: VERIFIED FIXED

(Core :: Audio/Video: Playback, defect, P3)

Description

[Jean-Yves Avenard [:jya]]

Site doesn't play because mimeType mp4a.40.42 isn't allowed there:
https://searchfox.org/mozilla-central/rev/443f87caa5fadba920b0382e12874693d6c6133a/dom/media/VideoUtils.cpp#664-673

Apple's AudioToolbox and recent version of Android will support it ; not sure about Windows Media Foundation.

Comment 1

[Jim Mathies [:jimm]]

May be patent encumbered. We'll have to investigate the feasibility here.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

It is definitely patent encumbered and I don't think there's much reason to choose it over Opus

Comment 3

[Lev]

Hi everyone! What's the status on this? Chromium-based browsers are able to decode xHE-AAC streams no problem.

It is definitely patent encumbered and I don't think there's much reason to choose it over Opus

Opus isn't supported by .mp4 containers, while xHE-AAC (USAC) is, as demonstrated by the video below. It's unwatchable in Firefox, since there's no sound:

https://odysee.com/@InfinityHD:0/mf_avqlab:c

Moreover, xHE-AAC decoding support has been introduced in ffmpeg 7.1

Could anyone look into this? Thanks.

Comment 4

[Greg Ogonowski]

As the title of this suggests, Firefox doesn't play xHE-AAC/USAC content.
This is not a patent encumberment issue, as Mac (Core Audio) and Windows 11 (Media Foundation) already include the USAC/xHE-AAC codec, and have for quite some time now. This is similar to the AAC codec family.

I believe all that is necessary is to add the following single line:

MIME type:
aCodec.EqualsLiteral("mp4a.40.42"); // MPEG-D USAC (xHE-AAC)

Mozilla source file:
/dom/media/VideoUtils.cpp
https://searchfox.org/mozilla-central/source/dom/media/VideoUtils.cpp
This is where it goes / Line 1118:

bool IsAACCodecString(const nsAString& aCodec) {
return aCodec.EqualsLiteral("mp4a.40.2") || // MPEG4 AAC-LC
aCodec.EqualsLiteral(
"mp4a.40.02") || // MPEG4 AAC-LC(for compatibility)
aCodec.EqualsLiteral("mp4a.40.5") || // MPEG4 HE-AAC
aCodec.EqualsLiteral(
"mp4a.40.05") || // MPEG4 HE-AAC(for compatibility)
aCodec.EqualsLiteral("mp4a.67") || // MPEG2 AAC-LC
aCodec.EqualsLiteral("mp4a.40.29"); // MPEG4 HE-AACv2
}

Here are a few Live HLS USAC/xHE-AAC StreamS with an HLS MSE Player you can use to test against. These play in Safari and Chrome on both Windows 11 and Mac Sequoia:

https://la2.indexcom.com/player/xheaac/
https://la2.indexcom.com/player/big8radiox/
https://la2.indexcom.com/player/eval/player3/

Please contact me if you need more details or more streams.

Thank you.
Greg Ogonowski
StreamS/Modulation Index LLC

Comment 5

[Greg Ogonowski]

(In reply to Greg Ogonowski from comment #4)

As the title of this suggests, Firefox doesn't play xHE-AAC/USAC content.
This is not a patent encumberment issue, as Mac (Core Audio) and Windows 11 (Media Foundation) already include the USAC/xHE-AAC codec, and have for quite some time now. This is similar to the AAC codec family.

I believe all that is necessary is to add the following single line:

MIME type:
aCodec.EqualsLiteral("mp4a.40.42"); // MPEG-D USAC (xHE-AAC)

Mozilla source file:
/dom/media/VideoUtils.cpp
https://searchfox.org/mozilla-central/source/dom/media/VideoUtils.cpp
This is where it goes / Line 1118:

bool IsAACCodecString(const nsAString& aCodec) {
return aCodec.EqualsLiteral("mp4a.40.2") || // MPEG4 AAC-LC
aCodec.EqualsLiteral(
"mp4a.40.02") || // MPEG4 AAC-LC(for compatibility)
aCodec.EqualsLiteral("mp4a.40.5") || // MPEG4 HE-AAC
aCodec.EqualsLiteral(
"mp4a.40.05") || // MPEG4 HE-AAC(for compatibility)
aCodec.EqualsLiteral("mp4a.67") || // MPEG2 AAC-LC
aCodec.EqualsLiteral("mp4a.40.29"); // MPEG4 HE-AACv2
}

Here are a few Live HLS USAC/xHE-AAC StreamS with an HLS MSE Player you can use to test against. These play in Safari and Chrome on both Windows 11 and Mac Sequoia:

https://la2.indexcom.com/player/big8radiox/
https://la2.indexcom.com/player/eval/player3/

Please contact me if you need more details or more streams.

Thank you.
Greg Ogonowski
StreamS/Modulation Index LLC

Comment 6

[Greg Ogonowski]

How to delete a poster's comment?
The previous got double posted, and I wanted to delete it.
Should I file another Bug Report?
:)

Comment 7

[Paul Adenot (:padenot)]

This errors out here: https://pernos.co/debug/Gzmkut1nJwQB5CFTEqfsTw/index.html#f{m[CLju,CG+h_,t[4g,JXyo_,f{e[CLju,CGFO_,s{aduH9YAAA,bAZU,uJ9o97A,oJ9rCoA___/, should be easy to add support (after adding the mimetype check above).

Comment 8

[Paul Adenot (:padenot)]

Attachment: sin1k-opus.mp4

(In reply to Lev from comment #3)

Opus isn't supported by .mp4 containers, while xHE-AAC (USAC) is, as demonstrated by the video below. It's unwatchable in Firefox, since there's no sound

Opus is supported in mp4 / ISOBMFF: https://www.opus-codec.org/docs/opus_in_isobmff.html. Attached is an example simply produced by doing:

ffmpeg -i sin1k.wav -acodec libopus sin1k.mp4

Comment 9

[Alastor Wu [:alwu]]

According to the comment0 and the comment 3, xHE-AAC can be supported on MacOS, Android and Linux (via ffmpeg). In addition, Windows supports xHE-AAC as well, so we should be able to support xHE-AAC on all platforms.

Comment 10

[Kagami Rosylight [:saschanaz] (they/them)]

Just found that Netflix tries to serve mp4a.40.42 to Wolvic and fails (whereas Firefox Nightly on Windows is served with mp4a.40.5 regardless of Chrome Mask being enabled or not).

(FYI in case anyone never heard of Wolvic - it's a VR browser based on Gecko 128 ESR)

Comment 11

[Kagami Rosylight [:saschanaz] (they/them)]

Heh, "Netflix Now Streaming with Fraunhofer’s xHE-AAC Audio on Android Mobile" in 2021, and I ran it on Meta Quest 3S which is based on Android. But the user agent string only says Linux...

Comment 12

[Kagami Rosylight [:saschanaz] (they/them)]

Confirmed that Firefox Nightly for Android gets the same console warning about mp4a.40.42 on Netflix.

Comment 13

[Paul Adenot (:padenot)]

Can someone attach a file (not a stream) that fails to play so we can get going?

Comment 14

[Kagami Rosylight [:saschanaz] (they/them)]

Fraunhofer seems to have some demos: https://www2.iis.fraunhofer.de/AAC/xhe-aac-abr.html

e.g. https://www2.iis.fraunhofer.de/AAC/xHE-AAC_demo_content_for_adaptive_bitrate/Sintel_24kbps_rap5s.mp4 fails to load on Firefox but loads well on Chrome.

Comment 15

[Paul Adenot (:padenot)]

Superb, thanks a lot. Demuxer patch in https://github.com/mozilla/mp4parse-rust/pull/435. Then we'll need to pull it in, do the mime type business previously mentioned and we should be good.

Comment 16

[Greg Ogonowski]

Here is an xHE-AAC(tm)/USAC file and stream resource:

https://db2.indexcom.com/xheaac/

Let me know if you need other samples.
There are plenty more available for the asking.

It would be best to use Mac Core Audio and Windows Media Foundation Audio Decoders.
xHE-AAC/USAC Decoders are available on both operating systems. Microsoft Windows 11 and up. No 10.
ffmpeg should be avoided.

Hope this helps move this overdue development along..
/greg.

Comment 17

[Paul Adenot (:padenot)]

Attachment: Bug 1711882 - Update mp4parse-rust 7128fb537edb806e2ad4e28ec5d804c28a043810. r?#media-playback-reviewers

Comment 18

[Paul Adenot (:padenot)]

Attachment: Bug 1711882 - mach vendor rust. r?#media-playback-reviewers

Comment 19

[Paul Adenot (:padenot)]

Attachment: Bug 1711882 - Add xHE-AAC mime type to IsAACCodecString. r?#media-playback-reviewers

Comment 20

[Paul Adenot (:padenot)]

Attachment: Bug 1711882 - Add a test. r?#media-playback-reviewers

Comment 21

[Paul Adenot (:padenot)]

Attachment: Bug 1711882 - Handle XHE-AAC in DecoderData and MP4Metadata. r?#media-playback-reviewers

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/e0efc1116234
https://hg.mozilla.org/integration/autoland/rev/ed23bded5ff3
Update mp4parse-rust 7128fb537edb806e2ad4e28ec5d804c28a043810. r=media-playback-reviewers,kinetik
https://github.com/mozilla-firefox/firefox/commit/f091421abd58
https://hg.mozilla.org/integration/autoland/rev/77c6aa450c18
mach vendor rust. r=media-playback-reviewers,supply-chain-reviewers,kinetik
https://github.com/mozilla-firefox/firefox/commit/5372352dae87
https://hg.mozilla.org/integration/autoland/rev/0be0f3aac289
Add xHE-AAC mime type to IsAACCodecString. r=media-playback-reviewers,alwu
https://github.com/mozilla-firefox/firefox/commit/9641b112af81
https://hg.mozilla.org/integration/autoland/rev/d8aa93a8f313
Add a test. r=media-playback-reviewers,alwu
https://github.com/mozilla-firefox/firefox/commit/85d4098c0e8f
https://hg.mozilla.org/integration/autoland/rev/e2e5087a5c86
apply code formatting via Lando

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/f92d4ea5e35f
https://hg.mozilla.org/integration/autoland/rev/09ffd71305bc
Revert "Bug 1711882: apply code formatting via Lando" for causing bustages at MP4Metadata.cpp.

Comment 24

[Atila Butkovits]

Backed out for causing bustages at MP4Metadata.cpp.

Backout link: https://hg-edge.mozilla.org/integration/autoland/rev/09ffd71305bc75087bd683ad9b50a765a69862ea

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=e2e5087a5c86dd91cddee8ae9e6c31cb543f3d5e&selectedTaskRun=DzjeHvtIQAqwUnlXqHZOiA.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=518778226&repo=autoland&lineNumber=22296

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/765919f9cea3
https://hg.mozilla.org/integration/autoland/rev/4cbe190b73c8
Update mp4parse-rust 7128fb537edb806e2ad4e28ec5d804c28a043810. r=media-playback-reviewers,kinetik
https://github.com/mozilla-firefox/firefox/commit/c72b178cc588
https://hg.mozilla.org/integration/autoland/rev/c33d4a159927
mach vendor rust. r=media-playback-reviewers,supply-chain-reviewers,kinetik
https://github.com/mozilla-firefox/firefox/commit/825888700bf1
https://hg.mozilla.org/integration/autoland/rev/4a13930ceb79
Add xHE-AAC mime type to IsAACCodecString. r=media-playback-reviewers,alwu
https://github.com/mozilla-firefox/firefox/commit/f91953118e99
https://hg.mozilla.org/integration/autoland/rev/c1981c4a4499
Add a test. r=media-playback-reviewers,alwu
https://github.com/mozilla-firefox/firefox/commit/690ed86afa85
https://hg.mozilla.org/integration/autoland/rev/8062b025a214
Handle XHE-AAC in DecoderData and MP4Metadata. r=media-playback-reviewers,alwu
https://github.com/mozilla-firefox/firefox/commit/ea8f46448a33
https://hg.mozilla.org/integration/autoland/rev/685a0dd21535
apply code formatting via Lando

Comment 26

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/4cbe190b73c8
https://hg.mozilla.org/mozilla-central/rev/c33d4a159927
https://hg.mozilla.org/mozilla-central/rev/4a13930ceb79
https://hg.mozilla.org/mozilla-central/rev/c1981c4a4499
https://hg.mozilla.org/mozilla-central/rev/8062b025a214
https://hg.mozilla.org/mozilla-central/rev/685a0dd21535

Comment 27

[Ryan VanderMeulen [:RyanVM]]

Should we call this out in the Fx143 relnotes?

Comment 28

[Paul Adenot (:padenot)]

Release Note Request (optional, but appreciated)
[Why is this notable]: New codec can be played back on supported OS: xHE-AAC
[Affects Firefox for Android]: Yes (Android 9+)
[Suggested wording]: xHE-AAC playback is now supported on Windows 11 (22H2 and more recent), macOS 10.15 and more recent, and Android 9 and more recent
[Links (documentation, blog post, etc)]: none

Comment 29

[Ryan VanderMeulen [:RyanVM]]

Added to the Fx143 relnotes, thanks.

Comment 30

[Ciprian Georgiu, Desktop QA]

I've repro this issue using an affected Nightly build 142.0a1 (2025-07-08) on Win 11.

The issue is verified as fixed on latest Beta 143.0b2 under Win 11 and macOS 28.