2060830 - Bilibili 96 kHz FLAC fragmented MP4 fails during MSE append with NS_ERROR_FAILURE

Status: ASSIGNED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Dreemurr]

Attachment: 96k-first-fragment.m4s

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:153.0) Gecko/20100101 Firefox/153.0

Steps to reproduce:

Every Bilibili 96 kHz FLAC fragmented MP4 fails during MSE append with NS_ERROR_FAILURE, While chromium accepts the original Bilibili stream on the same machine. Errors only happens when the flac file is 96 kHz. The example video could be https://www.bilibili.com/video/BV1MX3q6QE2b/?p=2

I'll upload the problem audio file that tricks the error when played by MSE+firefox

Actual results:

The player showed failure notice when flac option is selected

Below are bug report by gpt 5.6 sol

Bilibili shows the Hi-Res control and attempts to switch streams, but the
FLAC stream fails immediately.

The Bilibili telemetry sequence is:

• ctrl_flac_show
• ctrl_flac_effect, switch=1
• ctrl_flac_fail approximately 447 ms later

The source is a valid fragmented ISO-BMFF/MP4 audio stream:

• Codec: FLAC
• MP4 codec tag: fLaC
• Sample rate: 96000 Hz
• Bit depth: 24-bit
• Channels: 2
• Source encoder: Lavf59.27.100 / Bilibili XCoder v2.0.2
• Layout: ftyp + moov + sidx + repeated moof/mdat fragments

Zen reports:

• HTMLMediaElement.canPlayType('audio/mp4; codecs="fLaC"'): "probably"
• MediaSource.isTypeSupported('audio/mp4; codecs="fLaC"'): true

However, appending the initialization data and first complete Bilibili media
fragment produces:

• SourceBuffer event: error
• MediaSource.readyState: closed
• MediaError.code: 4
• MediaError.message: NS_ERROR_FAILURE (0x80004005)
• Playback position: 0 seconds

Loading the complete original stream directly also fails during metadata
decoding with:

NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006)

Expected result

Zen should accept the fragmented FLAC MP4 stream through Media Source
Extensions and play it, as Chromium does.

A/B test results

1. Original Bilibili 96 kHz/24-bit fragmented MP4 through MSE:
   Fails with sourcebuffer-error and NS_ERROR_FAILURE.

2. The same FLAC frames extracted to a native .flac file, still 96 kHz/24-bit:
   Plays successfully in Zen.

3. The same FLAC frames remuxed into a non-fragmented MP4, still 96 kHz/24-bit:
   Plays successfully in Zen.

4. The same audio regenerated as a standard fragmented MP4, still
   96 kHz/24-bit, and appended through MSE:
   Plays successfully. SourceBuffer contains 0-10.069333 seconds.

5. A 48 kHz/24-bit FLAC-in-MP4 control sample:
   Plays successfully.

6. Repeating the original test with media.resampling.enabled=true:
   Still fails with the same metadata/MSE error.

Conclusion

This is not a general lack of 96 kHz FLAC decoding support and is not caused
by the audio device or output resampling.

The failure is specific to the fragmented MP4 layout produced for this
Bilibili FLAC stream. Gecko rejects the original sidx/moof/trun fragment
structure, while it accepts the same FLAC frames after remuxing and accepts
a newly generated fragmented MP4 through MSE.

Expected results:

Audio plays normally

Comment 1

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[BugBot [:suhaib / :marco]]

The severity field is not set for this bug.
:jimm, could you have a look please?

For more information, please visit BugBot documentation.

Comment 3

[media-triaging-bot]

Thanks for the very thorough report and especially the A/B matrix — that narrowed this down quickly.

Analysis

Short answer on where the fault lies: this is a bug in Firefox's MP4 parser. There is a secondary non-conformance in the muxer that produced the file, but Firefox would still get this wrong even with a perfectly conformant file, so the Firefox side is the one that needs fixing. This is a container-parsing gap, not a FLAC decoding problem, and not related to the fragmented layout.

In ISO-BMFF, the AudioSampleEntry sample rate is a 16.16 fixed-point field, so its integer part cannot represent any rate above 65535 Hz. mp4parse reads it as f64::from(be_u32(src)? >> 16), so for a 96 kHz stream there is no value it could read that would be correct.

FLAC-in-MP4 handles that by carrying the real rate in the dfLa STREAMINFO block, and isoflac.txt §3.3.1 is explicit that a reader must use it: "the native value from the METADATA_BLOCK_STREAMINFO header in the FLACSpecificBox MUST be read to determine the correct sample rate of the bitstream." Firefox does not do this. mp4parse_capi repairs the rate from codec-specific data for AAC (the esds case), but the FLAC arm validates the 34-byte STREAMINFO and forwards it as an opaque blob without ever reading the rate out of it. Chromium does perform this override, which is why the same stream plays there.

The stream here was muxed by Lavf59.27.100, and FFmpeg through 8.0 writes 0 into the sample-rate field for rates above 65535 (changed in 8.1). That is also non-conformant — the same isoflac.txt section says the field "shall hold the greatest expressible regular division", i.e. 48000.0 for 96 kHz — but it is the lesser problem. Had the muxer written the conformant 48000, Firefox would have accepted the track and played it at half speed instead of refusing to load it. So Firefox is affected by every 96/192 kHz FLAC-in-MP4 stream, conformant ones included; this file just fails loudly rather than silently.

Once the rate is zero, the track fails MP4AudioInfo::IsValid() and is dropped in MP4Demuxer::Init, which demotes the failure to a warning and continues with no audio track (media.playback.warnings-as-errors defaults to false). The two load paths then surface the empty result differently — MediaFormatReader reports NS_ERROR_DOM_MEDIA_METADATA_ERR for a direct load, while TrackBuffersManager reports a bare NS_ERROR_FAILURE for the MSE append — which matches both errors you saw, and both end up as MediaError.code == 4. That also explains the A/B results: the 48 kHz control fits in the field, and a native .flac file goes through a different parser that does read STREAMINFO.

There are two possible places to fix it. It can be done upstream in mp4parse_capi's FLAC arm, mirroring the esds repair that already exists a few lines above, which then requires a vendoring round-trip. Or it can be done entirely in Gecko: the 34-byte STREAMINFO is already in hand in MP4AudioInfo::Update immediately before mRate is assigned from the unrepaired value, and FlacFrameParser already has the 20-bit extraction to reuse.

:kinetik, as the maintainer of mp4parse-rust — which of those two would you prefer? The upstream fix is the more principled home for it and benefits other consumers, but the Gecko-side one needs no vendoring and would be considerably easier to uplift if that is wanted.

Next Steps

Nothing here is blocking the diagnosis, but a few things would confirm it:

1. Open about:config, set media.playback.warnings-as-errors to true, then reload the Bilibili page and retry the Hi-Res FLAC stream. This makes Firefox report the underlying error instead of a generic one — please paste the new error message here. The expected text is along the lines of NS_ERROR_DOM_MEDIA_DEMUXER_ERR - Invalid audio track.
2. The attached 96k-first-fragment.m4s may contain only the moof/mdat fragment. If that is the case, could you also attach the initialization segment (the ftyp+moov part, usually served as a separate init.mp4)? That is the piece that carries the bad sample-rate field.
3. For A/B tests 3 and 4 (the remuxed non-fragmented MP4 and the regenerated fragmented MP4 that both worked), which tool and version did you use to produce them? If those were also muxed with FFmpeg 8.0 or older they would carry the same zero field, so the exact muxer string would be useful to know.

Comment 4

[Matthew Gregan [:kinetik]]

https://github.com/mozilla/mp4parse-rust/pull/454