1709329 - MP4 Audio parser breaks on specific audio files

Status: RESOLVED WONTFIX

(Core :: Audio/Video: Playback, defect, P3)

Description

[me]

Attachment: audio.m4a

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0

Steps to reproduce:

1. Download attached file
2. Open it in firefox (file:/// works)

The file has somewhat incorrect metadata in its Chapters:

ffmpeg version n4.4 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'audio.m4a':
Metadata:
major_brand : M4A
minor_version : 512
compatible_brands: M4A isomiso2
title : Chapter 9
artist : Martha Wells
album_artist : Martha Wells
album : Fugitive Telemetry: Murderbot Diaries, Book 6
date : 2021
encoder : Lavf58.76.100
comment : Chapter 9
genre : Audiobook
copyright : ©2021 Martha Wells (P)2021 Recorded Books
Duration: 00:00:30.22, start: 0.000000, bitrate: 127 kb/s
Chapters:
Chapter #0:0: start 0.000000, end 0.000000
Metadata:
title : Chapter 8
Chapter #0:1: start 0.000000, end 30.221000
Metadata:
title : Chapter 9
Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
Stream #0:1(eng): Data: bin_data (text / 0x74786574), 0 kb/s
Metadata:
handler_name : SubtitleHandler

Actual results:

1. got an error (No video with supported format and mimetype found)
2. Error in console:

Media resource file:///home/nemo/Music/audio.m4a could not be decoded. 009 - Chapter 9.m4a
Media resource file:///home/nemo/Music/audio.m4a could not be decoded, error: Error Code: NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006)
Details: virtual RefPtr<MP4Demuxer::InitPromise> mozilla::MP4Demuxer::Init(): Parse MP4 metadata failed

Expected results:

The audio file should have played correctly, as it does in Chromium

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[me]

Doesn't have to do with Chapter parsing. I removed chapter information from the file, and it still fails.

I stripped away the tag information from the file, optimized it with mp4file --optimize and it still fails.

Comment 3

[me]

Attachment: audio-stripped.m4a

ffmpeg version n4.4 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/audio-stripped.m4a':
Metadata:
major_brand : M4A
minor_version : 512
compatible_brands: M4A isomiso2
Duration: 00:00:30.22, start: 0.022993, bitrate: 127 kb/s
Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]

Comment 4

[C.M.Chang[:chunmin]]

Jon, would you mind having a look?

Comment 5

[Jon Bauman [:jbauman:]]

The issue appears to be in mp4parse::read_ds_descriptor. Here's the rest of the stack trace:

read_ds_descriptor at mp4parse/src/lib.rs:3544
find_descriptor at mp4parse/src/lib.rs:3423
read_dc_descriptor at mp4parse/src/lib.rs:3642
find_descriptor at mp4parse/src/lib.rs:3420
read_es_descriptor at mp4parse/src/lib.rs:3681
find_descriptor at mp4parse/src/lib.rs:3417
read_esds at mp4parse/src/lib.rs:3694
read_audio_sample_entry at mp4parse/src/lib.rs:4087
read_stsd at mp4parse/src/lib.rs:4188
read_stbl at mp4parse/src/lib.rs:2856
read_minf at mp4parse/src/lib.rs:2842
read_mdia at mp4parse/src/lib.rs:2830
read_trak at mp4parse/src/lib.rs:2739
read_moov at mp4parse/src/lib.rs:2624
read_mp4

I'm not that familiar with the ES_Descriptor code, so perhaps :kinetik would be able to provide a faster answer

Comment 6

[me]

I keep coming across more such files - have to keep switching to Chrome to play those.

Comment 7

[Matthew Gregan [:kinetik]]

(In reply to me from comment #6)

I keep coming across more such files - have to keep switching to Chrome to play those.

Where are you finding these files, are they common to a particular site?

Apologies for the delay, I'll investigate this week.

Comment 8

[me]

The common source seems to be Audible. I download my books from Audible and strip the DRM using this script.

The exact ffmpeg command that this uses:

ffmpeg -loglevel panic -y -activation_bytes ${activation_bytes[$FILE_CHECKSUM]} -i "$aax_file_from_audible" -c:a copy -vn "$m4a_file"

I tried it against publicly available Audible AAX samples and those seem too old to be affected by this.

Comment 9

[Matthew Gregan [:kinetik]]

The issue with the files I examined appears to be that the GASpecificConfig depend_on_core_coder flag is true, which requires the following 14 bits to indicate the core_coder_delay. The entire GASpecificConfig is only 16 bits in this case (and we've already consumed 15 bits) so core_coder_delay is not present and the parser runs out of bits while trying to read it. As far as I understand, depend_on_core_coder being set doesn't make sense for these files, since there are no other audio layers to reference.

ffmpeg (which Chrome uses for regular media playback) and VLC both play these files because they happen to ignore short reads during ESDS parsing. In both cases, the bit reader returns 0 when there are no more bits to consume, and there's no error handling present to notice the short reads.

Other players I tried, including Chrome's MSE playback (which uses a different AAC ESDS parser), reject these files for the same reason as mp4parse-rust.

The AAX converters I looked at all seem to use ffmpeg to convert the file, which is decrypting and copying the track without examining the ESDS at all. Based on this, the original AAX files must also have this flag set but also have a single audio layer, so I'm not sure why this flag is set. Ideally, the AAX converter tools would produce valid files that were playable by all MP4 parsers.

While trying to find other instances of this, I came across https://rentry.co/n4ost which describes the same issue, also for files converted from AAX. That site includes a script based on Bento4 to clear the depend_on_core_coder flag, making the files playable with mp4parse-rust and other parsers stricter than ffmpeg.

Comment 10

[me]

Thanks a lot. I've re-encoded all my files for now since this looks more like an Audible bug.

Comment 11

[Matthew Gregan [:kinetik]]

That's probably the best approach, thanks.

If it turns out there are more files (from sources other than the Audible conversions) affected by this issue, we can consider modifying mp4parse-rust to accept them. https://github.com/kinetiknz/mp4parse-rust/commit/90478b3020198defb0847c4aa74ce68656f10028 contains one approach, but it's not clear it's the right one. Following ffmpeg's relaxed error handling when parsing the ESDS would be another possibility.

For now, I'll mark this as WONTFIX given there's are workarounds available and the scope is limited. We can revisit if the situation changes.