1268929 - High memory usage decoding wav file [@mozilla::MediaRawDataWriter::SetSize]

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Tyson Smith [:tsmith]]

Attachment: test_case.html

Christoph came across this with his wav fuzzer. test_case.html should reproduce the issue quickly. test_case.wav is included for reference.

#0  0x000000000041ac95 in huge_palloc ()
#1  0x000000000041ad42 in imalloc ()
#2  0x000000000041ad78 in je_malloc ()
#3  0x00007f79076fcba2 in mozilla::AlignedBuffer<unsigned char, 32>::EnsureCapacity(unsigned long) () from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#4  0x00007f79077278f9 in mozilla::MediaRawDataWriter::SetSize(unsigned long) ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#5  0x00007f7907841bbe in mozilla::WAVTrackDemuxer::GetFileHeader(mozilla::media::Interval<long> const&) () from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#6  0x00007f7907841de1 in mozilla::WAVTrackDemuxer::FmtChunkParserInit() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#7  0x00007f7907842351 in mozilla::WAVTrackDemuxer::Init() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#8  0x00007f7907842776 in mozilla::WAVDemuxer::InitInternal() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#9  0x00007f7907843488 in mozilla::WAVDemuxer::Init() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#10 0x00007f790779a5cf in mozilla::MediaFormatReader::AsyncReadMetadata() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#11 0x00007f790775f6d7 in mozilla::detail::ProxyRunnable<mozilla::MozPromise<RefPtr<mozilla::MetadataHolder>, mozilla::ReadMetadataFailureReason, true>, mozilla::MediaDecoderReader>::Run() () from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#12 0x00007f79061f2562 in mozilla::AutoTaskDispatcher::TaskGroupRunnable::Run() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#13 0x00007f79061fce99 in mozilla::TaskQueue::Runner::Run() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#14 0x00007f79061fb74a in nsThreadPool::Run() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#15 0x00007f79061f9bde in nsThread::ProcessNextEvent(bool, bool*) ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#16 0x00007f790621f9be in NS_ProcessNextEvent(nsIThread*, bool) ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#17 0x00007f790658c29f in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) () from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#18 0x00007f790656b5ff in MessageLoop::RunInternal() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#19 0x00007f790656b626 in MessageLoop::Run() ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#20 0x00007f79061fa478 in nsThread::ThreadFunc(void*) ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libxul.so
#21 0x00007f79135eafbf in _pt_root ()
   from /home/user/Desktop/browsers/firefox_dbg_clean/libnspr4.so
#22 0x00007f7914a71182 in start_thread (arg=0x7f78f098d700) at pthread_create.c:312
#23 0x00007f7913b7247d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Comment 1

[Tyson Smith [:tsmith]]

Attachment: test_case.wav

Comment 2

[Jean-Yves Avenard [:jya]]

what is there to fix?

Memory allocation in AlignedBuffer are fallible. The OOM will be handled gracefully in a normal build and playback will stop.

https://dxr.mozilla.org/mozilla-central/source/dom/media/MediaData.h#232

reported all the way back to the wave demuxer:
https://dxr.mozilla.org/mozilla-central/source/dom/media/wave/WaveDemuxer.cpp#515

That crash will not happen in a "normal" build.
Sure there's a OOM, but no crash nor ill-effect will result from it.

Comment 3

[Jean-Yves Avenard [:jya]]

if I misread, and it is crashing if this wave is loaded in a audio element with a normal build. Then the issue is in jemalloc itself and it should gracefully returns null when it can't allocate memory

Comment 4

[Nicholas Nethercote [inactive]]

I tried this and I didn't get a crash, though it did bring my Linux desktop machine (which has 32 GiB of RAM) to its knees.

I don't have anything to add here, other than, yes, it looks like jemalloc itself is crashing or aborting.

Comment 5

[Jean-Yves Avenard [:jya]]

We could add a safety threshold in the amount of memory we should attempt to alloc.

2GiB like nsTArray sound like a reasonable upper number. Especially as AlignedBuffer is used in many places as a replacement to nsTArray.

Comment 6

[Tyson Smith [:tsmith]]

jya, you are correct this is not an OOM. I have updated the title to be more accurate. What we were actually seeing was the system (linux) detecting low memory and killing off the processes using a lot memory, in this case Firefox. 

You and njn seem to be on the right track here. Depending on the OS this could DOS the browser and also the machine itself (as njn experienced) I think having a sane memory limit is a good idea.

Comment 7

[Nicholas Nethercote [inactive]]

> You and njn seem to be on the right track here. Depending on the OS this
> could DOS the browser and also the machine itself (as njn experienced)

My mouse cursor wasn't even responsive for a while, which is unusual for this machine.

Comment 8

[Jean-Yves Avenard [:jya]]

Attachment: MozReview Request: Bug 1268929: Limit memory allocated for media to 2GiB max. r?gerald

Review commit: https://reviewboard.mozilla.org/r/50101/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/50101/

Comment 9

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8748017 [details]
MozReview Request: Bug 1268929: Limit memory allocated for media to 2GiB max. r?gerald

https://reviewboard.mozilla.org/r/50101/#review46873

Comment 10

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b844f8c5b192

Comment 11

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/b844f8c5b192