Unbounded loop in ReenqueueUnplayedAudioDataIfNeeded intermittently leads to OOM
Categories
(Core :: Audio/Video: Playback, defect)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox154 | --- | fixed |
People
(Reporter: yannis, Assigned: yannis)
References
(Blocks 1 open bug)
Details
Crash Data
Attachments
(1 external file)
Seen in CI failures that get incorrectly reported to bug 2049921.
Trees: autoland (18), mozilla-central (2)
Platform: windows11-32-25h2 (20)
Typical failure log:
GECKO(2664) | out of memory: 0x0000000033300000 bytes requested
GECKO(2664) | [7116] Hit MOZ_CRASH(out of memory: 0x0000000033300000 bytes requested) at checkouts\gecko\memory\mozalloc\mozalloc_abort.cpp:32
Initializing stack-fixing for the first stack frame, this may take a while...
GECKO(2664) | #01: mozalloc_abort(char const* const) [git:github.com/mozilla-firefox/firefox:memory/mozalloc/mozalloc_abort.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:32]
GECKO(2664) | #02: mozalloc_handle_oom [D:\task_178286284827486\build\application\firefox\mozglue.dll + 0x291f9]
GECKO(2664) | #03: moz_xrealloc [D:\task_178286284827486\build\application\firefox\mozglue.dll + 0x292ea]
GECKO(2664) | #04: nsTArray_base<nsTArray_RelocateUsingMemutils>::EnsureCapacityImpl<nsTArrayInfallibleAllocator>(unsigned int, unsigned int) [git:github.com/mozilla-firefox/firefox:xpcom/ds/nsTArray.h:629864ddf550a09ce5eb9bffc50f4452353b7f4f:3183]
GECKO(2664) | #05: mozilla::AudioSink::ReenqueueUnplayedAudioDataIfNeeded() [git:github.com/mozilla-firefox/firefox:dom/media/mediasink/AudioSink.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:253]
GECKO(2664) | #06: mozilla::AudioSink::ShutDown() [git:github.com/mozilla-firefox/firefox:dom/media/mediasink/AudioSink.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:295]
GECKO(2664) | #07: mozilla::AudioSinkWrapper::ShutDownAudioSink() [git:github.com/mozilla-firefox/firefox:dom/media/mediasink/AudioSinkWrapper.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:356]
GECKO(2664) | #08: mozilla::AudioSinkWrapper::Stop() [git:github.com/mozilla-firefox/firefox:dom/media/mediasink/AudioSinkWrapper.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:532]
GECKO(2664) | #09: mozilla::VideoSink::Stop() [git:github.com/mozilla-firefox/firefox:dom/media/mediasink/VideoSink.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:284]
GECKO(2664) | #10: mozilla::MediaDecoderStateMachine::StopMediaSink() [git:github.com/mozilla-firefox/firefox:dom/media/MediaDecoderStateMachine.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:3912]
GECKO(2664) | #11: mozilla::MediaDecoderStateMachine::ShutdownState::Enter() [git:github.com/mozilla-firefox/firefox:dom/media/MediaDecoderStateMachine.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:3358]
GECKO(2664) | #12: mozilla::MediaDecoderStateMachine::StateObject::SetState<mozilla::MediaDecoderStateMachine::ShutdownState>() [git:github.com/mozilla-firefox/firefox:dom/media/MediaDecoderStateMachine.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:269]
GECKO(2664) | #13: mozilla::MediaDecoderStateMachine::StateObject::HandleShutdown() [git:github.com/mozilla-firefox/firefox:dom/media/MediaDecoderStateMachine.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:2857]
GECKO(2664) | #14: mozilla::MediaDecoderStateMachine::Shutdown() [git:github.com/mozilla-firefox/firefox:dom/media/MediaDecoderStateMachine.cpp:629864ddf550a09ce5eb9bffc50f4452353b7f4f:3765]
GECKO(2664) | #15: mozilla::detail::ProxyFunctionRunnable<`lambda at .\..\..\..\..\checkouts\gecko\dom\media\MediaDecoderStateMachineBase.cpp:142:7',mozilla::MozPromise<bool,bool,0> >::Run() [git:github.com/mozilla-firefox/firefox:xpcom/threads/MozPromise.h:629864ddf550a09ce5eb9bffc50f4452353b7f4f:1836]
GECKO(2664) | #16: mozilla::AutoTaskDispatcher::TaskGroupRunnable::Run() [git:github.com/mozilla-firefox/firefox:xpcom/threads/TaskDispatcher.h:629864ddf550a09ce5eb9bffc50f4452353b7f4f:235]
...
Failing jobs:
- autoland · mochitest-browser-chrome-2 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-6 · windows11-32-25h2/debug
- mozilla-central · mochitest-browser-chrome-19 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-11 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-7 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-11 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-2 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-3 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-24 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-19 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-14 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-10 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-15 · windows11-32-25h2/debug
- mozilla-central · mochitest-browser-chrome-14 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-6 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-6 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-6 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-7 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-4 · windows11-32-25h2/debug
- autoland · mochitest-browser-chrome-6 · windows11-32-25h2/debug
Likely root cause: if typicalPacketFrameCount is zero, the unbounded loop below from ReenqueueUnplayedAudioDataIfNeeded loops forever, allocating more and more memory each time the data structure needs to grow, and eventually we OOM.
// Extract all audio data from the ring buffer, we can only read the data from
// the most recent, so we reenqueue the data, packetized, in a temporary
// array.
while (framesRemaining) {
uint32_t packetFrameCount =
std::min(framesRemaining, typicalPacketFrameCount);
framesRemaining -= packetFrameCount;
int packetSampleCount = packetFrameCount * channelCount;
AlignedAudioBuffer packetData(packetSampleCount);
DebugOnly<int> samplesRead =
mProcessedSPSCQueue->Dequeue(packetData.Data(), packetSampleCount);
MOZ_ASSERT(samplesRead == packetSampleCount);
packetsToReenqueue.AppendElement(packetData);
}
Note that the same pattern exists in PushFakeAudioDataIfNeeded.
The signatures I found on crash-stats that would match with this issue are low volume, so this is mostly a CI concern as far as I can tell. It seems that our tests can make mFrames reach zero but that this situation is unlikely in the wild. In Claude's words:
Verdict on "test-only": No. The reenqueue-on-teardown path itself is ordinary (it fires on pause / mute / volume change / seek / tab close whenever the ring buffer still holds processed audio — e.g. the
OnMuted -> SetVolume -> VolumeChangedstack seen in the dumps), and the 0-frame trigger is a genuineSetTrimWindowoutcome. What makes it predominantly a test finding is that hitting all conditions at once (trimmed 0-frame front packet and a non-empty ring buffer and sink teardown) is rare; theipc/glue/test/browsersuite hammers exactly this by looping a tiny clip and repeatedly stopping/muting/crashing decoders. A real user with looping/gapless/MSE content and an ill-timed pause could still hit it.
| Assignee | ||
Updated•4 months ago
|
| Assignee | ||
Comment 1•4 months ago
|
||
Both AudioSink::ReenqueueUnplayedAudioDataIfNeeded and
MediaDecoderStateMachine::PushFakeAudioDataIfNeeded derive their packet
size from the front/back audio packet's frame count. A packet can
legitimately report zero frames (e.g. fully trimmed via SetTrimWindow),
which makes packetFrameCount zero, so the loop never decrements its
remaining-frames counter and spins forever, allocating until OOM.
Fall back to the default 1024-frame packet size when the reference
packet has no frames.
| Assignee | ||
Updated•4 months ago
|
Comment 3•4 months ago
|
||
| bugherder | ||
Updated•4 months ago
|
Description
•