Closed Bug 2050988 Opened 4 months ago Closed 4 months ago

WebCodecs audio encoder mis-sizes the resampler output buffer (signed-int overflow and floor-to-zero)

Categories

(Core :: Audio/Video: Web Codecs, defect, P3)

defect

Tracking

()

RESOLVED FIXED
154 Branch
Tracking Status
firefox-esr115 --- unaffected
firefox-esr140 --- wontfix
firefox-esr153 --- wontfix
firefox152 --- wontfix
firefox153 --- wontfix
firefox154 --- fixed

People

(Reporter: chunmin, Assigned: chunmin)

References

Details

(Keywords: crash, csectype-intoverflow, sec-other, Whiteboard: [adv-main154-])

Attachments

(4 external files)

Summary

FFmpegAudioEncoder::EncodeInputWithModernAPIs sizes the resampler output buffer from a single estimate, int bufferLengthGuess = ceil(2 * audio.size() * codecRate / inputRate), and derives the per-channel output capacity as outputFrames = bufferLengthGuess / numberOfChannels. The estimate is computed in a signed 32-bit int with no overflow check and divided with truncation, so it is wrong at both extremes. For a large up-sampled input it overflows int and drives an out-of-range buffer allocation, aborting the process (crash/OOM) — reachable from a plain web page with no compromise. For a small multi-channel down-sampled input it floors to zero, so the resampler is handed zero output capacity, consumes no input, and the input frame is silently dropped (lost/garbled audio on release; a release-fatal MOZ_ASSERT on assertion builds). Both are reachable from ordinary web content through the WebCodecs AudioEncoder API. This is a correctness/stability issue, not a memory-safety one: the over-allocation aborts before any write, and the under-consumption only loses data.

Severity

Low to moderate. No memory unsafety. Failure mode A is a content-triggerable crash/OOM of the encoder (RDD) process — a denial of service. Failure mode B is silent loss of encoded audio on release builds and an assertion abort on debug/Nightly/early-Beta builds. Both require an unusual but entirely valid WebCodecs configuration, so real-world impact is mostly limited to misbehaving or adversarial callers.

Affected code

All citations are pinned to revision a88a631360c11753a059a9792693cbe12383404e. The defect is in the resampler branch of FFmpegAudioEncoder::EncodeInputWithModernAPIs: the size estimate at int bufferLengthGuess = std::ceil(2. * AssertedCast<float>(audio.size()) * mConfig.mSampleRate / mInputSampleRate), the allocation at mTempBuffer.SetLength(bufferLengthGuess), and the per-channel capacity at uint32_t outputFrames = bufferLengthGuess / mConfig.mNumberOfChannels that is then passed to speex_resampler_process_interleaved_float.

Root cause and crux

A single expression sizes the buffer for both directions of resampling, and it is unsafe at both ends. It is evaluated as a double and narrowed to a signed int with no range check, and mTempBuffer.SetLength(bufferLengthGuess) trusts the result. Separately, the 2x over-sizing margin — intended to guarantee that one resample call consumes all of its input — is applied to the total interleaved sample count and then divided by the channel count with truncating integer division, so the per-channel outputFrames can round down to zero. The crux is that bufferLengthGuess/outputFrames have neither a lower bound (at least one frame per channel) nor an upper bound (no more than INT_MAX, or a sane cap), so the same line over-allocates for large up-sampled input and under-allocates — to zero — for small down-sampled input.

Failure mode A: signed-int overflow leads to over-allocation / OOM crash

Trigger: a large AudioData whose input rate is below the codec rate, so the resampler up-samples (e.g. Opus snaps to 48000 Hz and the configured input rate is lower). 2. * audio.size() * codecRate / inputRate exceeds INT_MAX, so the double-to-int conversion is out of range (undefined; in practice a huge or negative value), and mTempBuffer.SetLength(bufferLengthGuess) requests an out-of-range allocation that aborts via infallible-allocation MOZ_CRASH/OOM before the resample runs.

  • Expected: a large up-sampled input is sized to a valid buffer, or the request is rejected gracefully.
  • Actual: the encoder process aborts (OOM/crash) from a plain web page, with no error surfaced to script.

Failure mode B: floor-to-zero leads to under-consumption / data loss

Trigger: a small multi-channel AudioData whose input rate is above the codec rate, so the resampler down-samples (e.g. input 384000 Hz, Opus codec rate 48000 Hz, two or more channels). bufferLengthGuess becomes smaller than the channel count, so outputFrames = bufferLengthGuess / numberOfChannels floors to 0. With one stereo frame at 384000 to 48000: bufferLengthGuess = ceil(2 * 2 * 48000 / 384000) = ceil(0.5) = 1, which is less than channels = 2, so outputFrames = 1 / 2 = 0. The interleaved-float resampler contract treats the output length as a hard cap and consumes input only while output capacity remains, so with zero capacity it consumes nothing: inputFramesProcessed comes back as 0 while inputFrames is non-zero. The resampled span is then rebuilt as a zero-length span at audio = Span<float>(mTempBuffer.Elements(), outputFrames * mConfig.mNumberOfChannels) and the empty span is handed to mPacketizer->Input(audio.data(), audio.Length() / mConfig.mNumberOfChannels) — the input frame is silently dropped.

  • Expected: small down-sampled frames are consumed and accumulate until a packet's worth is reached; the encoder produces output.
  • Actual: every such frame is dropped, so the encoder produces no or garbled audio on release builds; on assertion-enabled builds the drop first trips MOZ_ASSERT(inputFrames == inputFramesProcessed) and aborts the process.

How the configuration is reached from content

WebCodecs AudioEncoder accepts a numberOfChannels up to 256 and a sampleRate in the range 3000 to 384000 Hz. At encoder init the requested rate is recorded as the input rate at mInputSampleRate = AssertedCast<int>(mConfig.mSampleRate) and is snapped to the codec's nearest supported rate; when the recorded input rate differs from the encoder rate, a resampler is created. Each input AudioData must carry the configured input rate and channel count or it is rejected at the input rate/channel gate. Choosing an input rate below the codec rate with a large buffer reaches failure mode A; choosing an input rate above the codec rate with small multi-channel frames reaches failure mode B.

Expected vs actual results

Scenario Expected Actual
Mode A: large, up-sampled input (input rate < codec rate) buffer sized to a valid length, or graceful rejection int overflow -> out-of-range SetLength -> infallible-alloc OOM/crash, no error to script
Mode B: small multi-channel, down-sampled input (input rate > codec rate) input consumed; output produced after enough frames outputFrames == 0 -> input dropped (release) / MOZ_ASSERT abort (debug, Nightly, early Beta)

Suggested fix direction

Compute the per-channel output frame count first, apply the 2x margin and a floor of at least one frame per channel after the division, and bound the total with checked arithmetic so the estimate can neither round to zero nor overflow. For example, size the buffer as (ceil(perChannelInputFrames * codecRate / inputRate) + filterMargin) * numberOfChannels using a CheckedInt/saturating computation and a sane maximum. Once the per-channel capacity is guaranteed positive and the total is bounded, the existing consumption assertion holds, mode A can no longer over-allocate, and the regression tests pass.

The rating assigned by our bug rating tool (currently under development) is:

Rating: N/A

Explanation: Verified against source. Not a security vulnerability. The signed-int overflow (Mode A) only leads to an over-sized infallible nsTArray::SetLength that aborts on OOM before any write; there is no buffer overrun because outputFrames is derived from the same bufferLengthGuess (outputFrames*channels <= bufferLengthGuess), so the resampler never writes past the allocation. Net effect is a recoverable, content-triggered OOM/crash of the media utility (RDD) encoder process — a single-process DoS the page sets up itself with a multi-GB input — which the rules do not treat as a security bug. The floor-to-zero case (Mode B) is silent audio data loss on release; the MOZ_ASSERT it trips is debug-only, not a release assertion. No memory corruption, info disclosure, spoofing, or privilege escalation. Stability/correctness bug (S4), not security-sensitive.

chunmin: Are you ok with removing the security flag or is there something that was missed in the rating?

Flags: needinfo?(cchang)
Attached file (secure) —

A load crashtest: one large AudioData at an input rate below the codec rate makes the resampler output-buffer size estimate (int bufferLengthGuess) overflow, driving an out-of-range mTempBuffer allocation in the encoder process. Passes once the resampler allocation is bounded/validated.

Attached file (secure) —

Feeds the real FFmpegAudioEncoder (Opus, 384000 Hz snapped to 48000 -> a
downsampling resampler) a batch of one-frame stereo inputs and asserts encoded
output is produced. The per-channel resampler output capacity is
bufferLengthGuess / channels, which floors to zero for such small downsampled
frames, so today every frame is dropped and the test fails; it passes once the
output-buffer sizing keeps the capacity positive.

Attached file (secure) —

Implement PEMFactory::CreateEncoder for audio usage, mirroring the
video branch.

Attached file (secure) —

Size the resampler output buffer with checked uint32 arithmetic and a
fallible allocation. The previous unchecked signed-int expression could
mis-size the buffer: a small multi-channel downsample floored the
per-channel capacity to zero, so the resampler consumed no input. Use
floor+1 per channel so the per-channel capacity stays >= 1, reject sizes
the checked arithmetic reports as invalid, and allocate mTempBuffer
fallibly.

Depends on: 2052386
Pushed by cchang@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/68c1bc8f6c18 https://hg.mozilla.org/integration/autoland/rev/2bd5973cc9d6 Support audio encoders in PEMFactory::CreateEncoder r=media-playback-reviewers,karlt https://github.com/mozilla-firefox/firefox/commit/ea4e38dd00ab https://hg.mozilla.org/integration/autoland/rev/003ae5a8d75d Add gtest checking AudioEncoder rejects an oversized resampled output r=karlt https://github.com/mozilla-firefox/firefox/commit/462d2f76f4cd https://hg.mozilla.org/integration/autoland/rev/ff23fe2687e4 Add gtest asserting the audio encoder consumes small downsampled input r=karlt https://github.com/mozilla-firefox/firefox/commit/9b4ccc515ea8 https://hg.mozilla.org/integration/autoland/rev/9939a6ec957c Size the audio resampler output buffer with checked arithmetic r=media-playback-reviewers,karlt

Backed out for causing build bustages at FFmpegAudioEncoder.cpp
Backout Link
Push with failures
Failure Log
Failure line /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegAudioEncoder.cpp:X:31: error: cannot pass an arithmetic expression of built-in types to 'CheckedInt<unsigned long, 0>'

Pushed by cchang@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/3bdafc48e860 https://hg.mozilla.org/integration/autoland/rev/6e267fbc73ef Support audio encoders in PEMFactory::CreateEncoder r=media-playback-reviewers,karlt https://github.com/mozilla-firefox/firefox/commit/8e4a7da8a45a https://hg.mozilla.org/integration/autoland/rev/4b6a11ecb3c2 Add gtest checking AudioEncoder rejects an oversized resampled output r=karlt https://github.com/mozilla-firefox/firefox/commit/52af3b745a2a https://hg.mozilla.org/integration/autoland/rev/82c811476921 Add gtest asserting the audio encoder consumes small downsampled input r=karlt https://github.com/mozilla-firefox/firefox/commit/674af2e58ed1 https://hg.mozilla.org/integration/autoland/rev/6630b3b4f84f Size the audio resampler output buffer with checked arithmetic r=media-playback-reviewers,karlt
Flags: needinfo?(cchang) → in-testsuite+

The patch landed in nightly and beta is affected.
:chunmin, is this bug important enough to require an uplift?

For more information, please visit BugBot documentation.

Flags: needinfo?(cchang)
See Also: → 2059164

Is this something we should uplift to ESR153 still? Please add an uplift request if yes.

Flags: needinfo?(cchang)
Flags: needinfo?(cchang)

I think we can skip the uplift. The original code, without the fix, may feed the wrong value to the underlying resampler, but shouldn't expose the uninitialized memory to web content, and I found no OOB read happens.

Whiteboard: [adv-main154-]
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: