2021769 - (CVE-2026-6747) Missing ArrayBuffer detach in RTCEncodedAudioFrame/RTCEncodedVideoFrame destructor leads to use-after-free

Status: RESOLVED FIXED

(Core :: WebRTC, defect)

Description

[Nan Wang[:sakura]]

Attachment: asan.log

Missing ArrayBuffer detach in RTCEncodedAudioFrame/RTCEncodedVideoFrame destructor leads to use-after-free

Summary

The WebRTC Encoded Transform API exposes encoded frame payloads as ArrayBuffers backed by native memory. A destructor ordering issue in RTCEncodedAudioFrame and RTCEncodedVideoFrame prevents the base class from detaching the ArrayBuffer before the backing store is freed, leaving a dangling reference accessible from JavaScript. This affects all platforms. The preference media.peerconnection.scripttransform.enabled must be true, which is enabled by default in Nightly builds.

Affected Versions

The vulnerable code was introduced in commit 837490d82a91 on 2023-07-20 (Bug 1631263: Implement RTCRtpScriptTransform). That commit added RTCEncodedAudioFrame, RTCEncodedVideoFrame, and the base class RTCEncodedFrameBase with the destructor ordering issue described below. All Firefox versions from that point onward are affected. Testing was performed on mozilla-central commit 9deea8b0a7f3 (Firefox 150.0a1, Nightly build 20260307210637).

Root Cause

The constructor in RTCEncodedFrameBase (dom/media/webrtc/jsapi/RTCEncodedFrameBase.cpp, lines 48 through 50) creates an ArrayBuffer that directly aliases the webrtc frame payload memory:

mData = JS::NewArrayBufferWithUserOwnedContents(
    jsapi.cx(), mState.mFrame->GetData().size(),
    (void*)(mState.mFrame->GetData().data()));

SpiderMonkey's contract for NewArrayBufferWithUserOwnedContents requires the caller to invoke JS::DetachArrayBuffer before freeing the backing store. The base class destructor attempts to honor this contract by calling its DetachData() method:

RTCEncodedFrameBase::~RTCEncodedFrameBase() { DetachData(); }

void RTCEncodedFrameBase::DetachData() {
  if (mGlobal) {
    AutoJSAPI jsapi;
    if (NS_WARN_IF(!jsapi.Init(mGlobal))) {
      return;
    }
    JS::Rooted<JSObject*> rootedData(jsapi.cx(), mData);
    if (rootedData) {
      JS::DetachArrayBuffer(jsapi.cx(), rootedData);
    }
  }
}

However, both derived class destructors clear mData before the base class destructor runs:

RTCEncodedAudioFrame::~RTCEncodedAudioFrame() {
  mData = nullptr;
  mozilla::DropJSObjects(this);
}

RTCEncodedVideoFrame::~RTCEncodedVideoFrame() {
  mData = nullptr;
  mozilla::DropJSObjects(this);
}

C++ destruction order is derived first, so the derived destructor sets mData to nullptr, and then the base destructor's DetachData() finds rootedData is null and skips the detach call entirely. After that, the RTCEncodedFrameState destructor frees the webrtc frame payload through unique_ptr destruction. Any JavaScript ArrayBuffer references obtained from frame.data before the object was collected now point to freed memory.

The cycle collector UNLINK path (RTCEncodedFrameBase.cpp, lines 18 through 23) is safe because it calls DetachData() before clearing mData. The problem occurs only when objects are destroyed via the C++ destructor chain triggered by GC or CC deletion, where the ArrayBuffer is never detached.

Reproduce

Testing was performed on Firefox Nightly 150.0a1 (BuildID 20260307210637) from mozilla-central revision 9deea8b0a7f33f6410ce3ee1b718a9bc3b394ea5, downloaded via fuzzfetch as an ASAN-optimized build for Linux x86-64. No source modifications are required.

The proof of concept consists of two files: poc.html and poc_worker.js. The poc.html creates a loopback WebRTC connection using an AudioContext oscillator as the media source and attaches an RTCRtpScriptTransform to the sender. The poc_worker.js handles the rtctransform event, extracts frame.data ArrayBuffer references and saves them to a global array without retaining the frame objects themselves. After accumulating several references, it forces garbage collection through allocation pressure to trigger destruction of the frame wrappers. Once the frame wrappers are collected, the saved ArrayBuffer references point to freed payload memory. Subsequent reads or writes through these ArrayBuffers access freed heap memory, detected by AddressSanitizer.

To reproduce, serve the PoC files over HTTP from the directory containing poc.html and poc_worker.js:

python3 -m http.server 8080

Create a temporary profile with the preferences required to trigger the vulnerability. The media.navigator.streams.fake preference provides a synthetic audio source so that a real microphone is not needed. The media.autoplay.default preference set to 0 allows the AudioContext to start without a user gesture. The javascript.options.mem.gc_allocation_threshold_mb preference lowers the GC threshold so that the allocation pressure in the worker triggers garbage collection sooner, making the dangling ArrayBuffer references observable more quickly.

TMPDIR=$(mktemp -d)
cat > "$TMPDIR/user.js" << 'PREFS'
user_pref("media.navigator.streams.fake", true);
user_pref("media.autoplay.default", 0);
user_pref("javascript.options.mem.gc_allocation_threshold_mb", 1);
PREFS

Then launch Firefox with quarantine disabled to force immediate memory reuse:

ASAN_OPTIONS="detect_leaks=0,quarantine_size_mb=0" \
  ./firefox --no-remote \
  --profile "$TMPDIR" \
  http://localhost:8080/poc.html

The crash typically appears within 10 to 15 seconds.

The following ASAN output was observed from the fuzzfetch build (full log in asan.log):

==2717227==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7c50a573e3c0 at pc 0x558d5bb9c35b bp 0x7b6068ffb490 sp 0x7b6068ffac50
READ of size 8 at 0x7c50a573e3c0 thread T22
    #0 __asan_memcpy
    #1 TypedArrayBitwiseSlice js/src/vm/TypedArrayObject.cpp:3930:7
    #2 TypedArray_slice js/src/vm/TypedArrayObject.cpp:4118:9
    #3 CallNonGenericMethod<&IsTypedArrayObject, &TypedArray_slice>
    #4 TypedArray_slice js/src/vm/TypedArrayObject.cpp:4146:10
    ...

0x7c50a573e3c0 is located 790 bytes after 170-byte region [0x7c50a573e000,0x7c50a573e0aa)

SUMMARY: AddressSanitizer: heap-buffer-overflow js/src/vm/TypedArrayObject.cpp:3930:7 in TypedArrayBitwiseSlice

The shadow byte at the access address is 0xfa (heap left redzone), confirming the ArrayBuffer's backing store was freed and the memory has been reused by a different allocation.

Fix

The root cause is the derived class destructors clearing mData before the base class destructor can use it for DetachArrayBuffer. The fix should move the DetachData() call into the derived destructors before clearing mData, matching the order used by the cycle collector UNLINK path.

--- a/dom/media/webrtc/jsapi/RTCEncodedAudioFrame.cpp
+++ b/dom/media/webrtc/jsapi/RTCEncodedAudioFrame.cpp
@@ -90,8 +90,9 @@ RTCEncodedAudioFrame::RTCEncodedAudioFrame(nsIGlobalObject* aGlobal,
 }

 RTCEncodedAudioFrame::~RTCEncodedAudioFrame() {
-  // Clear JS::Heap<> members before unregistering as a script holder,
-  // so their destructors don't barrier against a finalized JS object.
+  // Detach the ArrayBuffer before clearing mData, so the base class
+  // DetachData() can still see it. This matches the CC UNLINK order.
+  DetachData();
   mData = nullptr;  // from RTCEncodedFrameBase (protected)
   // Base class needs this, but can't do it itself because of an assertion in
   // the cycle-collector.
--- a/dom/media/webrtc/jsapi/RTCEncodedVideoFrame.cpp
+++ b/dom/media/webrtc/jsapi/RTCEncodedVideoFrame.cpp
@@ -115,8 +115,9 @@ void RTCEncodedVideoFrame::InitMetadata() {
 }

 RTCEncodedVideoFrame::~RTCEncodedVideoFrame() {
-  // Clear JS::Heap<> members before unregistering as a script holder,
-  // so their destructors don't barrier against a finalized JS object.
+  // Detach the ArrayBuffer before clearing mData, so the base class
+  // DetachData() can still see it. This matches the CC UNLINK order.
+  DetachData();
   mData = nullptr;  // from RTCEncodedFrameBase (protected)
   // Base class needs this, but can't do it itself because of an assertion in
   // the cycle-collector.

This ensures the ArrayBuffer is detached before both mData is cleared and the backing store is freed, regardless of whether destruction happens through the cycle collector UNLINK path or the C++ destructor chain.

Comment 1

[Nan Wang[:sakura]]

Attachment: poc.html

Comment 2

[Nan Wang[:sakura]]

Attachment: poc_worker.js

Comment 3

[Byron Campen [:bwc]]

The patch cleans up the mess that lead to this bug in the first place, and would make a very good cover bug. We can run it through its paces here, and make sure we're convinced we got it right this time.

Comment 4

[Byron Campen [:bwc]]

Attachment: (secure)

Comment 6

[Nico Grunbaum [:ng, @chew:mozilla.org]]

jib, do you think https://bugzilla.mozilla.org/show_bug.cgi?id=2023564 could be related?

Comment 7

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Bug 2023564 seems a bit opaque still. Any evidence of RTCRtpScriptTransform use there? I'm not good at gleaning that from a profile.

Comment 10

[Byron Campen [:bwc]]

Comment on attachment 9552696 [details]
(secure)

Security Approval Request

• How easily could an exploit be constructed based on the patch?: It would probably be pretty difficult to spot the problem this fixes, as it looks like cleanup.
• Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?: No
• Which branches (beta, release, and/or ESR) are affected by this flaw, and do the release status flags reflect this affected/unaffected state correctly?: all
• If not all supported branches, which bug introduced the flaw?: None
• Do you have backports for the affected branches?: No
• If not, how different, hard to create, and risky will they be?: They probably will not be difficult, but I would be surprised if this applied cleanly to ESR 140.
• How likely is this patch to cause regressions; how much testing does it need?: Probably pretty unlikely; the new code is much cleaner with fewer footguns, and has not been altered much in terms of functionality.
• Is the patch ready to land after security approval is given?: Yes
• Is Android affected?: Yes

Comment 11

[Daniel Veditz [:dveditz]]

Comment on attachment 9552696 [details]
(secure)

Mysteriously Missing Approval Flag strikes again; putting this in the queue as was obviously intended. We're not sure why it happens, but you're not the only victim

Comment 12

[Daniel Veditz [:dveditz]]

(In reply to Byron Campen [:bwc] from comment #10)

• Which branches (beta, release, and/or ESR) are affected by this flaw, and do the release status flags reflect this affected/unaffected state correctly?: all
• If not all supported branches, which bug introduced the flaw?: None

The above answers imply ESR-115 needs this fix; the reporter said bug 1631263 introduced this feature which would mean ESR-115 is unaffected. Which is right?

Comment 13

[Daniel Veditz [:dveditz]]

Comment on attachment 9552696 [details]
(secure)

sec-approval+ to land now and request uplifts

Comment 14

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1631263

Comment 15

[Byron Campen [:bwc]]

(In reply to Daniel Veditz [:dveditz] from comment #12)

(In reply to Byron Campen [:bwc] from comment #10)

• Which branches (beta, release, and/or ESR) are affected by this flaw, and do the release status flags reflect this affected/unaffected state correctly?: all
• If not all supported branches, which bug introduced the flaw?: None

The above answers imply ESR-115 needs this fix; the reporter said bug 1631263 introduced this feature which would mean ESR-115 is unaffected. Which is right?

groan

No, ESR 115 is not affected.

Comment 16

[Byron Campen [:bwc]]

Ok. I'll start making uplift patches.

Comment 17

[Byron Campen [:bwc]]

Patch applies cleanly to beta, builds ok, and seems to test ok.
It does not apply cleanly to esr140.

Comment 18

[Byron Campen [:bwc]]

Attachment: (secure)

Comment 19

[Byron Campen [:bwc]]

Ok, have a patch for esr140. Will start landing now.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/66beba63ca35
https://hg.mozilla.org/integration/autoland/rev/b0bd3dfe8bec
Simplify the CC setup for these classes. r=sfink,smaug,jib

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/d194b115d1da
https://hg.mozilla.org/integration/autoland/rev/9ed19c9bee7c
Revert "Bug 2021769: Simplify the CC setup for these classes. r=sfink,smaug,jib" for causing android build bustages @RTCEncodedFrameBase.

Comment 22

[Byron Campen [:bwc]]

That's pretty weird. Try was fine. Maybe we're conflicting with something, let me try rebasing.

Comment 23

[Byron Campen [:bwc]]

Ahh, this looks like a non-unified build thing that isn't run on try by default.

Comment 24

[Byron Campen [:bwc]]

Ok, got non-unified working, and am re-landing. esr140 backport is fixed also.

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/b69c7188e514
https://hg.mozilla.org/integration/autoland/rev/5bf29e285ada
Simplify the CC setup for these classes. r=sfink,smaug,jib

Comment 26

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/5bf29e285ada

Comment 27

[BugBot [:suhaib / :marco]]

The patch landed in nightly and beta is affected.
:bwc, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
  • See https://wiki.mozilla.org/Release_Management/Requesting_an_Uplift for documentation on how to request an uplift.
• If no, please set status-firefox150 to wontfix.

For more information, please visit BugBot documentation.

Comment 28

[Phabricator Automation]

firefox-beta Uplift Approval Request

• User impact if declined/Reason for urgency: Sec bug!
• Code covered by automated testing?: no
• Fix verified in Nightly?: yes
• Needs manual QE testing?: no
• Steps to reproduce for manual QE testing:
• Risk associated with taking this patch: low
• Explanation of risk level: I think this is somewhere between low and medium. Any time I mess with cycle-collection macros I'm a little uneasy, but this change simplifies things tremendously, and is nearly exactly what DOMException does so there's good precedent.
• String changes made/needed?: None.
• Is Android affected?: yes

Comment 29

[Byron Campen [:bwc]]

Attachment: (secure)

Original Revision: https://phabricator.services.mozilla.com/D287591

Comment 30

[Pulsebot]

https://github.com/mozilla-firefox/firefox/commit/c93d60566ba1
https://hg.mozilla.org/releases/mozilla-beta/rev/f8ce57af9940

Comment 31

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr140/rev/5153c3a0c532

Comment 32

[Byron Campen [:bwc]]

How do we feel about a release uplift on this?

Comment 33

[Ryan VanderMeulen [:RyanVM]]

Sorry, I was on PTO last week. This has missed the window for dot release inclusion now unfortunately.