2071030 - AV1 receive stalls when the sender uses spatial layers (L2T3_KEY) — frames assembled but never emitted from the jitter buffer

Status: VERIFIED FIXED

(Core :: WebRTC: Audio/Video, defect, P1)

Description

[jaya.allamsetty]

Steps to reproduce:

Firefox cannot decode an AV1 stream that carries spatial layers. Frames arrive and are assembled, but the jitter buffer almost never releases them, so the video freezes after a frame or two and never recovers. Firefox requests keyframes continuously and the sender does supply them, with no effect. The same sender using L1T3 (single spatial layer) decodes fine in the same call — scalability mode is the only variable.

Steps to reproduce

1. Join https://meet.jit.si/<room> in Chrome with camera on.
2. Join the same room from a second Chrome. (Still peer-to-peer — works, not the bug.)
3. Join from Firefox as the third participant. The call moves to the SFU and Chrome starts sending AV1 with spatial layers.
4. In Firefox, click a Chrome participant to put them on stage. (Thumbnails stay at 180p, use L1T3, and decode fine.)
5. Open about:webrtc and watch that inbound video stream.

Actual results:

The tile freezes almost immediately and stays frozen; Firefox shows the avatar. Over ~24s on that stream: framesReceived 5 → 448, but jitterBufferEmittedCount only 5 → 13, framesDecoded stuck at 5, keyFramesDecoded stuck at 1, pliCount 1 → 118. Decoded resolution stays at the 320×180 base layer of a 640×360 stream. No packet loss (framesDropped = 1).

Expected results:

The stream decodes, as it does from the same sender when L1T3 is used.

Comment 1

[jaya.allamsetty]

7:29.90 INFO: Narrowed integration regression window from [bbb405c9, 808fcc0e] (3 builds) to [bbb405c9, 23e1f148] (2 builds) (~1 steps left)
7:29.90 INFO: No more integration revisions, bisection finished.
7:29.90 INFO: Last good revision: bbb405c989696975295e78ddbc00a2f0dffca223
7:29.90 INFO: First bad revision: 23e1f148280c992651d7a6b5c9db9c4aa7ccb620
7:29.90 INFO: Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=bbb405c989696975295e78ddbc00a2f0dffca223&tochange=23e1f148280c992651d7a6b5c9db9c4aa7ccb620

7:33.11 INFO: ************* Switching to autoland by process of elimination (no branch detected in commit message)
7:34.97 ERROR: Unable to exploit the merge commit. Origin branch is mozilla-central, and the commit message for 23e1f148 was:
Bug 2058693 - updated libwebrtc patch stack

Comment 2

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Core::WebRTC: Audio/Video' component, but is not confident enough to move the bug to that component.

Comment 3

[jaya.allamsetty]

We have patched the meet.jit.si release such that Firefox doesn't offer to receive AV1 anymore. You can still test this on meet.jit.si by appending #config.testing.disableAV1DecodeForFF=false. For example, this is how a meeting link would look like

https://meet.jit.si/testFirefoxAv1#config.testing.disableAV1DecodeForFF=false

Comment 4

[Andreas Pehrson [:pehrsons]]

If you munge DD into Firefox does that fix the issue?

Comment 5

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 2058693

Comment 6

[Andreas Pehrson [:pehrsons]]

Quickest fix should be to signal DD for recv.

Comment 7

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 2071030 - Merge directions when an RTP header extension is registered again with a different direction. r?#webrtc-reviewers

Registering the same extension URI with different directions used to create
two extmap entries with different ids for the same URI. Merge the directions
into the existing entry instead, so that a URI always maps to a single id.

Comment 8

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 2071030 - Negotiate the dependency descriptor RTP header extension for receiving video. r?#webrtc-reviewers

libwebrtc 152 rejects temporal units whose spatial indices are not strictly
increasing. Without the dependency descriptor, AV1 frames are handled by the
sequence-number-only reference finder which stamps every frame with spatial
index 0, so layered AV1 streams (e.g. L2T3_KEY) never decode. Always offer the
dependency descriptor for receiving video, and keep sending it for simulcast.

Comment 10

[Andreas Pehrson [:pehrsons]]

Only AV1 should be affected, since the other spatial SVC codec, VP9, provides the spatial layer ID in a payload header.

I think the simplest workaround for sites here should be to munge recv support for DD into the local sdp. There might be some corner case this does not cover, since D325876 came to be.

Comment 11

[jaya.allamsetty]

Jitsi bridge always offers DD header extension which is set in remote SDP and then Firefox generates the following SDP in response. Are you suggesting we change the direction to recvonly here?

m=video 9 UDP/TLS/RTP/SAVPF 101 100 107 41 97 96 99 42
c=IN IP4 0.0.0.0
a=rtpmap:101 VP9/90000
a=rtpmap:97 rtx/90000
a=rtpmap:100 VP8/90000
a=rtpmap:96 rtx/90000
a=rtpmap:107 H264/90000
a=rtpmap:99 rtx/90000
a=rtpmap:41 AV1/90000
a=rtpmap:42 rtx/90000
a=fmtp:107 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=fmtp:101 max-fs=12288;max-fr=60
a=fmtp:97 apt=101
a=fmtp:100 max-fs=12288;max-fr=60
a=fmtp:96 apt=100
a=fmtp:99 apt=107
a=fmtp:42 apt=41
a=rtcp-fb:101 nack
a=rtcp-fb:101 nack pli
a=rtcp-fb:101 ccm fir
a=rtcp-fb:101 transport-cc
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 transport-cc
a=rtcp-fb:107 nack
a=rtcp-fb:107 nack pli
a=rtcp-fb:107 ccm fir
a=rtcp-fb:107 transport-cc
a=rtcp-fb:41 nack
a=rtcp-fb:41 nack pli
a=rtcp-fb:41 ccm fir
a=rtcp-fb:41 transport-cc
**a=extmap:11/sendonly https://aomediacodec.github.io/av1-rtp-spec/#dependency-descriptor-rtp-header-extension
**a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=setup:active
a=mid:4
a=recvonly
a=ice-ufrag:fe59605f
a=ice-pwd:4d788aa25a1ff95ad5f5cb0e34802511
a=ssrc:2789563071 cname:{b9113cdc-327e-4724-8ab7-9bbd1f589f5b}
a=rtcp-mux
a=extmap-allow-mixed

Comment 12

[Andreas Pehrson [:pehrsons]]

Munge sendrecv between generating the answer and setting the local description. If that doesn't work you're probably hitting the corner case I mentioned -- I haven't checked the constraints of that case thoroughly.

Comment 13

[Andreas Pehrson [:pehrsons]]

Forgot the ni?

Comment 14

[Andreas Pehrson [:pehrsons]]

I wrote a query to figure out how common it may be with AV1 for recv. A remote sender won't necessarily send the first codec in their SDP though even that is customary, so it may not be entirely accurate. But it's an indication based on signaling, which is the telemetry we have.

Comment 15

[jaya.allamsetty]

Munging the SDP to set sendrecv for the header extension fixed the issue.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/38b41b6b5139
https://hg.mozilla.org/integration/autoland/rev/bccf9519e829
Merge directions when an RTP header extension is registered again with a different direction. r=webrtc-reviewers,bwc
https://github.com/mozilla-firefox/firefox/commit/5458391c4313
https://hg.mozilla.org/integration/autoland/rev/f3e80405ec26
Negotiate the dependency descriptor RTP header extension for receiving video. r=webrtc-reviewers,bwc

Comment 17

[jaya.allamsetty]

@pehrsons, Thank you for the quick fix. Will this be pushed to 156 branch before it hits stable? We have AV1 enabled by default in our prod deployments. We would like to understand if we need to patch our existing releases to mitigate this.

Comment 18

[Andreas Pehrson [:pehrsons]]

156 releases tomorrow so that ship has sailed. Given the fix strikes much broader (all codecs) than the bug (AV1), I worry having it uplifted to the 156 dot release in a week may give it too little bake time in beta.

Impact wise, AV1 isn't that commonplace as preferred recv codec (~0.1% of VP8 per comment 14), making it a bit less urgent. For those inclined there is a workaround as confirmed in comment 15.

It should be in 157 which releases already in two weeks, on september 29.

If other services are also impacted we can revisit this. We'll keep an eye on incoming bug reports.

Comment 19

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 2071030 - Merge directions when an RTP header extension is registered again with a different direction.

Registering the same extension URI with different directions used to create
two extmap entries with different ids for the same URI. Merge the directions
into the existing entry instead, so that a URI always maps to a single id.

Original Revision: https://phabricator.services.mozilla.com/D325876

Comment 20

[Phabricator Automation]

firefox-beta Uplift Approval Request

• User impact if declined/Reason for urgency: Receiving AV1 spatial SVC (WebRTC) is broken, which affects some comferencing sites. See more details on the bug.
• Code covered by automated testing?: no
• Fix verified in Nightly?: no
• Needs manual QE testing?: yes
• Steps to reproduce for manual QE testing: See comment 0.
• Risk associated with taking this patch: low
• Explanation of risk level: I don't anticipate much risk with this change. It enables signaling of a (implemented upstream in libwebrtc) webrtc rtp header extension which we have already been using in some cases, and is used a lot more in chromium browsers. The bigger risk is around webcompat since we will start offering the extension for all receive codecs whereas the bug impacts only AV1 -- who knows what assumptions sites in the wild rely on.
• String changes made/needed?: None
• Is Android affected?: yes

Comment 21

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 2071030 - Negotiate the dependency descriptor RTP header extension for receiving video.

libwebrtc 152 rejects temporal units whose spatial indices are not strictly
increasing. Without the dependency descriptor, AV1 frames are handled by the
sequence-number-only reference finder which stamps every frame with spatial
index 0, so layered AV1 streams (e.g. L2T3_KEY) never decode. Always offer the
dependency descriptor for receiving video, and keep sending it for simulcast.

Original Revision: https://phabricator.services.mozilla.com/D325877

Comment 22

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/bccf9519e829
https://hg.mozilla.org/mozilla-central/rev/f3e80405ec26

Comment 23

[jaya.allamsetty]

@pehrsons Can you confirm the target release for this fix? You had mentioned 157 but the target milestone on the bug has been updated to 158. We need the correct version for the hot patches that we will need to apply to our releases.

We are considering bringing back sub-optimal browser experience banner for Firefox since it is not easy to get all the Jitsi deployments updated to the latest release that fast.

Comment 24

[Andreas Pehrson [:pehrsons]]

157 assuming relman approves the uplift request for beta 2, due on sep 16. See https://whattrainisitnow.com/release/?version=157 for full schedule.

Comment 25

[Pulsebot]

https://github.com/mozilla-firefox/firefox/commit/1ee3e966a86a
https://hg.mozilla.org/releases/mozilla-beta/rev/75f605f4a899
https://github.com/mozilla-firefox/firefox/commit/4bad8a9259b3
https://hg.mozilla.org/releases/mozilla-beta/rev/ea9d050f439d

Comment 26

[Daniel Bodea [:danibodea]]

I am attempting to verify this fix, but I can't seem to reproduce it using the steps in comment in supposably affected Nightly v158.0a1 from 2026-09-13. The Chrome video streams would never freeze even when being "put on stage".

Could you provide some information?

1. Regarding step 5, where ca I find the "inbound video stream" in about:webrtc? What am I looking for?
2. Is there anything missing from the reproduction process?
3. Do I need to use argument #config.testing.disableAV1DecodeForFF=false to reproduce?
4. I would assume it's necessary to use high quality webcams. Is that right? Would a 1080p webcam suffice?
5. Any other dependecies come to mind?
   Thanks!

Comment 27

[jaya.allamsetty]

Yes, the issue reproduces only when #config.testing.disableAV1DecodeForFF=false is used.

Comment 28

[jaya.allamsetty]

I was able to verify the fix on 157.0b3

Comment 29

[Andreas Pehrson [:pehrsons]]

(In reply to Daniel Bodea [:danibodea] from comment #26)

I am attempting to verify this fix, but I can't seem to reproduce it using the steps in comment in supposably affected Nightly v158.0a1 from 2026-09-13. The Chrome video streams would never freeze even when being "put on stage".

Could you provide some information?

1. Regarding step 5, where ca I find the "inbound video stream" in about:webrtc? What am I looking for?

Here's an updated step 5:

• Open about:webrtc in a new tab.
• Expand the RTCPeerConnection Statistics section.
• Click Show Details on the jitsi peer connection.
• Check the Auto refresh checkbox.
• For those parts that start with video/ and have moving graphs for inbound-rtp's packetsReceived/s stats, check that they are for AV1 (should say video/AV1) and that the framesPerSecond stat is moving and non-zero.
• To verify that a higher layer than the base layer is used, you can check the Video Frame Statistics table towards the bottom:
  • Match SSRC from the video/AV1 line to the Local Receiving SSRC column.
  • The bottom-most row for each SSRC is the currently received resolution. The width should be > 320 and height > 180.

2. Is there anything missing from the reproduction process?
3. Do I need to use argument #config.testing.disableAV1DecodeForFF=false to reproduce?

Right, jitsi worked around this on their end so you need to add this as a search param to the URL.

4. I would assume it's necessary to use high quality webcams. Is that right? Would a 1080p webcam suffice?

I don't think that's required. This is about codec, not resolution; and recv video layers working vs not working at all.

5. Any other dependecies come to mind?

Follow the instructions on what browser to use when, because I had some trouble reproducing with Firefox being the remote peer.

Thanks!

Comment 30

[Daniel Bodea [:danibodea]]

I managed to reproduce the issue in Nightly v156.0a1 (2026-08-15) on Windows 10. Every time a Chrome participant is put on stage, the video stream freeses in Firefox and the framesPerSecond from "inboundRtpStreamStats" is 0.

The same issue no longer reproduces in Nightly v158.0a1 (2026-09-22) and Beta v157.0b4. The video stream of the Chrome participant being put on stage no longer freezes, the SSRC from the video/AV1 line match to the Local Receiving SSRC column and the bottom-most row for each SSRC width is > 320 and height > 180.

I have to mention that the argument (#config.testing.disableAV1DecodeForFF=false) necessary for testing does not remain displayed in the address bar AFTER Clicking the "Join meeting" button to join the call. (For clarity, The FX system joins the 2 Chrome partcipant call by loading https://meet.jit.si/LocatedEarthquakesInterruptOpenly#config.testing.disableAV1DecodeForFF=false, then clicks on "Join meeting", but the argument mentioned above is no longer displayed when in the call.)
What does this mean regarging the verification of the fix? Is is valid?

Comment 31

[Andreas Pehrson [:pehrsons]]

This is fine, thanks!

For completion, you can see in the JS console that the disableAV1DecodeForFF=false option took effect.

Comment 32

[Daniel Bodea [:danibodea]]

I can find this in the Web Console:
"2026-09-23T08:11:40.638Z [INFO] [app:config] <Sx/</<>: Extending config with: {"testing":{"disableAV1DecodeForFF":false}}"

This being considered, this report is closed and verified. Thank you.

Comment 33

[Andreas Pehrson [:pehrsons]]

Upstream seems to have turned around after another service was broken in Chromium and Edge too.