2073405 - WebRTC VP9 encoder stops producing frames when scaleResolutionDownBy yields an odd width or height

Status: NEW

(Core :: Audio/Video, defect, P3)

Description

[David Négrier]

Attachment: A reproduction of the test starting a VP9 encoding of a canvas with an odd resolution after scaleResolutionDownBy

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:154.0) Gecko/20100101 Firefox/154.0

Steps to reproduce:

I'm developing WorkAdventure, a video conferencing platform. Since Firefox 154, the webcam from remote Firefox users is stuck at the first frame.

We tracked that down to an odd resolution after applying scaleResolutionDownBy on a VP9 codec.

Here is the analysis (helped by Claude): Since Firefox 154, the built-in libwebrtc VP9 encoder stops encoding for good as soon as a scaleResolutionDownBy value makes the scaled frame odd-sized in either dimension, for example 1280x720 / 5.74 = 222x125. setParameters() resolves, the connection stays up, but the sender's outbound-rtp stats show framesEncoded frozen and, a few seconds later, frameWidth/frameHeight at 0. The receiving peer keeps showing the last decoded frame.

Even scaled sizes (220x124, 426x240, 640x360) keep encoding. VP8 and AV1 keep encoding at 222x125.
Firefox 153.0.3 keeps encoding VP9 at 222x125. Chrome 153 encodes VP9 at 223x125 fine.

The encoder recovers as soon as setParameters() is called again with a scale that gives an even size.

Steps to reproduce

1. Open the attached vp9-odd-size.html (a loopback between two RTCPeerConnections in the same page, 1280x720 canvas source, VP9 forced with setCodecPreferences). No camera or permission needed.
2. Click "Start". The sender starts with scaleResolutionDownBy: 4 (320x180), then after 3 s switches to 5.74 (222x125). The page logs framesEncoded and frameWidth x frameHeight every second and prints a verdict after 6 s.
3. For comparison, enter 5.8 (220x124, both even) or select VP8 or AV1, and click "Start" again. ?auto=1&codec=VP9&scale=5.74 in the URL runs it without clicking.

Actual results:

setParameters scaleResolutionDownBy=4 -> expected 320x180
t=3s video/VP9 framesEncoded=47 size=320x180
setParameters scaleResolutionDownBy=5.74 -> expected 222x125
t=4s video/VP9 framesEncoded=47 size=320x180
...
t=8s video/VP9 framesEncoded=47 size=0x0
t=9s video/VP9 framesEncoded=47 size=0x0
STALLED: 0 frames encoded in 6 s after the scale change (expected ~100)

Expected results:

Firefox 153.0.3, same page:

setParameters scaleResolutionDownBy=4 -> expected 320x180
t=3s video/VP9 framesEncoded=51 size=320x180
setParameters scaleResolutionDownBy=5.74 -> expected 222x125
t=4s video/VP9 framesEncoded=66 size=222x125
t=5s video/VP9 framesEncoded=83 size=222x125
...
t=9s video/VP9 framesEncoded=148 size=222x125
OK: 97 frames encoded in 6 s after the scale change

Versions tested (official Linux x86_64 tarballs, fresh profile, headless)

Build	VP9 222x125	VP9 220x124	VP8 222x125	AV1 222x125
153.0.3	OK	OK	OK	OK
154.0.1	STALLED	OK	OK	OK
157.0b2	STALLED
158.0a1 (nightly 2026-09-18)	STALLED
nightly 2026-06-16 (154.0a1)	OK
nightly 2026-06-17 (154.0a1)	STALLED
Ubuntu snap 154.0.1, headed, real webcam	STALLED	OK	OK

Regression window (mozilla-central nightlies, same testcase):

• 2026-06-16 14:00 build (da4fdb0674c4): OK, 94 frames after the scale change
• 2026-06-17 10:10 build (41bab63bf46a): STALLED, 0 frames

Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=da4fdb0674c4744a9489b4a113eb61927d11216d&tochange=41bab63bf46ac59a75d77e8dd277c8278072417d

Notes

• media.webrtc.encoder_creation_strategy = 1 (prefer the platform encoder, i.e. the ffvpx VP9 encoder wrapped with the libwebrtc encoder as fallback) makes 154.0.1 encode 222x125 fine. The default value 0 (libwebrtc's built-in CreateVp9Encoder) is the failing path. So this looks like a change in libwebrtc's LibvpxVp9Encoder or in libvpx as called by it, rather than in the media pipeline around it.
• Two changes landed in that window: Bug 2047679 (libvpx update to 41e48324, 2026-06-16) and Bug 2047265 (libwebrtc vendored from b515dc61f5, 2026-06-17).

Comment 1

[Donal Meehan [:dmeehan]]

chunmin, could you take a look at this? Setting Bug 2047679 as the regressor for now
Claude:

2412a9c7f917 (Bug 2047679, Fx154). High confidence: an updatebot pair where upstream changed y_height from d_h to h and deliberately deleted the input-resolution check, then Mozilla's stale patch re-added it verbatim. img_alloc_helper's chroma rounding makes img->h = 126 ≠ cfg.g_h = 125, so it returns VPX_CODEC_INVALID_PARAM and the encoder never recovers. Also check VP8 — same patch treatment

Comment 2

[BugBot [:suhaib / :marco]]

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

Comment 3

[Jacek Stypuła]

I confirm the bug exists in Firefox 156.0.1 on Windows, Mac and Linux.