1855636 - For certain recvonly streams, most/all P frames are failing to decode (H264)

Status: VERIFIED FIXED

(Core :: WebRTC, defect, P1)

Description

[andrew]

Steps to reproduce:

This is a regression (good: 7e011293b43, bad: 70417eb0e3b).
The first public Firefox release with this regression is 118.

To repro, using a bad release of Firefox, do the following:

1. Go to the URL <redacted>
2. Click "View in 3D" button. Wait a few seconds for the page to load.

Actual results:

After the repro steps, the following unexpected results occur with a bad release of firefox (e.g. 118 or 70417eb0e3b):

1. You will probably see only a black page. (expected: A video stream of a simple 3D object (teapot) should be shown).
2. LMB dragging in the window will have no effect. (expected: Should cause the video stream to show the 3D object being moved with respect to your mouse movement)
3. Pressing Alt+L (just once) will also have no effect, or will show a frame or two and then freeze (expected: Should cause the video stream to show the 3D object spinning in place)
4. After pressing Alt+L (which ensures that frames are continuously sent from the server), you should notice that about:webrtc shows an increasing number of dropped frames, and very few decoded frames - probably none at all.

Expected results:

With a good release of Firefox (e.g. 117 or 7e011293b43),

1. A video stream of a simple 3D object (teapot) will be shown.
2. LMB dragging will cause the video stream to show the 3D object being moved with respect to your mouse movement.
3. Pressing Alt+L (just once) will cause the video stream to show the 3D object spinning in place.
4. After pressing Alt+L (which ensures that frames are continuously sent from the server), you should notice that about:webrtc shows an increasing number of decoded frames, and very few dropped frames.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::WebRTC' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1846796

Comment 3

[Paul Adenot (:padenot)]

Attachment: Bug 1855636 - Disable part of 1846796 to unbreak some content on release and beta. r?#media-playback-reviewers

Comment 4

[Paul Adenot (:padenot)]

https://pernos.co/debug/mcHmpQLC8KY97z1STlvytA/index.html#f{m[B+TV,DUw_,t[Ajs,Giuq_,f{e[B+TV,DSs_,s{af2nmYAAA,bAZI,uFNYACA,oFNcAXw___/ is a pernosco trace of this -- the offending code is commented out like the patch for release, but it allows inspecting the AVCC struct that's likely invalid.

Comment 5

[Paul Adenot (:padenot)]

Comment on attachment 9355594 [details]
Bug 1855636 - Disable part of 1846796 to unbreak some content on release and beta. r?#media-playback-reviewers

Beta/Release Uplift Approval Request

• User impact if declined: Some inbound h264 WebRTC video doesn't work anymore. This breaks content.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: repro URL is not public, ping @padenot ([e-mail]) for repro steps.
• List of other uplifts needed: none
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is a low-risk beta / release-only patch while we figure out the real-fix for Nightly.
• String changes made/needed: none
• Is Android affected?: Yes

Comment 6

[Dianna Smith [:diannaS]]

Comment on attachment 9355594 [details]
Bug 1855636 - Disable part of 1846796 to unbreak some content on release and beta. r?#media-playback-reviewers

Approved for 119.0b5

Comment 7

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/118184ccfcd8

Comment 12

[Vlad Lucaci, Thunderbird QA (:vlucaci)]

Hello,

Managed to reproduce this issue using the STR from the description with an affected build from 2023-09-28 , 120.0a1(20230928215127) with macOS 14.1.

Confirming this issue as verified fixed on 119.0b5(20231004091611) with macOS 14.1, Windows 11 and Ubuntu 22.

Comment 13

[Pascal Chevrel:pascalc]

Comment on attachment 9355594 [details]
Bug 1855636 - Disable part of 1846796 to unbreak some content on release and beta. r?#media-playback-reviewers

Approved for our next 118 dot release, thanks.

Comment 14

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-release/rev/2b702aed691e

Comment 15

[Pascal Chevrel:pascalc]

https://hg.mozilla.org/releases/mozilla-release/rev/2b702aed691e

Comment 16

[Vlad Lucaci, Thunderbird QA (:vlucaci)]

Hello,

Confirming this issue as verified fixed on 118.0.2(20231009140911) using macOS 13, Windows 11 and Ubuntu 22.

Comment 17

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:mjf, could you have a look please?

For more information, please visit BugBot documentation.

Comment 18

[Paul Adenot (:padenot)]

Clearing NI for Michael -- I handled this.

Comment 19

[Paul Adenot (:padenot)]

Can you land this on beta? We haven't done the "real" fix yet and we don't want to break this again.

Comment 20

[C.M.Chang[:chunmin]]

Comment on attachment 9355594 [details]
Bug 1855636 - Disable part of 1846796 to unbreak some content on release and beta. r?#media-playback-reviewers

Hi dsmith, we need more time to fix this properly so D189507 has to be landed on beta again to avoid the problem

Beta/Release Uplift Approval Request

• User impact if declined: Some inbound h264 WebRTC video doesn't work anymore. This breaks content.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: repro URL is not public, ping @padenot ([e-mail]) for repro steps.
• List of other uplifts needed: none
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is a low-risk beta / release-only patch while we figure out the real-fix for Nightly.
• String changes made/needed: none
• Is Android affected?: Yes

Comment 21

[Dianna Smith [:diannaS]]

Approved for 120.0b3
... since this never made it to central during the 120 nightly cycle.
121 is still affected but we should prob disable it there too unless a fix is almost finished.

Comment 22

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/c9e3fb5ed0f4

Comment 23

[Ryan VanderMeulen [:RyanVM]]

Agreed - we should land this on m-c and move fixing and re-enabling to a new bug rather than having to remember to do this every cycle.

Comment 24

[Paul Adenot (:padenot)]

Chun-Min, can you handle this?

Comment 25

[C.M.Chang[:chunmin]]

I'll revert the changes made in 1846796 first and work on a new fix.

Comment 26

[C.M.Chang[:chunmin]]

Attachment: Bug 1855636 - Revert "Bug 1846796 - Parse extra data from converted avcc data"

This reverts commit 39e19ad141e661b878acaf3ed8c214dfd3fe2dae.

Comment 27

[C.M.Chang[:chunmin]]

Attachment: Bug 1855636 - Return unsupported for H264 in WebCodecs on MacOS

Depends on D192436

Comment 28

[C.M.Chang[:chunmin]]

Attachment: Bug 1855636 - Change WPT expectations

Depends on D192437

Comment 29

[C.M.Chang[:chunmin]]

I've uploaded patches that revert the patches in bug 1846796

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8455d0b70dcd
Revert "Bug 1846796 - Parse extra data from converted avcc data" r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/e46ff3037f51
Return unsupported for H264 in WebCodecs on MacOS r=jolin
https://hg.mozilla.org/integration/autoland/rev/93b0e65b5ad2
Change WPT expectations r=jolin

Comment 31

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/8455d0b70dcd
https://hg.mozilla.org/mozilla-central/rev/e46ff3037f51
https://hg.mozilla.org/mozilla-central/rev/93b0e65b5ad2

Comment 32

[Vlad Lucaci, Thunderbird QA (:vlucaci)]

Confirming this issue as verified fixed on 121.0a1(20231107214948) and 120.0b7(20231106091614) using macOS 14, Windows 11 and Ubuntu 22.