1525777 - WebRTC decoded video incorrectly scales when the stream resolution changes

Status: RESOLVED INCOMPLETE

(Core :: WebRTC: Audio/Video, defect)

Description

[Quinn Damerell]

Attachment: mixer_webrtc_repo.mp4

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

Hey! I'm a video engineer on Mixer.com. I noticed with the latest build of Firefox (65) our dynamic resolution logic for WebRTC is no longer working. I reverted to Firefox version 64 and saw the issue didn't reproduce.

Quick note: We use the term “Quality” to indicate different resolutions and bitrates for the video playback. So when I say to select a different quality, you asking our service switch the current RTP video stream to a different resolution and bitrate without disrupting the video flowing to WebRTC. (We switch on a keyframe, but use the same RTP sequence number and SSRC)

Here are the steps to repo:

1. Go to a channel on mixer.com with FTL (our low latency WebRTC based video playback)
   a. Example: https://mixer.com/ChannelOne

2. Verify that the channel has FTL
   a. Hover the video
   b. Click the gear icon
   c. Ensure the check box for “Low Latency” is checked

3. Observe the video being played. Take note of what is shown on the corners of the screen, since the video rendered size will change.

4. Change the video quality
   a. Hover the video
   b. Click the gear icon
   c. Click quality
   d. Choose a quality option that you aren’t on.

5. Wait for the stream to switch quality (2-5 seconds).

6. Notice that the video is being rendered at the wrong resolution. If the quality selected was lower than the original, the video will be a smaller rect in the top left of the video element with a green fill. If the quality chose is larger than the original stream quality, you will only see a subset of the video rendered in the video tag.

Here’s a quick video that shows the issue.
https://quinnd.me/i/vuAqSlhf2O-Mature-Xiphosuran.mp4

Actual results:

The new stream resolution doesn't fill the video element properly. The new video is either in the top left of the space padded by green fill or the video is rendered larger than the video container.

See video demo:
https://quinnd.me/i/vuAqSlhf2O-Mature-Xiphosuran.mp4

Expected results:

The video should remain rendering the correct size for the video element. When the resolution switches to the new resolution, the decoded video bitmap should be scaled correctly to match the size of the video element.

See video demo:
https://quinnd.me/i/vuAqSlhf2O-Mature-Xiphosuran.mp4

Comment 1

[Gingerbread Man]

(In reply to Quinn Damerell from comment #0)

I reverted to Firefox version 64 and saw the issue didn't reproduce.

It would be helpful if you could find the exact regression range.
https://mozilla.github.io/mozregression/quickstart.html

Comment 2

[Cristian Fogel [:cfogel]]

Hey Quinn,

We might need some extra info from you.

Failing to reproduce the issue.
Tried with on several devices: a PC with AMD+ATI Radeon 3000, a PC with NVidia GeForce GT 620, and a laptop using Intel HD Graphics 5500.
Best we can get as far as issues go, is a black and a white flash while loading after a full-screen toggle.

Is there anything else that might help or was overlooked?