Closed Bug 1730748 Opened 3 years ago Closed 2 years ago

WebRTC downgrades screen sharing resolution

Categories

(Core :: WebRTC, enhancement)

Firefox 91
enhancement

Tracking

()

RESOLVED DUPLICATE of bug 1654112

People

(Reporter: btham, Unassigned)

References

Details

Attachments

(3 files)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

  1. Join a Webex meeting with 2 Firefox clients.
  2. Start screen sharing from 1 Firefox client to the other.
  3. Put the Firefox clients in the background and do something else.

Actual results:

After some time, the screen sharing resolution downgrades from 1080p to 1440x813.

Expected results:

After some time, the screen sharing resolution remains at 1080p.

I'm a developer for the Webex browser client. For screen sharing, I would imagine that keeping it at 1080p is important, especially when sharing documents with lots of text. Chrome does not have this issue.

The Bugbug bot thinks this bug should belong to the 'Core::WebRTC' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: Untriaged → WebRTC
Product: Firefox → Core

I have created a simple code example that can reproduce this issue. After starting the call, the resolution is 1080p. After some time, I get the following console log: "Remote video size changed to 1440x810".

It seems to trigger most often when CPU usage is high.

btham, thanks! Do you see the frame rate drop before the resolution drops? I don't suppose you have access to the estimated bandwidth reported by the sender over the course of the call.

Flags: needinfo?(btham)

In my initial tests, I put Firefox in the background so I did not see whether or not the frame rate drops. However, if I manually increase my CPU usage (using a tool such as CPUSTRES), I do see some framerate drops but that might simply be because I increased my CPU usage.

I've attached some outbound-rtp statistics from getStats if it's useful.

I will say that I understand that such downgrade in resolution might be useful for webcam video to make sure the user's video is smooth, but I feel such a downgrade is inappropriate for screen sharing where framerate isn't as important. 1080p to 810p can be the difference between being able to read a text on a webpage and not being able to read it at all.

Flags: needinfo?(btham)

The severity field is not set for this bug.
:mjf, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(mfroman)

Hi Btham, are you able to reproduce using https://jsfiddle.net/jib1/a3q0918v/ ? Can you compare it to Chrome?

I'm able to reproduce with that fiddle eventually, by cranking CPUSTRES >80% CPU, but I'm not sure that's a bug. In that case I see:

Resized to 1440 x 810

...when I visit a different tab and come back (or sometimes after just leaving the tab visible as well). But after a few seconds it seems to recover:

Resized to 1920 x 1080

I suspect in such a CPU-starved condition, Firefox is having trouble sending and receiving at full rate, and tries to compensate by lowering the resolution it is sending. I'd consider this normal and something all browsers will do given a sufficiently constrained environment. Exactly how these decisions are made is browser-dependent, and not standardized.

Whether there's a bug here or not is subjective, since there are a lot of factors, like the speed of your system. It sounds like this is creating a sub-par experience in WebEx in Firefox, and you say Chrome is handling it better, so I think we'd like to improve this if we can. I'd be curious if Chrome still performs better if you replace screen share with camera.

Are you asking Firefox to treat screen capture differently from camera?

It sounds like you're expecting Firefox to make different decisions for screen-sharing vs. camera, which I think is a reasonable request. Firefox might be able to detect this automatically since it knows what source is used. — But I'm not sure if that would be 100% spec compliant. Do you happen to know if Chrome does this? According to the spec, we should probably implement bug 1734729 and rely on the app to tell us what to do (IOW I'm not sure whether "balanced" allows the browser to try to be smart about it or not — e.g. not all screen captures are google slides, they could be youtube or a game).

Depends on: 1734729
Flags: needinfo?(btham)

Yup, I see the same issue with the jsFiddle. Firefox will downgrade the resolution when CPU is high, but Chrome will not.

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #7)

Are you asking Firefox to treat screen capture differently from camera?

So my main ask is just to have the option to disallow automatic resolution downgrading while screen sharing. Whether that is accomplished via auto-detecting the source, implementing RTCDegredationPreference, or some other means is up to y'all :)

Flags: needinfo?(btham)
Severity: -- → S3
Status: UNCONFIRMED → NEW
Type: defect → enhancement
Ever confirmed: true
Flags: needinfo?(mfroman)

We already give screenshares special treatment. Or well, we tell the libwebrtc encoder that it is a screenshare track and let it handle the downgrading. I suppose libwebrtc has improved this handling since the version we currently employ, since chromium functions better.

Depends on: 1654112

I've been unable to reproduce this on either 95 or 96 (where bug 1654112 is present). Could you check whether you can still reproduce this on the latest Nightly (96)?

I see you're on windows. Note you may hit bug 1738787 (a deadlock related to desktop capture). If that hinders you, please check back in later.

Flags: needinfo?(btham)

Thanks. I tried with FF Nightly 96.0a1. I cannot reproduce the issue there :)

The sharing content itself looks a little messed up (I've attached a screenshot) but I assume that will be fixed.

Flags: needinfo?(btham)

Great, thanks! The distorted image is bug 1738628.

Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → DUPLICATE

Thanks Andreas! Which release version do we expect this bug (1730748) to be fixed? Is it 95?

Bug 1654112 landed in 96.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: