1453269 - Remove screen capture resolution hack from MediaEngineRemoteVideoSource.cpp

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, enhancement, P3)

Description

[Andreas Pehrson [:pehrsons]]

In bug 1211656 we fixed constraints for screen sharing by hacking the desired resolution into a capability (essentially encoding the ideal and max constraint into one variable for each dimension) and handing it to the capture backend.

This hack later resulted in bug 1359662 and bug 1450954.

With bug 1388219 we support downscaling in the child process and would no longer need this hack. It's not optimal for perf to always capture at full resolution and pass such frames over IPC, but if that means we can leave the upstream capture backend unmodified it is still preferred IMO.

We'd need a way to figure out the captured screen's resolution ahead of selecting a capbility to be able to decide on the settings for a capture before resolving any capture requests.

Extra care needs to be taken for window captures where the size of the window may change dynamically while still fulfilling constraints.

[1] https://searchfox.org/mozilla-central/rev/6bfadf95b4a6aaa8bb3b2a166d6c3545983e179a/dom/media/webrtc/MediaEngineRemoteVideoSource.cpp#820-839
[2] https://searchfox.org/mozilla-central/rev/6bfadf95b4a6aaa8bb3b2a166d6c3545983e179a/dom/media/webrtc/MediaEngineRemoteVideoSource.cpp#490-501

Comment 2

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1453269 - Track constraints for desktop frame downscaling. r?jib

Comment 3

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1453269 - Update settings based on constraints, not just capability. r?jib

Comment 4

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1453269 - Update settings on reconfigure. r?jib

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/870e5a9de960
https://hg.mozilla.org/integration/autoland/rev/356810980517
Track constraints for desktop frame downscaling. r=jib
https://github.com/mozilla-firefox/firefox/commit/68d3c968259e
https://hg.mozilla.org/integration/autoland/rev/231ed181a011
Update settings based on constraints, not just capability. r=jib
https://github.com/mozilla-firefox/firefox/commit/23078210821f
https://hg.mozilla.org/integration/autoland/rev/239b6758940f
Update settings on reconfigure. r=jib

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/7c160ed750a0
https://hg.mozilla.org/integration/autoland/rev/cac7401ab455
Revert "Bug 1453269 - Update settings on reconfigure. r=jib" for causing WPT crashes @ mozilla::CalculateDesiredSize

Comment 7

[pstanciu]

Backed out for causing WPT crashes @ mozilla::CalculateDesiredSize

• https://hg.mozilla.org/integration/autoland/rev/cac7401ab45583f7c70d3c77145d1f58df491174
• Push with failures
• Failure Log

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/984b4d25da8a
https://hg.mozilla.org/integration/autoland/rev/a644d2fbf7eb
Track constraints for desktop frame downscaling. r=jib
https://github.com/mozilla-firefox/firefox/commit/2c45fab0ab9d
https://hg.mozilla.org/integration/autoland/rev/21f2a0fc8200
Update settings based on constraints, not just capability. r=jib
https://github.com/mozilla-firefox/firefox/commit/207d68677278
https://hg.mozilla.org/integration/autoland/rev/392139ad83bc
Update settings on reconfigure. r=jib

Comment 9

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/a644d2fbf7eb
https://hg.mozilla.org/mozilla-central/rev/21f2a0fc8200
https://hg.mozilla.org/mozilla-central/rev/392139ad83bc