1918096 - Hook up an SCContentSharingPicker MVP integration

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, task)

Description

[Andreas Pehrson [:pehrsons]]

We already have Pipewire desktop capture using a system dialog. Let's hook up macOS >= 15 similarly, behind a pref, for now. This let's us integrate SCContentSharingPicker without much other patches (e.g. frontend), and provides an avenue for potential uplift into 131.

Comment 1

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - Integrate SCContentSharingPicker into ScreenCapturerSck. r?#webrtc-reviewers

Comment 2

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - Generate libwebrtc moz.build files. r?#webrtc-reviewers

Comment 3

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - Wire up SCContentSharingPicker using Pipewire desktop capture UX. r?#webrtc-reviewers!, r?grulja

Comment 4

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - Reject gUM/gDM request if video track ends before first frame. r?#webrtc-reviewers

This case gets hit if the SCContentSharingPicker is cancelled by the user.

Comment 5

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - In ScreenCapturerSck improve dictionary ergonomics. r?#webrtc-reviewers

Comment 6

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - In ScreenCapturerSck adapt to sources that change resolution. r?#webrtc-reviewers

If the resolution is smaller than the allocated surface, we crop.
If the resolution is larger than the allocated surface, we reconfigure with a
larger surface.

The allocated surface has the size we told it to have in the
SCStreamConfiguration, in pixels.

If the source is a screen, the size is pretty static. Changing display settings
could affect it.
If the source is a window, the size is initially the size of the window. The
user resizing the window affects the size.
If the source is multiple windows, the size initially seems to be that of the
display they're sitting on. Windows across multiple displays are not captured
into the same surface, as of macOS 14.

Comment 7

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - In ScreenCapturerSck skip some processing when possible. r?#webrtc-reviewers

No need to update the last frame unless it has changed.

Comment 8

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - Enable SCContentSharingPicker by default (limited to macOS 15). r?#webrtc-reviewers

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/86a5d2db126e
Integrate SCContentSharingPicker into ScreenCapturerSck. r=webrtc-reviewers,padenot,dbaker
https://hg.mozilla.org/integration/autoland/rev/c8c8702310d4
Generate libwebrtc moz.build files. r=webrtc-reviewers,ng,dbaker

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/eb9b355882a9
Wire up SCContentSharingPicker using Pipewire desktop capture UX. r=grulja,webrtc-reviewers,dbaker
https://hg.mozilla.org/integration/autoland/rev/97e405de0635
Reject gUM/gDM request if video track ends before first frame. r=webrtc-reviewers,dbaker

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/aabf0bd94f28
In ScreenCapturerSck improve dictionary ergonomics. r=webrtc-reviewers,ng
https://hg.mozilla.org/integration/autoland/rev/bc6215142fea
In ScreenCapturerSck adapt to sources that change resolution. r=webrtc-reviewers,ng
https://hg.mozilla.org/integration/autoland/rev/8d52f2fd5754
In ScreenCapturerSck skip some processing when possible. r=webrtc-reviewers,ng

Comment 12

[pstanciu]

https://hg.mozilla.org/mozilla-central/rev/86a5d2db126e
https://hg.mozilla.org/mozilla-central/rev/c8c8702310d4
https://hg.mozilla.org/mozilla-central/rev/eb9b355882a9
https://hg.mozilla.org/mozilla-central/rev/97e405de0635

Comment 13

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1918096 - Make SckPickerHandle thread safe. r?#webrtc-reviewers, r?padenot

We cannot guarantee that the thread it is used on is static, as both the
VideoCapture and DesktopCapture threads can come and go.

Comment 14

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/aabf0bd94f28
https://hg.mozilla.org/mozilla-central/rev/bc6215142fea
https://hg.mozilla.org/mozilla-central/rev/8d52f2fd5754

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cfcdd5339805
Make SckPickerHandle thread safe. r=padenot

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/07dd5fcced62
Enable SCContentSharingPicker by default (limited to macOS 15). r=webrtc-reviewers,dbaker

Comment 17

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/cfcdd5339805
https://hg.mozilla.org/mozilla-central/rev/07dd5fcced62

Comment 18

[Jim Mathies [:jimm]]

[Tracking Requested - why for this release]:
We would like to uplift this work (plus the fixes in the two regressions) once it's had more bake time.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

FWIW, if this is going to go to ESR128, it'll need a rebased patch stack to do so. I'm also not super thrilled about taking this large of a set of changes there, so more explanation as to why it's necessary would be appreciated.

Comment 20

[Andreas Pehrson [:pehrsons]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #19)

FWIW, if this is going to go to ESR128, it'll need a rebased patch stack to do so. I'm also not super thrilled about taking this large of a set of changes there, so more explanation as to why it's necessary would be appreciated.

This is mainly to avoid the new system alerts in macOS 15 that are mentioned in their release notes (see ScreenCaptureKit Deprecations). There's some code in here affecting the existing desktop capture framework, but note most of it is behind a pref and a macOS 15 runtime check. We also would like this to have more time to bake in release before uplifting.