1580900 - [macOS 10.15] Extend authorizationStatusForMediaType/requestAccessForMediaType platform support to screen-sharing

Status: RESOLVED FIXED

(Core :: Widget: Cocoa, enhancement)

Description

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

+++ This bug was initially created as a clone of Bug #1487204 +++

On Catalina, access to screen-sharing appears to require permission similar to what Mojave introduced for camera/mic. We need to extend what we did in bug 1487204 to cover screen sharing as well in order to not break getDisplayMedia.

Side-note: Even though the OS api exposed window metadata without permission, for privacy reasons and because our window/screen selector UX includes a preview area, we probably want to prompt for this OS access up front, like we do for camera.

Comment 1

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

For more info on the Catalina feature see https://developer.apple.com/videos/play/wwdc2019/701/

Comment 2

[Haik Aftandilian [:haik]]

With this bug, we want to provide platform support for checking whether or not Firefox has been granted access to record the screen. Front end code can then use this information to prompt or inform the user when Firefox doesn't have access so the user understands why some windows are not available for screen capture.

With camera and microphone, macOS provides an API (used in bug 1487204) for an application to check if it has been authorized to capture. The state of the authorization can be allowed, not allowed, or not yet determined (where the user hasn't been asked yet.)

For screen recording, this API is not available. There appear to be two options 1) start trying to record the screen and if the call returns nil that indicates permission has not been granted 2) use the fact that CGWindowListCopyWindowInfo() only returns the names of windows if screen recording permission has been granted. With option 1, if the permission has not already been granted/denied, macOS will prompt the user with its own dialog about screen recording.

I'm experimenting with CGWindowListCopyWindowInfo(), but so far I've noticed that even when screen recording permission has not been granted some window names are present in the returned list. Edit: I've also noticed that Firefox windows don't have names with or without screen recording permission. Firefox windows are named as expected, but the names are in the window list both with/without screen recording access.

Comment 3

[Haik Aftandilian [:haik]]

When screen recording has been authorized for the application, all the entries in the window list include the name. When access has not be authorized, a subset of the windows have the name. And it appears there are always some entries in the list for built-in applications such as the Desktop which only have names when access is allowed. So we can use this (hopefully stable property) until we have a better approach.

Comment 4

[Haik Aftandilian [:haik]]

Attachment: Bug 1580900 - Part 1 - Extend nsIOSPermissionRequest to support screen-recording permission checks r?spohl

Adds nsIOSPermissionRequest::GetScreenCapturePermissionState() to allow front-end code to check if Firefox has already been granted permission to record the screen.

Comment 5

[Haik Aftandilian [:haik]]

Attachment: Bug 1580900 - Part 2 - Extend nsIOSPermissionRequest to support screen-recording permission requests r?spohl

Adds nsIOSPermissionRequest::MaybeRequestScreenCapturePermission() to allow front-end code to trigger a screen-recording permission request dialog.

Depends on D46093

Comment 6

[Haik Aftandilian [:haik]]

The posted patches add an implementation for nsIOSPermissionRequest::getScreenCapturePermissionState() and nsIOSPermissionRequest::maybeRequestScreenCapturePermission(). I'm not entirely confident the implementation of maybeRequestScreenCapturePermission() makes it worthwhile exposing to JS. On Catalina, there isn't an API to request permission for screen recording and hence no support for asynchronous notifications once the user has made a choice. Given that the macOS dialog for screen recording gives the user a choice between "Deny" and go to the preferences, I expect users to skip allowing the permission, defer, or deny more frequently.

@jib, could you comment on whether you think maybeRequestScreenCapturePermission() is going to be useful given the above? Here's the comment from the patch on the functionality. As an alternative, JS code could just try to start recording.

  /*                                                                                   
   * Request permission to capture the screen using an unreliable method.              
   * Attemps to trigger a screen capture permission dialog. Whether or not             
   * the dialog is displayed and whether or not the user grants permission             
   * to record the screen is not available to the caller. This method has              
   * limited utility because it does not block to wait for a dialog                    
   * prompt or the user's reponse if a dialog is displayed. And the dialog             
   * is not guaranteed to be displayed per OS restrictions.                            
   */                                                                                  
  void maybeRequestScreenCapturePermission();

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b7055567f699
Part 1 - Extend nsIOSPermissionRequest to support screen-recording permission checks r=spohl
https://hg.mozilla.org/integration/autoland/rev/68dee5d0b648
Part 2 - Extend nsIOSPermissionRequest to support screen-recording permission requests r=spohl

Comment 8

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/b7055567f699
https://hg.mozilla.org/mozilla-central/rev/68dee5d0b648

Comment 9

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

(In reply to Haik Aftandilian [:haik] from comment #6)

@jib, could you comment on whether you think maybeRequestScreenCapturePermission() is going to be useful given the above?

Hi, sorry for the late reply. Was at TPAC last week. I see the landed comment is slightly different from comment 6:

// allowed or denied. Here we attempt to trigger the dialog by attempting to
// capture a 1x1 pixel section of the screen. The permission dialog is not

Was this resolved? (I haven't upgraded yet but) does Catalina add any user-observable indicators on successful screen capture?

If not, I'd hoped we could wait for success or failure of the attempt to capture 1x1 pixels, and learn that we have permission that way.

Comment 10

[Haik Aftandilian [:haik]]

Jib and I discussed this offline and Jib has a good plan to address bug 1580905 with the information available with the current API.

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #9)

Was this resolved? (I haven't upgraded yet but) does Catalina add any user-observable indicators on successful screen capture?

No, not that I'm aware of. When we capture the 1x1 area of the screen, that is synchronous. If we don't have screen recording access, the attempt appears to succeed but the result image is empty. The attempt will trigger the permission prompt if it is the first request for the application.

If not, I'd hoped we could wait for success or failure of the attempt to capture 1x1 pixels, and learn that we have permission that way.

Unfortunately we would have to poll by repeatedly calling nsIOSPermissionRequest::GetScreenCapturePermissionState() unless we can come up with a better way handle the async nature of the permission prompt.

Comment 11

[Haik Aftandilian [:haik]]

To exercise the API added in this bug, one can use the following commands in the browser console.

Register interface:

XPCOMUtils.defineLazyServiceGetter(this, "OSP", "@mozilla.org/ospermissionrequest;1", "nsIOSPermissionRequest");

Check screen sharing permission status:

screenStatus = {}; OSP.getScreenCapturePermissionState(screenStatus); screenStatus;

Attempt to trigger the prompt:

OSP.maybeRequestScreenCapturePermission();