1580905 - [macOS 10.15] screen sharing (getDisplayMedia) offers nothing to share except Firefox window

Status: RESOLVED DUPLICATE of bug 1588640

(Core :: WebRTC: Audio/Video, defect, P2)

Description

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

+++ This bug was initially created as a clone of Bug #1479051 +++

Catalina appears to add OS-level dialogs letting the user allow/deny access to the screen sharing, much like Mojave did for camera/microphone. See bug 1580900.

This interferes with getDisplayMedia functionality for all sources except the Firefox window itself.

STRs:

1. Open https://jan-ivar.github.io/dummy/gdm.html

Expected result:

• (On Catalina) An OS prompt asking the user to allow Firefox to use screen sharing
• (After clicking "Allow") Screen sharing selector appears, and clicking on "Select Window or Screen" shows a list of all application windows and "Entire screen".

Actual result:

• (No OS prompt on Catalina) Screen sharing selector appears, and clicking on "Select Window or Screen" shows only the Firefox Nightly window.

Note: for privacy reasons web surfaces are extra "scary" to share, so having them alone on the list is not great.

For this reason, we probably want to prompt the OS for permission before showing our selector, to ensure the list is populated with safer choices first. And if the user rejects the permission getDisplayMedia() should probably fail (i.e. not fall back to just showing lone Firefox window choice).

User flow is therefore hopefully similar to that of camera.

Comment 1

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

From bug 1580900 comment 6:

Given that the macOS dialog for screen recording gives the user a choice between "Deny" and go to the preferences, I expect users to skip allowing the permission, defer, or deny more frequently.

Discussed this with Haik, and without an allow/deny choice to wait on, we think all we can do here is:

• If we don't have permission, trigger OS's deny/settings prompt, and fail immediately with NotAllowedError.

It would be up to app (the user effectively, bug 1580944) to try again later.

For the privacy reasons mentioned in comment 0, we think it makes sense to put sharing of the Firefox window itself behind this OS permission as well, even though macOS does not demand it (because, ironically, in the special case of a browser window, self-sharing is the least secure option, not the most secure option).

This gives us a consistent API that fails with NotAllowedError when lacking OS permission.

Comment 2

[Bram Pitoyo [:bram]]

Jan-Ivar and Haik, I’ve updated the user flow diagram to reflect our discussion earlier today.

• If user rejects OS-level permission, getDisplayMedia() should fail. Firefox does nothing.
  • Ideally, we’d like to show something useful instead of nothing, but this waits for bug 1494172
• If user accepts OS-level permission, System Preferences will open. We wait until the user ticks the “Firefox” box there.
  • After OS-level permission has been granted, the user must also manually return to Firefox
  • Then we can show them our site-level permission dialogue, followed by the screen/window selector

Comment 3

[Haik Aftandilian [:haik]]

For some more missing context here (see also bug 1580900 comment 2), screen recording in Catalina does not use the same API that was added in Mojave for camera and microphone recording. We have a method to detect if the application has already been given permission by the user to record the screen. But we don't have OS support for triggering the permission dialog and getting a callback when the request is either allowed or denied. The prompt for screen recording lets the user "Open System Preferences" or "Deny" the request.

With camera/mic recording, macOS Mojave provides an API to trigger the (Allow/Deny) request dialog and receive a callback when the user approves or denies the prompt. If the permission has already been denied, the API returns an result indicating that.

With screen recording, with what we know so far about Catalina, we can perform an operation that triggers the permission dialog if it has not already been displayed for Firefox, but we don't know if the dialog is shown and we don't receive a callback when the user denies the request or goes into System Preferences and allows Firefox to record the screen.

Comment 4

[Bram Pitoyo [:bram]]

(In reply to Haik Aftandilian [:haik] from comment #3)

With screen recording […] we can perform an operation that triggers the permission dialog if it has not already been displayed for Firefox, but […] we don't receive a callback when the user denies the request or goes into System Preferences and allows Firefox to record the screen.

In that case, how would Firefox know whether screen recording permission has been allowed/denied, and how can we take next steps: either a) show window/tab to share, or b) show modal that leads to System Preferences?

When Firefox is uncertain, I wonder whether it’s possible to re-trigger the OS permission dialogue before showing our own UI??

1. Firefox triggers OS permission dialogue
2. User responds with Allow or Deny
   • Either way, Firefox doesn’t know their answer
   • Firefox minifies the UI into a URL bar icon
3. When the icon is clicked, Firefox triggers OS permission dialogue again
   • If OS shows nothing, then Firefox knows that permission status is Allowed
     • Then, Firefox shows list of window/tab to share
   • If OS shows something, then Firefox knows that permission status is Denied
     • When the URL bar icon is clicked, Firefox shows modal that leads to System Preferences
     • But we expect most users to act on the OS permission dialogue and Allow the permission

Comment 5

[Haik Aftandilian [:haik]]

(In reply to Bram Pitoyo [:bram] from comment #4)

(In reply to Haik Aftandilian [:haik] from comment #3)

With screen recording […] we can perform an operation that triggers the permission dialog if it has not already been displayed for Firefox, but […] we don't receive a callback when the user denies the request or goes into System Preferences and allows Firefox to record the screen.

In that case, how would Firefox know whether screen recording permission has been allowed/denied, and how can we take next steps: either a) show window/tab to share, or b) show modal that leads to System Preferences?

We could display something indicating the user needs to reload the page after granting access or we could repeatedly recheck and if we detect the permission has been granted move forward with window/tab selection. But it's not feasible to check with high frequency for a long period of time. Or we could add a button to trigger a re-check.

When Firefox is uncertain, I wonder whether it’s possible to re-trigger the OS permission dialogue before showing our own UI??

Not reliably. If the dialog was already displayed once, it won't be displayed again. I added a few notes to the user flow to clarify what we could do:

Site attempts to record the screen.
Firefox checks if it has OS access to record the screen
If it has access, display window/tab selection. Done.
If it does not have access, go to 1 below.

1. Firefox triggers OS permission dialogue

Firefox could display something here indicating we're waiting for the user to enable access in the system preferences and include a link to the preferences. We could also include a button such as "Try Again" where we check if we've been given access and either dismiss the notice or re-display it depending on the outcome.

2. User responds with Allow or Deny
   • Either way, Firefox doesn’t know their answer
   • Firefox minifies the UI into a URL bar icon
3. When the icon is clicked, Firefox triggers OS permission dialogue again
   • If OS shows nothing, then Firefox knows that permission status is Allowed
     • Then, Firefox shows list of window/tab to share
   • If OS shows something, then Firefox knows that permission status is Denied
     • When the URL bar icon is clicked, Firefox shows modal that leads to System Preferences
     • But we expect most users to act on the OS permission dialogue and Allow the permission

That sounds good to me.

Comment 6

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Firefox minifies the UI into a URL bar icon

That sounds too much like how Firefox permissions used to work years ago, which was generally panned as problematic (bug 1004055).

Web sites also yell at us if we hold on to the promise forever, as it prevents them from taking action. Better to return NotAllowedError immediately I think (after prodding the OS to show their prompt). Then the web site can follow up with their user, giving them instructions on what they need to do if they want to screen share, and include special instructions for Catalina.

I suggest we not do anything more than comment 1 here.

Comment 7

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Having upgraded to Catalina and trying this a bit, I'm backing off comment 1 about doing OS prompt upfront.

I'd forgotten we ask for OS permission after our camera/mic prompts in Mojave, presumably to not throw OS prompts at the user without context, and instead only show them when they're primed to approve, so Firefox isn't accidentally blocked by the OS forever.

Likewise, it might be a mistake to show this OS prompt without context.

Given this restriction, I think two things:

1. We'd probably need UX work (bug 1494172) to improve this any over what we have today.
2. What we have today isn't too bad, except for bug 1588658 and bug 1588640.

Comparing what we have today with Chrome:

• In Chrome, users see both Chrome's screen sharing prompt and the OS prompt at the same time.
• In Firefox, users see Firefox's prompt with only two choices
  • If they pick "Firefox" then that works without OS permission
  • If they pick "Entire screen" then the OS prompt should appear. If the user clicks "Go to settings" in that and checks ☑ Firefox before they hit "Allow" in Firefox, that works too (no need to restart even AFAICT). When the web page is refreshed, more window choices appear in the list.

Unlike Chrome, we don't blow our chance to show the helpful OS prompt if the user rejects the request for the site in question. We don't show the OS prompt until the user shows at least some interest by picking a potential source (likely triggered by our preview render).

There's some kinks perhaps, like the "Entire screen" dud stream showing only the firefox window that appears to be shared successfully if OS permission is not granted. The OS seems to think this is a good idea. Hard to say. Would NotAllowedError be preferable?

Comment 8

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

(likely triggered by our preview render)

See bug 1588640 comment 4 for fixes.

Comment 9

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Temporary fixes added in bug 1588640 to raise and gate on the OS prompt consistently. Remaining UX work should go in bug 1494172.