Open Bug 1637791 Opened 10 months ago Updated 3 months ago

Show a highlight around any windows shared over WebRTC, or around the edge of the screen when sharing a screen.

Categories

(Firefox :: Site Permissions, enhancement, P3)

enhancement

Tracking

()

People

(Reporter: mconley, Unassigned)

References

(Depends on 1 open bug, Blocks 1 open bug, )

Details

Attachments

(8 files, 1 obsolete file)

47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
2.34 MB, image/png
Details
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review

For bonus points, ideally, these highlights would not be captured by the WebRTC desktop capture layer. That should be a fun challenge.

Hey dminor,

It looks like there's a ::SetExcludedWindow capability in some of the WebRTC screen capturing stuff, but it looks to me as if it's only implemented for macOS... looking through the Chromium source code, I don't see them using it at all over there, too.

It's likely that we'll need to implement this highlight using a special transparent OS-window, and if we don't want it to be captured during a screen capture, I presume we'd want to use something like ::SetExcludedWindow.

Hey dminor, suppose I were to try to implement ::SetExcludedWindow for Windows... do you know if there'd be any difficulty in getting that change landed in the third-party webrtc code?

Flags: needinfo?(dminor)

Hi Mike, In this case, since it's already implemented for one platform, I suspect your chances of getting it landed upstream for a different platform are pretty good. The desktop capture code seems to change pretty slowly, so I expect your patch for Firefox would apply upstream without too many difficulties. I have a webrtc.org account, I can help get it up for review once done, although I might need your help addressing any review comments from upstream.

Flags: needinfo?(dminor)
Priority: -- → P1
Blocks: 1642798
Group: mozilla-employee-confidential
Attached file Bug 1637791 - Cocoa window hack (obsolete) —

Depends on D75408

So the WIP implementation here is a prototype that uses a transparent DOM window to draw the highlight. This appears to mostly work, however there's a bit of a performance penalty on the graphics side of things - since the widget layer isn't drawing the highlight, we can't take advantage of the OS compositor to cheaply composite it, which means that the highlights add overhead. Not a good thing when sharing over WebRTC (which is already pretty intensive from the OS compositor's standpoint).

After consulting with mstange, I'm going to change tack here a little and try having the widget layer draw the highlights. The general idea is to add properties to the nsIAppWindow and nsIScreen interfaces to draw the highlights.

Here's a demo from shorlander on how the highlight should be layered to maximize visibility across a variety of backgrounds: http://stephenhorlander.com/sharing-highlight/sharing-highlight.html

Attachment #9158611 - Attachment is obsolete: true

This seems like a nice feature. Mike, are you still working on this? I'm assuming it's backlog for now.

Severity: -- → N/A
Flags: needinfo?(mconley)
Priority: P1 → P3

Yeah, this is backlogged, thanks.

Flags: needinfo?(mconley)
You need to log in before you can comment on or make changes to this bug.