1625694 - getDisplayMedia media stream is missing cursor on Linux

Status: VERIFIED FIXED

(Core :: WebRTC, defect, P3)

Description

[Graham King]

Attachment: cursor.html

User Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0

Steps to reproduce:

When capturing screen using navigator.mediaDevices.getDisplayMedia({video: {cursor: "always"}}) the cursor does not render in the MediaStream.

This is a regression. Firefox renders the cursor correctly in 72.0.2 and 73.0.1. It does not render it in 74.0 and 75.0b10. Tested on Windows and MacOS.

I have attached an html file which reproduces it. That file is also here: https://darkcoding.net/cursor.html

Actual results:

The MediaStream did not include the cursor in FF 74 and 75. It DID include it in 72 and 73.

Expected results:

The MediaStream should include the cursor, according to the docs here: https://developer.mozilla.org/en-US/docs/Web/API/DisplayMediaStreamConstraints/video

Comment 1

[Dan Minor [:dminor]]

Thank you for your bug report! I did some testing with Firefox Nightly (Firefox 76). It is working for me under Windows and OS X, but is broken on Linux. But it doesn't appear to be a recent regression, I tested back to Firefox 71 and it was still affected.

Comment 2

[Graham King]

I can confirm it works on Windows and Mac with Nightly 76.0a1 (2020-03-30) (64-bit). That's great news!

I think Linux support should be a different bug. On Fedora / Wayland the cursor didn't show with FF 70 either, so probably it never worked. More importantly, Firefox only lets you select the full screen to share. Chromium can only share itself (full screen is black). I'm just happy it's already fixed for Mac and Win.

Comment 3

[Dan Minor [:dminor]]

We don't seem to already have a bug on file for Linux. Since this seems to now be working on Nightly, I'll change this bug to refer specifically to the Linux problem. Thanks for checking again with Nightly!

Comment 4

[Murz]

Here https://community.jitsi.org/t/show-mouse-cursor-when-sharing-the-screen/25009/3 is related Jitsi community thread.

Comment 5

[Dan Minor [:dminor]]

The problem seems to be that at [1] the result is always OUTSIDE. If I hard code it to INSIDE, the cursor is rendered as expected.

[1] https://searchfox.org/mozilla-central/rev/567b68b8ff4b6d607ba34a6f1926873d21a7b4d7/media/webrtc/trunk/webrtc/modules/desktop_capture/linux/mouse_cursor_monitor_x11.cc#161

Comment 6

[Dan Minor [:dminor]]

Attachment: Bug 1625694 - Remove unnecessary local modifications to mouse_cursor_monitor_x11; r=ng!

I think inner_window_ was added to handle the fact that window_ may not be
the same as child_window which could throw off the coordinates. Since we made
these modifications, upstream has added the XTranslateCoordinates call that
handles this case, so our local modifications are no longer necessary.

Comment 7

[Dan Minor [:dminor]]

Attachment: Bug 1625694 - Set use_desktop_relative_cursor_position in DesktopAndCursorComposer; r=ng!

Upstream has removed the mode used when use_desktop_relative_cursor_position_
is set to false as well as the associated override of OnMouseCursorPosition,
so the next time we update libwebrtc, this is the behaviour we will have.
Enabling it now fixes cursor display on Linux, which at the moment only works
for some windows.

Depends on D71392

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/407781119c09
Remove unnecessary local modifications to mouse_cursor_monitor_x11; r=jib,ng
https://hg.mozilla.org/integration/autoland/rev/191aba83fa73
Set use_desktop_relative_cursor_position in DesktopAndCursorComposer; r=ng

Comment 9

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/407781119c09
https://hg.mozilla.org/mozilla-central/rev/191aba83fa73

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:dminor, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.

Comment 11

[gwarser]

Possible regression https://bugzilla.mozilla.org/show_bug.cgi?id=1638091

Comment 12

[Dan Minor [:dminor]]

(In reply to gwarser from comment #11)

Possible regression https://bugzilla.mozilla.org/show_bug.cgi?id=1638091

This seems unlikely, this bug was about the cursor not being drawn during window sharing, but as far as I can tell, the symptoms in Bug 1638091 occur without a window being shared.

Comment 13

[Alexandru Trif, Desktop QA [:atrif]]

Reproduced the issue using Firefox 76.0a1 (20200328214049) on Ubuntu 18.04 and information from comment 0.
The issue is verified fixed with Firefox 78.0b7 (20200612174529) on Ubuntu 18.04.

Comment 14

[7baums]

Actually, this issue appears again in 78.0.1 but only when sharing a second screen, not the main one. It was working fine in 77.0.1, though.

Comment 15

[david.kirschenheuter]

I just encountered this problem as well.
What I tested:

• Windows 10, Firefox 83 -> no cursor (or the cursor is placed on the wrong position, but could not reproduce this behavior)
• Windows 10, Firefox Beta (84) -> no cursor
• Windows 10, Firefox nightly (85) -> no cursor
• Ubuntu 20.04, Firefox 83 -> cursor present

For testing I used Bigbluebutton and on Windows with Firefox 83 additional Zoom. But there are no differences between zoom or Bigbluebutton.

So I think we should change the status?

Comment 16

[Dan Minor [:dminor]]

(In reply to david.kirschenheuter from comment #15)

I just encountered this problem as well.
What I tested:

• Windows 10, Firefox 83 -> no cursor (or the cursor is placed on the wrong position, but could not reproduce this behavior)
• Windows 10, Firefox Beta (84) -> no cursor
• Windows 10, Firefox nightly (85) -> no cursor
• Ubuntu 20.04, Firefox 83 -> cursor present

For testing I used Bigbluebutton and on Windows with Firefox 83 additional Zoom. But there are no differences between zoom or Bigbluebutton.

So I think we should change the status?

Hi David,

This bug was specifically for Linux, which seems to still be working based on your testing. Please file a new bug for Windows.

Thanks!

Comment 17

[Chris]

Was there ever a Windows bug raised for this (as its still an issue)? I am looking but cannot see at the moment.

Comment 18

[Dan Minor [:dminor]]

(In reply to Chris from comment #17)

Was there ever a Windows bug raised for this (as its still an issue)? I am looking but cannot see at the moment.

Not that I am aware of. Please file one :)

Comment 19

[Chris]

For reference (in case anyone finds this thread) - https://bugzilla.mozilla.org/show_bug.cgi?id=1691329

Comment 20

[Dan Minor [:dminor]]

(In reply to Chris from comment #19)

For reference (in case anyone finds this thread) - https://bugzilla.mozilla.org/show_bug.cgi?id=1691329

Thank you!