1691329 - Cursor missing on Windows WebRTC calls when sharing screen (dual-monitor setup)

Status: VERIFIED FIXED

(Core :: WebRTC, defect, P2)

Description

[Chris]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36 Edg/88.0.705.63

Steps to reproduce:

Connect to a WebRTC call (Pexip), share screen.
Similar to https://bugzilla.mozilla.org/show_bug.cgi?id=1625694 (which was for Linux). Exists in current Nightly.

Actual results:

The far side sees the full screen but doesn't see the mouse pointer

Expected results:

The far side sees the full screen and sees the mouse pointer

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::WebRTC' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[david.kirschenheuter]

(In reply to Chris from comment #0)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36 Edg/88.0.705.63

Steps to reproduce:

Connect to a WebRTC call (Pexip), share screen.
Similar to https://bugzilla.mozilla.org/show_bug.cgi?id=1625694 (which was for Linux). Exists in current Nightly.

Actual results:

The far side sees the full screen but doesn't see the mouse pointer

Expected results:

The far side sees the full screen and sees the mouse pointer

Thanks for reporting this, I should have done this already a few month ago...
I can confirm this Bug on Firefox using several WebRTC-Tools (Bigbluebutton, Jitsi...) and Windows (Not more a problem on Linux).

Comment 3

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Hi, can you confirm using https://jsfiddle.net/jib1/q75yb8pf/ ?

I'm not able to repro. The cursor appears for me using that fiddle on both MacOS and Windows 10.

What OS and Firefox version is this on? Are you sharing "Entire screen"?

Comment 4

[Chris]

Using https://jsfiddle.net/jib1/q75yb8pf/ in FF 85.0.2 does not work on Windows 10. In Edge and Chrome, it works fine.

In addition, here is a screen recording of a WebRTC conference via Pexip using FF and Edge -https://www.screencast.com/t/VaprJtzWNx.Again Edge is fine, FF not so.

Comment 5

[Chris]

(In reply to Chris from comment #4)

Using https://jsfiddle.net/jib1/q75yb8pf/ in FF 85.0.2 does not work on Windows 10. In Edge and Chrome, it works fine.

In addition, here is a screen recording of a WebRTC conference via Pexip using FF and Edge -https://www.screencast.com/t/VaprJtzWNx.Again Edge is fine, FF not so.

For the record, the exact Windows 10 version is :

Edition Windows 10 Pro
Version 20H2
Installed on ‎25/‎07/‎2020
OS build 19042.804
Experience Windows Feature Experience Pack 120.2212.551.0

An as can be seen in the screencast, it is the entire screen that is shared in FF

Comment 6

[Chris]

Further as per @david.kirschenheuter comment's, this is replicated on Jitsi meet.

Comment 7

[Michael Froman [:mjf]]

(In reply to Chris from comment #4)

Using https://jsfiddle.net/jib1/q75yb8pf/ in FF 85.0.2 does not work on Windows 10. In Edge and Chrome, it works fine.

In addition, here is a screen recording of a WebRTC conference via Pexip using FF and Edge -https://www.screencast.com/t/VaprJtzWNx.Again Edge is fine, FF not so.

Chris, can you say more about the fiddle not working under Firefox 85.0.2? What exactly is happening? I just did updates on my windows 10 box to match yours and the fiddle worked so I'm curious what behavior you're seeing.

Also, would you mind testing in a fresh profile? If you go to about:profiles and click "Create a New Profile", you can launch that new profile in a new copy of the browser to test the fiddle. You can delete the new profile when you're finished.

Comment 8

[Chris]

hi @micheal, WRT to what I meant that was not working in jsfiddle in FF, it was identical to that shown in the WebRTC screencast above, and as is detailed as the subject of this bug - I.e. no pointer is a seen when sending a presentation.

FYI, I created a new profile launch in that profile and re-opened https://jsfiddle.net/jib1/q75yb8pf/, but get the same result. FYI, another screencast showing this: https://www.screencast.com/t/geSWKKpE

Comment 9

[Michael Froman [:mjf]]

Chris, is this a dual-screen setup?

Comment 10

[Chris]

Correct, you can see Screen 1 and Screen 2 clearly in both screencasts on both FF and Chrome/Edge.

Comment 11

[Chris]

Clarification, you can see clearly the screen and screen 2 can be shared as presentation sources in both screencasts. The screencast itself was only one screen. Did you view the screencasts? They speak volumes.

Comment 12

[Michael Froman [:mjf]]

(In reply to Chris from comment #11)

Clarification, you can see clearly the screen and screen 2 can be shared as presentation sources in both screencasts. The screencast itself was only one screen. Did you view the screencasts? They speak volumes.

I don't have dual-monitors, so after viewing I wanted to verify that "Screen 1" and "Screen 2" indicated a dual-monitor setup.

Comment 13

[Michael Froman [:mjf]]

Would you mind trying the same test, but without Firefox in full-screen mode? I've recently seen another bug where there was an issue directly related to full-screen mode and sharing. I'm trying to pin down the variables.

Comment 14

[Chris]

@Michael. sure - I assume when you say, "trying the same test, but without Firefox in full-screen mode", you mean that you would like me to share an application?

Comment 15

[Chris]

Ooooh, I have some interesting results - captured in their screencast (https://www.screencast.com/t/NXfJA3dUgNXt), which now captures both screens. FF on the left, Chrome on the right.

I have shared a notepad application and move it from one screen to the next. Looks like the mouse tracking from FF is completely off, whilst Chrome is fine. Moving the application to the other screen, and the mouse disappears, but perhaps it just that the tracking is so far out, you can't see the mouse.

Fells like someone has applied some wrong mathematics in the code somewhere....

Comment 16

[Michael Froman [:mjf]]

Chris, sorry that wasn't clear. I'm not asking for you to share an application. In the videos, it appears you have Firefox maximized (taking the full screen). Could you try the same steps as before, but with Firefox not maximized to consume the entire screen?

Comment 17

[Michael Froman [:mjf]]

Also, thank you for continuing to try new things to help debug! This is very helpful!

Comment 18

[Chris]

@michael - actually looks can be deceptive ;)

FF was NOT maximised in the last video (it was in the previous ones). I have a slightly different resolution on screen 2 (right) to screen 1, meaning I simply resized the FF window to make the best use of the video area. It makes no real difference if FF is maximised or windowed.

I would say though that sharing screen 1 (left) does seem to work fine!!!!

Comment 19

[david.kirschenheuter]

Just to add some more info: I can confirm the problem on a dual-display setup. On the same machine and using the same WebRTC tool but plugging out the second display the cursor appears in the screen sharing.
Fullscreen or not seems not to influence the behaviour.
If there is any more info I can support with, let me know!

Comment 20

[Chris]

This is replicable on the Mac as well.

Comment 21

[Michael Froman [:mjf]]

Dan, here is another new screensharing issue if you have any thoughts. Thanks!

Comment 22

[Dan Minor [:dminor]]

This is broken for me whether or not I have a second monitor installed on Windows. It also a problem on OS X.

This was regressed by Bug 1625694, where I fixed a missing cursor in Linux, apparently fixing it for Linux, broke things on OS X and Windows. That is my fault, I thought I was changing OS independent code, but I should have still tested on each OS to make sure.

The patch for Bug 1625694 was very small, changing true to false here https://searchfox.org/mozilla-central/source/third_party/libwebrtc/webrtc/modules/desktop_capture/desktop_and_cursor_composer.cc#133 fixes things for me on OS X, but we'll have to retest on Windows as well. This will presumably break Linux again :/.

From Bug 1625694#c7 it looks like I thought the mode that is currently working for us has been removed upstream, so even if we fix this here, it might break again as part of the libwebrtc update.

Comment 23

[Dan Minor [:dminor]]

I think we want to flip the bool back to false here, which will unfortunately regress Linux temporarily until the branch update lands.

Comment 24

[Dan Minor [:dminor]]

Attachment: Bug 1691329 - Fix cursor capture for Windows and OS X; r=ng

This hacks around the fact we want to use desktop relative cursor position
on Linux but not on Windows and OS X. This version of the constructor has
been removed upstream, so this will need to be fixed in a different way
as part of the libwebrtc update.

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5dc83a74daab
Fix cursor capture for Windows and OS X; r=ng

Comment 26

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/5dc83a74daab

Comment 27

[Chris]

Nice - thanks

Comment 28

[Giorgia Nichita, Release Desktop QA]

I have reproduced the issue using STR from comment 0, on an affected Nightly build 2021-02-07.
The fix is verified on 88RC (20210207095322) on Windows 10x64. The issue does not reproduce on macOS 10.15, but on Firefox 88.0 the cursor is correctly displayed on the second screen.