1643846 - getDisplayMedia for capturing a window does not stop MediaStreamTrack if the captured window is closed

Status: RESOLVED DUPLICATE of bug 1615282

(Core :: WebRTC: Audio/Video, defect)

Description

[devin.wilson]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36

Steps to reproduce:

Capture a window with getDisplayMedia and then close that window.

Here's a JS fiddle to make it easier to test:

1. Click capture and then select a window (different one than the window you are capturing from)
2. Close the window being captured.
3. Click the Update Stream Info button to see the current state
   https://jsfiddle.net/p9jk1ry8/2/

Actual results:

The MediaStreamTrack readyState is 'live' which means if the track is being sent through a peer connection, it will appear to be frozen on the remote side (if the receiver is Chrome the track muted oscillates between true and false, if Firefox there is no indication other than the content of the stream).

Expected results:

The MediaStreamTrack readyState should be 'ended' and the 'ended' event should have been emitted on the MediaStreamTrack instance. This is how Chrome behaves and is consistent with how getUserMedia behaves when a webcam is unplugged for example.

Comment 1

[Dan Minor [:dminor]]

Hi Jan-Ivar, could you please help me triage this one?

Comment 2

[devin.wilson]

Here's a link to the relevant part of the screen-capture spec.

Firefox also seems to not follow the mediacapture-streams spec when it comes to MediaStreamTracks transitioning muted to true for various scenarios for both screen capture and camera streams but cameras at least transition to 'ended' when unplugged.

Comment 4

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Hi Devin, and thanks for filing. Just wanted to follow up on this part:

(In reply to devin.wilson from comment #2)

Firefox also seems to not follow the mediacapture-streams spec when it comes to MediaStreamTracks transitioning muted to true for various scenarios for both screen capture and camera streams but cameras at least transition to 'ended' when unplugged.

Could you clarify which in scenarios you expected Firefox to fire mute and unmute, and which other browsers do this? I want to make sure I'm not missing some feature we should have.

Comment 5

[devin.wilson]

Yeah, the main thing I've noticed is that there are multiple scenarios where the stream is visibly frozen but track.muted is false and track.readyState is live on the receiving side of a peer connection. Below is a revelant part of the webrtc spec.

When one of the SSRCs for RTP source media streams received by an RTCRtpReceiver is removed either due to reception of a BYE or via timeout, it MUST queue a task to set the muted state of the corresponding MediaStreamTrack to true. Note that setRemoteDescription can also lead to the setting of the muted state of the track to the value true.

https://www.w3.org/TR/webrtc/#mediastreamtrack

Scenarios

In all of these, I have two RTCPeerConnections connected (in the same tab for simplicity so not cross-browser) and pc1 is sending a video track to pc2 which is recvonly. Testing done with Firefox 79.0 and Chrome 84.0.4147.105 on MacOS 10.15.6. Note: where not specified, assume readyState is live and muted is false.

Scenarios Where Chrome/Firefox Behave Differently

For each of these, on the receiving side in Firefox, there is no way that I know of to determine that the track is not playing back other than looking at webrtc stats or attaching it to a video element so you can see it.

1. If I call pc1.getSenders()[0].replaceTrack(null) (without negotiating)
   In Chrome: pc2.getReceivers()[0].track.muted is true
   In Firefox: pc2.getReceivers()[0].track.muted is false (though the media stops playing if it was attached to a video element).

2. If I unplug the camera I am sharing or close a screen share window that is being shared or revoke camera permissions (the last one only works in Firefox for an active camera, Chrome will not revoke permissions from an already captured stream)
   In Chrome: pc1.getSenders()[0].track.readyState is ended, pc1.getSenders()[0].track.muted is false, pc2.getReceivers()[0].track.readyState is live and pc2.getReceivers()[0].track.muted is true
   In Firefox: pc1.getSenders()[0].track.readyState is ended, pc1.getSenders()[0].track.muted is false, pc2.getReceivers()[0].track.readyState is live and pc2.getReceivers()[0].track.muted is false

3. If I call pc1.getSenders()[0].track.stop()
   In Chrome: pc1.getSenders()[0].track.readyState is ended, pc1.getSenders()[0].track.muted is false, pc2.getReceivers()[0].track.readyState is live and pc2.getReceivers()[0].track.muted is true
   In Firefox: pc1.getSenders()[0].track.readyState is ended, pc1.getSenders()[0].track.muted is false, pc2.getReceivers()[0].track.readyState is live and pc2.getReceivers()[0].track.muted is false

4. If I call pc1.removeTrack(pc1.getSenders()[0])
   In Chrome: pc1.getSenders()[0].track is null (but the actual track is live/unmuted), pc2.getReceivers()[0].track.readyState is live and pc2.getReceivers()[0].track.muted is true
   In Firefox: pc1.getSenders()[0].track is null (but the actual track is live/unmuted), pc2.getReceivers()[0].track.readyState is live and pc2.getReceivers()[0].track.muted is false

5. If I call pc1.getTransceivers()[0].stop() (sender)
   In Chrome: N/A because Chrome doesn't support transceiver.stop() and I didn't try
   In Firefox: pc1.getSenders()[0].track.readyState is live, pc1.getSenders()[0].track.muted is false, pc2.getReceivers()[0].track.readyState is live and pc2.getReceivers()[0].track.muted is false

6. If I call pc2.getTransceivers()[0].stop() (receiver)
   In Chrome: N/A because Chrome doesn't support transceiver.stop
   In Firefox: pc1.getSenders()[0].track.readyState is live, pc1.getSenders()[0].track.muted is false, pc2.getReceivers()[0].track.readyState is ended and pc2.getReceivers()[0].track.muted is false

Scenarios Where Chrome/Firefox Behave The Same

7. If I call pc1.getSenders()[0].track.enabled = false
   In Chrome and Firefox: pc1.getSenders()[0].track.muted is false, pc2.getReceivers()[0].track.muted is false, and pc2.getReceivers()[0].track.enabled is true

This scenario is consistent across browsers (other than Chrome not actually stopping capturing from the camera so the camera light is still on) but this behavior seems odd since the receiving side has no indication that the track is not playing back (unless I am missing something).

8. Set pc1.getTransceivers()[0].direction = 'recvonly' and negotiate the peer connections
   In Chrome and Firefox: pc1.getSenders()[0].track.readyState is live, pc1.getSenders()[0].track.muted is false, pc2.getReceivers()[0].track.readyState is live and pc2.getReceivers()[0].track.muted is true

This one works as I would expect