1340069 - [WebRTC] Firefox doesn't send RTP during muting video

Status: RESOLVED INCOMPLETE

(Core :: WebRTC: Audio/Video, defect)

Description

[Mat]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0
Build ID: 20170125094131

Steps to reproduce:

1. Request stream with MediaDevices.getUserMedia(constraints)
2. Assign stream to video-element, video.srcObject = mediaStream

var constraints = { audio: true, video: { width: 1280, height: 720 } }; 

navigator.mediaDevices.getUserMedia(constraints)
.then(function(mediaStream) {
  var video = document.querySelector('video');
  video.srcObject = mediaStream;
  video.onloadedmetadata = function(e) {
    video.play();
  };
})
.catch(function(err) { console.log(err.name + ": " + err.message); });

3. Mute video with video.srcObject.getVideoTracks()[0].enabled = false;


Actual results:

Firefox ceases to send RTP-video.


Expected results:

Is it possible to send empty RTP (black screen) like Chrome?

Comment 1

[Byron Campen [:bwc]]

This seems like an implementation detail to me. Is there some spec that says we should send a black video stream?

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

No, we're not required to send video when disabled.  Typically we wait for a frame (or two) after muting to ensure we send _a_ black frame, and we send occasional black frames thereafter (if the source generates frames).  We send frames at max 1fps when forcing to black.

Comment 3

[Mat]

Let me describe my case, and why I think this is bug. I have application, which handles webrtc streams. Publisher create stream and push it to app, then calling mute video. Subscriber requests stream, which was muted, and doesn't receive audio because there is no video RTP.

Comment 4

[Byron Campen [:bwc]]

Wait, why is no audio getting through? Is Firefox not sending audio RTP when the video track is disabled?

Comment 5

[Mat]

Audio RTP is sending, but if you request A\V stream with muted video, then browser can't play audio stream if there is no video RTP (may be because of sync).

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Mat from comment #5)
> Audio RTP is sending, but if you request A\V stream with muted video, then
> browser can't play audio stream if there is no video RTP (may be because of
> sync).

Can you show a case of that?  Is the application failing firefox?

No application should fail to play audio just because video frames aren't present...  Though IIRC there is a bug in Chrome's <video> implementation that blocks audio until the first video frame - though it should play audio after the first frame.  This is a (well-known and annoying) bug in Chrome, unless they've fixed it in the last 6 months.

Comment 7

[Mat]

I will try to describe my case step by step:

1) Working case:
- Publisher (Chrome) pushes A\V stream to application, then calling "mute" video. Video RTP from publisher is present (black screen).
- Subscriber (Chrome/Firefox) requests muted stream from application and playing audio (I can hear it)because the stream contains both audio and video RTP.

2) NOT working case:
- Publisher (Firefox) pushes A\V stream to application, then calling "mute" video. Video RTP from publisher is not present.
- Subscriber (Chrome/Firefox) requests muted stream and not playing audio (I can't hear it).

In both cases SDP from app to subscriber contains audio and video parts.

Comment 8

[Andreas Pehrson [:pehrsons]]

Since the subscriber is also a browser, could you provide a complete test case that reproduces this? For instance putting both the publisher and subscriber in the same jsfiddle.

Like jesup said, lack of video shouldn't affect audio playback. Usually sync happens the other way around (video frames synced to audio clock).

Comment 9

[Byron Campen [:bwc]]

Cannot reproduce without reporter.