1465473 - [WebRTC] No video when using RTCPeerConnection multitrack

Status: RESOLVED FIXED

(Core :: WebRTC, defect, P2)

Description

[mparisdiaz]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20180517113820

Steps to reproduce:

Using an SFU and one RTCPeerConnection managing multi tracks in each Firefox client:

  1 - ClientA joins the SFU
    1.1 - ClientA offers SFU to send 1 audio and 1 video tracks.
    1.2 - SFU answers ClientA accepting to receive the tracks

  2 - ClientA joins the SFU
    2.1 - ClientB offers sending 1 audio and 1 video tracks to the SFU
    2.2 -  SFU answers ClientB accepting the tracks
    2.3 - SFU offers ClientA to send the audio and video tracks from ClientB
    2.4 - ClientA answers SFU accepting to receive the audio and video tracks from ClientB
    2.5 - SFU offers ClientB to send the audio and video tracks from ClientA
    2.6 - ClientB answers SFU accepting to receive the audio and video tracks from ClientA

  3 - ClientC joins the SFU
    3.1 - ClientC offers sending 1 audio and 1 video tracks to the SFU
    3.2 -  SFU answers ClientC accepting the tracks
    3.3 - SFU offers ClientA to send the audio and video tracks from ClientC
    3.4 - ClientA answers SFU accepting to receive the audio and video tracks from ClientC
    3.5 - SFU offers ClientB to send the audio and video tracks from ClientC
    3.6 - ClientB answers SFU accepting to receive the audio and video tracks from ClientC
    3.7 - SFU offers ClientC to send the audio and video tracks from ClientA and ClientB
    3.8 - ClientC answers SFU accepting to receive the audio and video tracks from ClientA and ClientB
    


Actual results:

After step 3:
ClientA renders video and audio from ClientB and ClientC
ClientB renders video and audio from ClientA and ClientC
ClientC renders video and audio from ClientA, but only audio from ClientB (**no video form ClientB**)


Expected results:

ClientC should render video from ClientB

Comment 1

[mparisdiaz]

After deeply debugging the issue, I have noticed that ClientC does not render video from ClientB due to ClientB does not generate a KeyFrame on RTCP PLI requests, so ClientC cannot start decoding the video until the first KeyFrame arrives (this might happen after some minutes).

First of all I pointed out to a possible bug in the SFU generating the RTCP PLI packets, but I did not see anything and the next cases tell me that there should be some bug in Firefox side when there are more than 1 remote video track:
 - We have not suffered this issued if Chrome is used.
 - At step 2 (from the bug description case), ClientA is requested by a KeyFrame through an RTCP PLI request and it is generated.
 - Time to time, at step 3, ClientC renders video from ClientB.
 - If we add a 4th step, where ClientA unjoins the SFU (forcing the related SDP renegotiations), ClientC starts rendering video from ClientB, that is, ClientB generates a KeyFrame (I do not know if it is generated because the PLI requests are listened, due to the SDP renegotiation or whatever).

I hope this info is enough to start debugging the issue, if not, please let me know ;).

Comment 2

[Nils Ohlmeier [:drno]]

Thanks for the report. Since this problem does not exist with all SFU's out there I'm wondering what is different in your case to trigger this.

In case you haven't done it already: it would be super helpful to turn on RT(C)P logging (https://blog.mozilla.org/webrtc/debugging-encrypted-rtp-is-more-fun-than-it-used-to-be/) on the Firefox which appears to ignore the PLI, to confirm that it actually receives the PLI.

If we know that the PLI is received, but ignored by Firefox I think we would need the logging from Firefox internal webrtc.org code https://wiki.mozilla.org/Media/WebRTC/Logging#Media_.28the_GIPS_stack:_RTP.2FRTCP.29

Alternatively a way for us to repro it ourself would be super helpful.

Comment 3

[mparisdiaz]

First of all,
thanks for the feedback Nils!!

For now, it will be difficult to give you a way to reproduce the problem, so I will try to get the info you requested to see if it is enough.
Only one clue more: it does not work with PLIs neither FIRs.

Comment 4

[mparisdiaz]

Attachment: aboutWebrt-001.html

about:webrtc page related to PLI requested for MediaStreamTrack {aa7a3655-9745-4bc3-9de4-1b64eefab2b1} (ssrc: 2607954657)

Comment 5

[mparisdiaz]

Attachment: moz-plis-001-rtcp.log

RTCP logs related to PLI requested for MediaStreamTrack {aa7a3655-9745-4bc3-9de4-1b64eefab2b1} (ssrc: 2607954657)

Comment 6

[mparisdiaz]

Attachment: rtp-plis-001.pcap

RTP/RTCP pcap related to PLI requested for MediaStreamTrack {aa7a3655-9745-4bc3-9de4-1b64eefab2b1} (ssrc: 2607954657)

Comment 7

[mparisdiaz]

From attached docs, we could say that Firefox is receiving PLI requests.

Comment 8

[Andreas Pehrson [:pehrsons]]

Byron, do you or someone else on networking have time to look at this?

Comment 9

[Byron Campen [:bwc]]

So it looks like those PLI packets for ssrc 2607954657 are all in a compound RTCP packet beginning with a sender report. There's a pretty good chance this is causing the bundle filter to route them incorrectly. Let me see what happens if I disable the bundle filter for RTCP entirely.

Comment 10

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4d9eef7456d501b23e429d7f8aa07bb4a0409072

Comment 11

[mparisdiaz]

Attachment: treeherder-mozilla.png

Hello Byron!!
First of all thanks for providing this version.

How can I get it to install and test it with the SFU?
I am following the steps Nils told me in [1] but there is not any link next to the word "Build" as you can see in treeherder-mozilla.png picture.

Refs
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1323723#c12

Comment 12

[Andreas Pehrson [:pehrsons]]

(In reply to mparisdiaz from comment #11)
> Created attachment 8993243 [details]
> treeherder-mozilla.png
> 
> Hello Byron!!
> First of all thanks for providing this version.
> 
> How can I get it to install and test it with the SFU?
> I am following the steps Nils told me in [1] but there is not any link next
> to the word "Build" as you can see in treeherder-mozilla.png picture.
> 
> Refs
> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1323723#c12

If you want to test the build you can get a tarball "target.tar.bz2" (linux64 opt or pgo, similarly for other platforms) of the build from the "Job Details" tab.

Comment 13

[mparisdiaz]

Thanks for the support Andreas!!

I have been able to test it and there is good news:
I am not able to reproduce the issue with the Nightly version provided by Byron while I am able with Firefox 61.0.1.

Hence, it seems that the problem is in the "RTCP bundle filtering".
I guess you should perform more work and not only disable it. In that case when it is fixed and if you need it I could help you to test it again ;).

Comment 14

[Byron Campen [:bwc]]

(In reply to mparisdiaz from comment #13)
> Thanks for the support Andreas!!
> 
> I have been able to test it and there is good news:
> I am not able to reproduce the issue with the Nightly version provided by
> Byron while I am able with Firefox 61.0.1.
> 
> Hence, it seems that the problem is in the "RTCP bundle filtering".
> I guess you should perform more work and not only disable it. In that case
> when it is fixed and if you need it I could help you to test it again ;).

In the meantime, you can probably work around this by not putting RTCP related to streams you're receiving in compound RTCP packets beginning with a SR. I am still not sure whether disabling this filtering is going to break something else. We might end up needing to do something more difficult like divide these compound packets into their individual RTCP packets, and demux each separately.

Comment 15

[mparisdiaz]

Hello Byron!!

From my side, I would like to perform the work-around you propose but it is so dramatic and imply a lot of effort. So I guess we will declare this as known issue for our customers.

From your side and with all the respects I think you should fix it as it belongs to the RFC 3550 basics [1] should be breaking the compatibility with other systems, even with p2p communications with other browsers :S.

Refs
[1] https://tools.ietf.org/html/rfc3550#section-3

Comment 16

[Byron Campen [:bwc]]

Attachment: Bug 1465473: Disable RTCP bundle filtering, because webrtc.org can do that for us.

Review commit: https://reviewboard.mozilla.org/r/258142/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/258142/

Comment 17

[Byron Campen [:bwc]]

Comment on attachment 8993358 [details]
Bug 1465473: Disable RTCP bundle filtering, because webrtc.org can do that for us.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/258142/diff/1-2/

Comment 18

[Byron Campen [:bwc]]

Comment on attachment 8993358 [details]
Bug 1465473: Disable RTCP bundle filtering, because webrtc.org can do that for us.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/258142/diff/2-3/

Comment 19

[Michael Froman [:mjf]]

Comment on attachment 8993358 [details]
Bug 1465473: Disable RTCP bundle filtering, because webrtc.org can do that for us.

https://reviewboard.mozilla.org/r/258142/#review265810

Looks good to me.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/aaa759a132c7
Disable RTCP bundle filtering, because webrtc.org can do that for us. r=mjf

Comment 21

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/aaa759a132c7