Closed Bug 1465473 Opened 2 years ago Closed 2 years ago

[WebRTC] No video when using RTCPeerConnection multitrack

Categories

(Core :: WebRTC, defect, P2)

60 Branch
defect

Tracking

()

RESOLVED FIXED
mozilla63
Tracking Status
firefox63 --- fixed

People

(Reporter: mparisdiaz, Assigned: bwc)

References

Details

Attachments

(5 files)

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20180517113820

Steps to reproduce:

Using an SFU and one RTCPeerConnection managing multi tracks in each Firefox client:

  1 - ClientA joins the SFU
    1.1 - ClientA offers SFU to send 1 audio and 1 video tracks.
    1.2 - SFU answers ClientA accepting to receive the tracks

  2 - ClientA joins the SFU
    2.1 - ClientB offers sending 1 audio and 1 video tracks to the SFU
    2.2 -  SFU answers ClientB accepting the tracks
    2.3 - SFU offers ClientA to send the audio and video tracks from ClientB
    2.4 - ClientA answers SFU accepting to receive the audio and video tracks from ClientB
    2.5 - SFU offers ClientB to send the audio and video tracks from ClientA
    2.6 - ClientB answers SFU accepting to receive the audio and video tracks from ClientA

  3 - ClientC joins the SFU
    3.1 - ClientC offers sending 1 audio and 1 video tracks to the SFU
    3.2 -  SFU answers ClientC accepting the tracks
    3.3 - SFU offers ClientA to send the audio and video tracks from ClientC
    3.4 - ClientA answers SFU accepting to receive the audio and video tracks from ClientC
    3.5 - SFU offers ClientB to send the audio and video tracks from ClientC
    3.6 - ClientB answers SFU accepting to receive the audio and video tracks from ClientC
    3.7 - SFU offers ClientC to send the audio and video tracks from ClientA and ClientB
    3.8 - ClientC answers SFU accepting to receive the audio and video tracks from ClientA and ClientB
    


Actual results:

After step 3:
ClientA renders video and audio from ClientB and ClientC
ClientB renders video and audio from ClientA and ClientC
ClientC renders video and audio from ClientA, but only audio from ClientB (**no video form ClientB**)


Expected results:

ClientC should render video from ClientB
After deeply debugging the issue, I have noticed that ClientC does not render video from ClientB due to ClientB does not generate a KeyFrame on RTCP PLI requests, so ClientC cannot start decoding the video until the first KeyFrame arrives (this might happen after some minutes).

First of all I pointed out to a possible bug in the SFU generating the RTCP PLI packets, but I did not see anything and the next cases tell me that there should be some bug in Firefox side when there are more than 1 remote video track:
 - We have not suffered this issued if Chrome is used.
 - At step 2 (from the bug description case), ClientA is requested by a KeyFrame through an RTCP PLI request and it is generated.
 - Time to time, at step 3, ClientC renders video from ClientB.
 - If we add a 4th step, where ClientA unjoins the SFU (forcing the related SDP renegotiations), ClientC starts rendering video from ClientB, that is, ClientB generates a KeyFrame (I do not know if it is generated because the PLI requests are listened, due to the SDP renegotiation or whatever).

I hope this info is enough to start debugging the issue, if not, please let me know ;).
Status: UNCONFIRMED → NEW
Rank: 12
Component: Untriaged → WebRTC
Ever confirmed: true
Flags: needinfo?(drno)
Priority: -- → P2
Product: Firefox → Core
Thanks for the report. Since this problem does not exist with all SFU's out there I'm wondering what is different in your case to trigger this.

In case you haven't done it already: it would be super helpful to turn on RT(C)P logging (https://blog.mozilla.org/webrtc/debugging-encrypted-rtp-is-more-fun-than-it-used-to-be/) on the Firefox which appears to ignore the PLI, to confirm that it actually receives the PLI.

If we know that the PLI is received, but ignored by Firefox I think we would need the logging from Firefox internal webrtc.org code https://wiki.mozilla.org/Media/WebRTC/Logging#Media_.28the_GIPS_stack:_RTP.2FRTCP.29

Alternatively a way for us to repro it ourself would be super helpful.
Flags: needinfo?(drno)
First of all,
thanks for the feedback Nils!!

For now, it will be difficult to give you a way to reproduce the problem, so I will try to get the info you requested to see if it is enough.
Only one clue more: it does not work with PLIs neither FIRs.
Attached file aboutWebrt-001.html
about:webrtc page related to PLI requested for MediaStreamTrack {aa7a3655-9745-4bc3-9de4-1b64eefab2b1} (ssrc: 2607954657)
Attached file moz-plis-001-rtcp.log
RTCP logs related to PLI requested for MediaStreamTrack {aa7a3655-9745-4bc3-9de4-1b64eefab2b1} (ssrc: 2607954657)
Attached file rtp-plis-001.pcap
RTP/RTCP pcap related to PLI requested for MediaStreamTrack {aa7a3655-9745-4bc3-9de4-1b64eefab2b1} (ssrc: 2607954657)
From attached docs, we could say that Firefox is receiving PLI requests.
Byron, do you or someone else on networking have time to look at this?
Flags: needinfo?(docfaraday)
So it looks like those PLI packets for ssrc 2607954657 are all in a compound RTCP packet beginning with a sender report. There's a pretty good chance this is causing the bundle filter to route them incorrectly. Let me see what happens if I disable the bundle filter for RTCP entirely.
Attached image treeherder-mozilla.png
Hello Byron!!
First of all thanks for providing this version.

How can I get it to install and test it with the SFU?
I am following the steps Nils told me in [1] but there is not any link next to the word "Build" as you can see in treeherder-mozilla.png picture.

Refs
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1323723#c12
(In reply to mparisdiaz from comment #11)
> Created attachment 8993243 [details]
> treeherder-mozilla.png
> 
> Hello Byron!!
> First of all thanks for providing this version.
> 
> How can I get it to install and test it with the SFU?
> I am following the steps Nils told me in [1] but there is not any link next
> to the word "Build" as you can see in treeherder-mozilla.png picture.
> 
> Refs
> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1323723#c12

If you want to test the build you can get a tarball "target.tar.bz2" (linux64 opt or pgo, similarly for other platforms) of the build from the "Job Details" tab.
Thanks for the support Andreas!!

I have been able to test it and there is good news:
I am not able to reproduce the issue with the Nightly version provided by Byron while I am able with Firefox 61.0.1.

Hence, it seems that the problem is in the "RTCP bundle filtering".
I guess you should perform more work and not only disable it. In that case when it is fixed and if you need it I could help you to test it again ;).
(In reply to mparisdiaz from comment #13)
> Thanks for the support Andreas!!
> 
> I have been able to test it and there is good news:
> I am not able to reproduce the issue with the Nightly version provided by
> Byron while I am able with Firefox 61.0.1.
> 
> Hence, it seems that the problem is in the "RTCP bundle filtering".
> I guess you should perform more work and not only disable it. In that case
> when it is fixed and if you need it I could help you to test it again ;).

In the meantime, you can probably work around this by not putting RTCP related to streams you're receiving in compound RTCP packets beginning with a SR. I am still not sure whether disabling this filtering is going to break something else. We might end up needing to do something more difficult like divide these compound packets into their individual RTCP packets, and demux each separately.
Hello Byron!!

From my side, I would like to perform the work-around you propose but it is so dramatic and imply a lot of effort. So I guess we will declare this as known issue for our customers.

From your side and with all the respects I think you should fix it as it belongs to the RFC 3550 basics [1] should be breaking the compatibility with other systems, even with p2p communications with other browsers :S.

Refs
[1] https://tools.ietf.org/html/rfc3550#section-3
Assignee: nobody → docfaraday
Flags: needinfo?(docfaraday)
See Also: → 1464448
Attachment #8993358 - Flags: review?(mfroman)
Comment on attachment 8993358 [details]
Bug 1465473: Disable RTCP bundle filtering, because webrtc.org can do that for us.

https://reviewboard.mozilla.org/r/258142/#review265810

Looks good to me.
Attachment #8993358 - Flags: review?(mfroman) → review+
Pushed by bcampen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/aaa759a132c7
Disable RTCP bundle filtering, because webrtc.org can do that for us. r=mjf
https://hg.mozilla.org/mozilla-central/rev/aaa759a132c7
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
You need to log in before you can comment on or make changes to this bug.