Open Bug 1576576 Opened 2 years ago Updated 5 months ago

RTCP REMB is always send via one video m-line in UnifiedPlan multi video line setup

Categories

(Core :: WebRTC, defect, P2)

68 Branch
defect

Tracking

()

People

(Reporter: s.wuttich, Unassigned)

References

(Depends on 1 open bug)

Details

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36

Steps to reproduce:

  1. Create peer connection with multiple video receivers which does not have bundle enabled (so separate ports per m-line).
  2. Send video to this peer connection (e.g. via remote SFU) on a video m-line which is not the first video m-line.

We have a video SFU here which relays up to 4 video streams to the clients. We cannot use bundle with our SFU, so we cannot verify if it would work correctly in this case.

Actual results:

REMB packets are send via a random video m-line (sometimes first or last, seems to be random with each negotiation) and not the one the video is received on.

Expected results:

REMB rtcp packets are send via m-line that receives the video.

Hi,
I'm setting the component, so the development team can take a look at this issue.
Thanks!

Component: Untriaged → WebRTC
Product: Firefox → Core

:bwc, any thoughts here?

Flags: needinfo?(docfaraday)

I don't think we have ever supported REMB in our JSEP code (I note that the draft for this expired more than 5 years ago...). So it seems like webrtc.org is sending REMB without it being negotiated?

Flags: needinfo?(docfaraday) → needinfo?(dminor)

(In reply to Byron Campen [:bwc] from comment #3)

I don't think we have ever supported REMB in our JSEP code (I note that the draft for this expired more than 5 years ago...). So it seems like webrtc.org is sending REMB without it being negotiated?

Maybe I do not get the point, but REMB is offered in the SDP from Firefox

v=0
o=mozilla...THIS_IS_SDPARTA-68.0.2 7003115358241982128 0 IN IP4 0.0.0.0
s=-
t=0 0
a=fingerprint:sha-256 38:78:44:E8:EA:AE:4E:48:6C:BC:A4:AF:0F:F1:6C:9F:AC:34:92:68:47:A8:9F:B4:1E:17:0E:FF:62:47:3E:AE
a=ice-options:trickle
a=msid-semantic:WMS *
m=video 9 UDP/TLS/RTP/SAVPF 120 121 126 97
c=IN IP4 0.0.0.0
a=sendrecv
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:5 urn:ietf:params:rtp-hdrext:toffset
a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1
a=fmtp:120 max-fs=12288;max-fr=60
a=fmtp:121 max-fs=12288;max-fr=60
a=ice-pwd:4fdd5bd3c05365474a285ea11e178fb7
a=ice-ufrag:2deb0801
a=mid:0
a=msid:- {723049b9-2e96-42e9-98d6-8d5d40dddddd}
a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir
a=rtcp-fb:120 goog-remb
a=rtcp-fb:121 nack
a=rtcp-fb:121 nack pli
a=rtcp-fb:121 ccm fir
a=rtcp-fb:121 goog-remb
a=rtcp-fb:126 nack
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 ccm fir
a=rtcp-fb:126 goog-remb
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=rtcp-fb:97 ccm fir
a=rtcp-fb:97 goog-remb
a=rtcp-mux
a=rtpmap:120 VP8/90000
a=rtpmap:121 VP9/90000
a=rtpmap:126 H264/90000
a=rtpmap:97 H264/90000
a=setup:actpass
a=ssrc:837655437 cname:{c76877a4-7c03-4e3a-85bd-6a5a98cae329}

Yes, based upon this comment [1] and associated code, REMB is sent without being negotiated, likely since the webrtc.org branch 64 update. After a quick look, it seems we could notch it out by modifying this function [2]. It would take more digging in the bitrate estimation code to see if there is a nicer way of disabling it. Since we're offering it in our SDP, perhaps that is not what we want to do.

[1] https://searchfox.org/mozilla-central/rev/9bb55ae4d808fc48afcf93f99da6a685265b86c6/media/webrtc/trunk/webrtc/call/video_receive_stream.h#155
[2] https://searchfox.org/mozilla-central/rev/9bb55ae4d808fc48afcf93f99da6a685265b86c6/media/webrtc/trunk/webrtc/modules/pacing/packet_router.cc#271

Flags: needinfo?(dminor)

(In reply to Dan Minor [:dminor] from comment #5)

Yes, based upon this comment [1] and associated code, REMB is sent without being negotiated, likely since the webrtc.org branch 64 update. After a quick look, it seems we could notch it out by modifying this function [2]. It would take more digging in the bitrate estimation code to see if there is a nicer way of disabling it. Since we're offering it in our SDP, perhaps that is not what we want to do.

[1] https://searchfox.org/mozilla-central/rev/9bb55ae4d808fc48afcf93f99da6a685265b86c6/media/webrtc/trunk/webrtc/call/video_receive_stream.h#155
[2] https://searchfox.org/mozilla-central/rev/9bb55ae4d808fc48afcf93f99da6a685265b86c6/media/webrtc/trunk/webrtc/modules/pacing/packet_router.cc#271

Any idea why we are enabling REMB seemingly at random, and not on all m-sections?

(In reply to Byron Campen [:bwc] from comment #6)

(In reply to Dan Minor [:dminor] from comment #5)

Yes, based upon this comment [1] and associated code, REMB is sent without being negotiated, likely since the webrtc.org branch 64 update. After a quick look, it seems we could notch it out by modifying this function [2]. It would take more digging in the bitrate estimation code to see if there is a nicer way of disabling it. Since we're offering it in our SDP, perhaps that is not what we want to do.

[1] https://searchfox.org/mozilla-central/rev/9bb55ae4d808fc48afcf93f99da6a685265b86c6/media/webrtc/trunk/webrtc/call/video_receive_stream.h#155
[2] https://searchfox.org/mozilla-central/rev/9bb55ae4d808fc48afcf93f99da6a685265b86c6/media/webrtc/trunk/webrtc/modules/pacing/packet_router.cc#271

Any idea why we are enabling REMB seemingly at random, and not on all m-sections?

Not offhand, I'd have to dig in a bit, which I can definitely do. The webrtc.org code seems to generate them as part of bitrate estimation, so I'm guessing there is something different with that between streams.

Is this a high priority bug or mostly an annoyance? Do we want to support REMB?

Mark it as P2 for now, but feel free to change its priority.

Priority: -- → P2
Assignee: nobody → dminor
Status: UNCONFIRMED → NEW
Ever confirmed: true

Just a brief comment: since Firefox doesn't support any other way of doing bandwidth estimation we should try to keep REMB support working.

It looks like what is going on is that each VideoSendStream registers its RTP modules with the PacketRouter here [1], and each RTPVideoStreamReceiver registers its RTP modules with the PacketRouter here [2]. This ends up being added to lists of potential modules on which to send REMB here [3]. Then PacketRouter::DetermineActiveRembModule() is called to determine when module on which to send REMB, but it just takes the first module on the lists, prioritizing send over receive [4], which would be consistent with the bug report that the module chosen seems random.

[1] https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/media/webrtc/trunk/webrtc/video/video_send_stream.cc#785
[2] https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/media/webrtc/trunk/webrtc/video/rtp_video_stream_receiver.cc#119
[3] https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/media/webrtc/trunk/webrtc/modules/pacing/packet_router.cc#236
[4] https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/media/webrtc/trunk/webrtc/modules/pacing/packet_router.cc#271

Hi Dan Minor,

I understand that there is no update on this issue, just wanted to check if this is something has/had to be fixed by FF. If yes, what is the status on it.
I'm facing some issue related to REMB only but it is with its bitrate value being very low send from FF to my client. Not sure if this is related to your above explanation or not.
We are facing issue as even at 2% of packet loss, FF is sending very low bandwidth in REMB(bitrate = ~100kbps) only where as real bandwidth available is close to 2-5mbps. We are using SIMULCAST(720,360,180p) for video-1 being send to my client. Not sure of SIMULCAST can create problem or not. I need just help to check that if it is related to this issue or should I open a new one.

(In reply to Pradeep Kumar from comment #11)

Hi Dan Minor,

I understand that there is no update on this issue, just wanted to check if this is something has/had to be fixed by FF. If yes, what is the status on it.
I'm facing some issue related to REMB only but it is with its bitrate value being very low send from FF to my client. Not sure if this is related to your above explanation or not.
We are facing issue as even at 2% of packet loss, FF is sending very low bandwidth in REMB(bitrate = ~100kbps) only where as real bandwidth available is close to 2-5mbps. We are using SIMULCAST(720,360,180p) for video-1 being send to my client. Not sure of SIMULCAST can create problem or not. I need just help to check that if it is related to this issue or should I open a new one.

Hi Pradeep, that sounds like a separate issue to me. Please file a new bug. Thank you!

(In reply to Dan Minor [:dminor] from comment #10)

It looks like what is going on is that each VideoSendStream registers its RTP modules with the PacketRouter here [1], and each RTPVideoStreamReceiver registers its RTP modules with the PacketRouter here [2]. This ends up being added to lists of potential modules on which to send REMB here [3]. Then PacketRouter::DetermineActiveRembModule() is called to determine when module on which to send REMB, but it just takes the first module on the lists, prioritizing send over receive [4], which would be consistent with the bug report that the module chosen seems random.

[1] https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/media/webrtc/trunk/webrtc/video/video_send_stream.cc#785
[2] https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/media/webrtc/trunk/webrtc/video/rtp_video_stream_receiver.cc#119
[3] https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/media/webrtc/trunk/webrtc/modules/pacing/packet_router.cc#236
[4] https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/media/webrtc/trunk/webrtc/modules/pacing/packet_router.cc#271

We've updated webrtc.org since they implemented unified plan, right? This may simply be a case of code that expects plan B (ie; a single video m-section, with multiple ssrcs, and a single transport).

Our version of webrtc.org is from before unified plan. But you're right, this is probably either fixed upstream or much easier to fix once we've done an update.

Assignee: dminor → nobody
Depends on: 1654112
You need to log in before you can comment on or make changes to this bug.