1674463 - The highest quality simulcast stream will cut out for approx. 3 seconds with Jitsi.

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect, P1)

Description

[Nico Grunbaum [:ng, @chew:mozilla.org]]

The bandwidth estimation causes a dropout of video with Jitsi on the largest of simulcast stream.

Comment 1

[Michael Froman [:mjf]]

Attachment: Bug 1674463 - allow HD simulcast stream to start a bit earlier. r?ng!

Changing the min bitrate from 600k to 200k to allow the HD stream to stay
on more consistently.

Comment 2

[Nico Grunbaum [:ng, @chew:mozilla.org]]

The code assumes that the streams will be a constant 30FPS, which isn't true in practice. This should give us more leeway in the minimum bitrate.

Comment 3

[Michael Froman [:mjf]]

As a word of explanation after much digging around:

1. As simulcast streams are stopped due to reduced available bandwidth on the send-side,
   the receive side will freeze for 3 seconds. This has been verified when Chrome is
   the receiver as well. This seems to be a Jitsi bug.
2. Since we were seeing freezes more often than when Chrome was the sender that could
   only mean that we were switching from the HD to SD stream more often than when Chrome
   was the sender. This pointed to a codec configuration issue.
3. We had a very large gap between the LD+SD target bitrates (650k) and the HD minimum
   bitrate of 600k. This meant that available bitrate would need to get to 1250k before
   we started sending the HD stream. If the bitrate dropped below 1250k, we would stop
   sending the HD stream causing the 3 second freeze on the receive side.

The minimum bitrate for the HD stream comes from here:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/dom/media/webrtc/libwebrtcglue/VideoStreamFactory.cpp#34

The code for assigning the bitrate allocations for simulcast is here:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/simulcast_rate_allocator.cc#37-88

RTCPReceiver::TriggerCallbacksFromRtcpPacket eventually causes VideoSender::SetChannelParameters[1] to execute which stores the new bitrate allocation in encoder_params_ here[2];
[1] https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/video_sender.cc#215
[2] https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/video_sender.cc#234

VideoSender::AddVideoFrame is calling SetEncoderParameters (with encoder_params_) which eventually calls VP8EncoderImpl::SetRateAllocation.

Whether a stream is transmitted or not is decided here in send_stream_:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc#955

The send_stream_ variable is set here in response to VP8EncoderImpl::SetRateAllocation:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc#309

On the receiving side, the freeze happens here waiting for a frame:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/video/video_receive_stream.cc#420-421

Finally, the key to nailing this issue down was forcing the HD simulcast stream to off,
and then predictably changing between the LD and SD streams approximately every 10 seconds.
This allow the freeze on downgrading streams to be easily observed. The patch to try
this is here:

diff --git a/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc b/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc
--- a/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc
+++ b/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc
@@ -299,10 +299,22 @@ int VP8EncoderImpl::SetRateAllocation(co
     }
   }

+static std::vector<unsigned int> fake_kbps{150, 500, 0};
+static time_t last_change(time(0));
+
   size_t stream_idx = encoders_.size() - 1;
   for (size_t i = 0; i < encoders_.size(); ++i, --stream_idx) {
+time_t now = time(0);
+if (now - last_change > 10) {
+if (fake_kbps[1]) fake_kbps[1] = 0;
+else fake_kbps[1] = 500;
+printf("MJF: %p VP8EncoderImpl::SetRateAllocation switch to: %s\n",
+       this, (!fake_kbps[1]?"LD":"SD"));
+last_change = now;
+}
     unsigned int target_bitrate_kbps =
-        bitrate.GetSpatialLayerSum(stream_idx) / 1000;
+        fake_kbps[stream_idx];
+//         bitrate.GetSpatialLayerSum(stream_idx) / 1000;

     bool send_stream = target_bitrate_kbps > 0;
     if (send_stream || encoders_.size() > 1)

Comment 4

[Michael Froman [:mjf]]

Before gtest fix using mach try auto and adding media mochitests:
https://treeherder.mozilla.org/jobs?repo=try&revision=d197c78ddef5bec8809cbbc8953151a12ab4a38e

After gtest fix:
https://treeherder.mozilla.org/jobs?repo=try&revision=65eacf17b2d0e80ba7f16610435fed4e22e9f834

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4ad213bd3283
allow HD simulcast stream to start a bit earlier. r=ng

Comment 6

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/4ad213bd3283