1341595 - Investigate resolution scaling at low bandwidth

Status: NEW

(Core :: WebRTC: Networking, defect, P3)

Description

[Gavin Llewellyn]

Attachment: Wireshark IO graph for data sent to remote client

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36

Steps to reproduce:

* Set up an WebRTC session between two Firefox instances on different machines using https://appr.tc
* Restricted the send bandwidth on one of the two clients. I used a hardware device to do this, but other options are available (e.g. https://github.com/urbenlegend/netimpair). Ideally it should use a queue when applying a rate limit, so the network latency increases before packets are dropped; I was testing with a queue size equivalent to 250ms at the target rate.


Actual results:

As I reduced the throttle in 100kbps steps, the congestion control mechanism would adjust the sending bitrate to match, avoiding ongoing packet loss.  However, once I reached 300kbps, Firefox failed to reduce the send rate sufficiently, resulting in ongoing significant packet loss (the hardware device reported that it was dropping about 20-25% of packets), laggy video, and audio artefacts.

The attached graph shows the Firefox send rate as reported by Wireshark.  You can see throttle steps from 500kbps->600->500->400->300.  At 300, note that the send rate is marginally above the 300kbps line; after a period of laggy video I had an ICE disconnect, and had to reconnect to the room.  After reconnecting the throttle was still in place, but the send rate was generally even higher than before.


Expected results:

Firefox should have made adjustments to allow it to send a lower bitrate (e.g. reduce the video resolution), or have disabled the video entirely if further bitrate reduction is not supported.  Applying a minimum to the send data rate even when congestion is evident (packet loss, lower reported REMB than the send rate) does not seem like a good idea.

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

What version did you test with?

Somewhat unlike Chrome (though they've added some of this more recently), we limit bandwidth relative to the resolution and framerate.  For VGA, we normally use 200-1500Kbps, and start at 300.  Note that on top of that you have audio (~70Kbps) plus overhead (~25Kbps perhaps).  So the lower limit right now is around 300Kbps for VGA 30fps.  I think Chrome uses lower minimum limits, but below 200Kbps, VGA@30fps will get pretty ugly.

What it should do if bandwidth is estimated below the minimum (for some period) is to downscale the resolution, with hysteresis, and upscale when it goes high enough again.

We do downscale or reduce framerate on CPU overloads; note we changed from a system-CPU-LoadMonitor to just encode-time monitoring (similar to chrome) recently; I think this goes to release in 52.

You should also try with Nightly/54; we recently enabled abs-send-time and toffset rtp header extensions; those won't solve the issue of too-little bandwidth, but will improve speed and accuracy of control.

Also try with a lower resolution, to see if that changes your results (perhaps 320x240).   And a similar graph from Chrome would provide some useful comparison.

Thanks

Comment 2

[Gavin Llewellyn]

The testing was with current stable: 51.0.1.

I agree that there needs to be a minimum bitrate for a given resolution/frame rate to be usable, but rather than enforcing a minimum send rate based on the resolution, it should be altering the sending resolution based on the available bandwidth.  Some hysteresis would likely be needed, as you point out.

I will attempt the tests again with Nightly/54 and Chrome stable for comparison (hopefully tomorrow).  However, I don't think the CPU usage is likely be a factor on the hardware I am using.  The abs-send-time extension may help ensure an more responsive/accurate REMB, but the problem here is that the REMB is being ignored once it drops too low.

I expect a lower resolution would solve the problem (or at least move the problem to a lower minimum bitrate), but I consider that to be a workaround if the JS needs to get involved; congestion control is something the user agent should be managing.

Comment 3

[Gavin Llewellyn]

Chrome version: 56.0.2924.87 (current stable)
Sent video remained at 1280x720 throughout the test (according to webrtc-internals). The frame rate had dropped to about 1fps by the end of the test, and the displayed video became visibly blocky, but while not pretty the video was still functional (not frozen).  There wasn't ongoing packet loss until the throttle was lowered to 100kbps, at which point it hovered around 15% loss.

In the attached graph, the session was started with a throttle of 600kbps, 50ms delay in each direction, 250ms throttling queue.  Approximately every two minutes, the throttle was dropped by 100kbps; the session was ended after two minutes at 100kbps.

Summary: room for improvement (lowering resolution would probably be better than just the frame rate), also has a limit at which congestion signals are ignored, but able to work at much lower bandwidth than Firefox (100 vs 300kbps).

Comment 4

[Gavin Llewellyn]

Attachment: Wireshark IO graph for sent data with throttling: Chrome 56

Comment 5

[Gavin Llewellyn]

Attachment: Wireshark IO graph for sent data with throttling: Nightly 54

Firefox Nightly 54.0a1
Worked well during throttle changes at higher rates (600->500->400), where adjustments were made with little or no packet loss (presumably due to improved latency detection of congestion).  At 300kbps the send rate went up, likely due to handling lots of NACKs. Hardware throttle reporting about 33% packet loss. Video picture was clear, but was delayed by about 10 seconds, and very low frame rate; not usable.

Comment 6

[Alex Chronopoulos [:achronop]]

@jesup: Logs and info requested by reporter are available.

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

I don't see any logs.  In any case, I think we've identified the cause here - we limit to 200Kbps for VGA.  I will investigate increasing the options for scaling in low-bandwidth, which is certainly possible since we do it for over-CPU already

Comment 8

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.