1329352 - High video latency observed during a WebRTC session

Status: RESOLVED INCOMPLETE

(Core :: WebRTC: Audio/Video, defect, P3)

Description

[rengana]

Attachment: ff_jitter_buffer.patch

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:53.0) Gecko/20100101 Firefox/53.0
Build ID: 20170104030214

Steps to reproduce:

1) Connect to AWS Workspace instance using Firefox,
url: https://clients.amazonworkspaces.com/webclient. Please advice me on your best practice to provide you the login credentials for accessing the Workspace instance
2) Monitor latency over FF webrtc internals tab
3) Log into the workspaces system to access your desktop session. Open a notepad or browser (or any application for that matter)
4) you can see latency > 2 seconds
5) Leave the session idle for 5-10 minutes
6) move the notepad window or maximize and minimize the browser window (done to generate network traffic)
7) you can observe the difficulty in doing actions mentioned in step (6)
8) Latency observed were > 12 seconds
9) after 2-3 minutes, latency climbs down to less than 100 ms


Actual results:

Video latency at steps 4 and 8 were high for over a 2-3 minutes time period causing a laggy desktop experience


Expected results:

smooth remote desktop experience. Note: Chrome does not show this abnormal video latency. Remote desktop experience is smooth on chrome

Initial investigation (under good network conditions)
Start of the session: 
During the start of the session, Jitter buffer takes some time to adjust to the I frame burst of data. At times > 2 seconds of sustained latency is noted
Middle of the session: 
Keep the remote desktop idle for 5-10 minutes. Then start to move a notepad window on the remote desktop, sender sends an I frame to FF client by now. FF jitter buffer not able to handle this sudden burst of I frame data, also it starts to send repetitive PLIs to the sender. When the sender honors the PLIs by sending a new frame it further aggravates FF jitter buffer adding up to the video latency. note: Sender has a 300 ms protection from repetitive PLIs from receiver
Jitter buffer experiments:
When PLI was removed from the SDPs, there is a relative improvement in the video latency. Also, video latency improved when jitter buffer is allowed to grow without any restrictions. Have attached the patch for not restricting jitter buffer based on timestamps. Chromium’s jitter buffer is adaptive to sudden burst of video data

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

I've seen some instances of the repetitive NACK/PLIs, but haven't been able to track down the source.  If this is reproducible, I should be able to repro it under 'rr' (or at least dump detailed MOZ_LOG=webrtc_trace:65535 WEBRTC_TRACE_FILE=nspr logs), and track the source of the problem.

Chrome and Firefox actually use the same jitter buffer implementation (though ours is an older version).

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

Comment on attachment 8824587 [details] [diff] [review]
ff_jitter_buffer.patch

Review of attachment 8824587 [details] [diff] [review]:
-----------------------------------------------------------------

This is not a patch for landing, it's a debug patch

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

I've been trying to reproduce this working with the reporter.  So far I've been unable to reproduce from my location.

Comment 4

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

This is an assigned P1 bug without activity in two weeks. 

If you intend to continue working on this bug for the current release/iteration/sprint, remove the 'stale-bug' keyword.

Otherwise we'll reset the priority of the bug back to '--' on Monday, August 28th.

Comment 5

[Bulk Bug Changes for mreavy's org]

Mass change P1->P2 to align with new Mozilla triage process

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Moving to p3 because no activity for at least 1 year(s).
See https://github.com/mozilla/bug-handling/blob/master/policy/triage-bugzilla.md#how-do-you-triage for more information