(meta) WebRTC Audio latency is too high (static latency, and increases in latency)

NEW
Assigned to

Status

()

Core
WebRTC: Audio/Video
--
critical
5 years ago
11 months ago

People

(Reporter: derf, Assigned: jesup)

Tracking

(Depends on: 2 bugs, {dogfood, meta})

Trunk
dogfood, meta
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 8 obsolete attachments)

(Reporter)

Description

5 years ago
Testing an Opus call, the audio latency is very high (multiple seconds) after less than a minute on the call. Video latency remains sub 1 second, so this is not a networking issue. I don't know if G.711 would present a similar problem.
(Reporter)

Updated

5 years ago
Blocks: 694810

Updated

5 years ago
Whiteboard: [WebRTC], [blocking-webrtc+]
(Assignee)

Comment 1

5 years ago
Assigned to padenot, but I may work on it as well.  This is likely largely related to mediastreams and the interface to them (clock drift and queuing of data in the graph).
Assignee: nobody → paul
Priority: -- → P1
I'm going to start by writing patches check where latency is added. Maybe someone here has a couple ideas.
Blocks: 834687
Created attachment 708636 [details] [diff] [review]
Add a way to get cubeb latency.

This patch adds a way to get the latency of the backend of cubeb. I've
implemented it for alsa and audiounit (macos), because I believe it is what most
of us use. I'm not too sure about the audiounit part, though.
Created attachment 708640 [details] [diff] [review]
Log latency, and adds a python script to understand the log.

This adds another log (MediaLog), and adds logging in a couple locations in the
tree. The output can be processed by the python script included, that produces a
graph. The script itself needs matplotlib.

The neteq stuff stays at 0 all the time because I probably put it at the wrong
location.
Created attachment 708655 [details]
Example output graph

This is an example of the output the script can give you. This particular graph represents a gUM session, on Linux (hence the terrible cubeb latency, because Alsa over Pulseaudio).
(Assignee)

Updated

4 years ago
Severity: normal → critical
Experimenting with dogfooding for 1:1 meetings today, QA definitely concluded this bug greatly impacts having an effective dogfooding strategy work well for 1:1 meetings.
Keywords: dogfood

Updated

4 years ago
Keywords: meta
Whiteboard: [WebRTC], [blocking-webrtc+] → [WebRTC], [blocking-webrtc-]
Duplicate of this bug: 867462
Depends on: 866675

Comment 8

4 years ago
We too have experienced a significant latency with audio/video. We have a consistently reproducible example of this. If anyone wants to see a demo of this latency effect, please let me know and I will send you a URL and instructions to view this.
(Assignee)

Comment 9

4 years ago
Repurposing explicitly as the master audio latency meta bug
Assignee: paul → rjesup
Depends on: 879213, 884365, 886886
OS: Linux → All
Hardware: x86_64 → All
Summary: Audio latency is too high → (meta) WebRTC Audio latency is too high (static latency, and increases in latency)
(Assignee)

Updated

4 years ago
Depends on: 901539, 901831

Comment 10

4 years ago
Created attachment 789084 [details] [diff] [review]
Add a way to get cubeb latency.

Updated

4 years ago
Attachment #708636 - Attachment is obsolete: true

Updated

4 years ago
Assignee: rjesup → snandaku

Comment 11

4 years ago
This version is verified on latest m-c on OSX.

(In reply to Suhas from comment #10)
> Created attachment 789084 [details] [diff] [review]
> Add a way to get cubeb latency.

Comment 12

4 years ago
Created attachment 789585 [details]
Part 1 - Add a way to get cubeb latency

Padenot's patch updated to latest m-c
Attachment #789084 - Attachment is obsolete: true

Comment 13

4 years ago
Created attachment 789586 [details]
Part -2 Exposing audio stream latency

Patch to add latency reporting in AudioStream.

Updated

4 years ago
Attachment #789585 - Attachment description: cubeb-latency-report → Part 1 - Add a way to get cubeb latency

Comment 14

4 years ago
Created attachment 789587 [details]
Part 3 - Add latency API and python script to generate graph for logs
Attachment #708640 - Attachment is obsolete: true

Comment 15

4 years ago
Created attachment 789590 [details]
Part 4 - Added the possibility to mark a location in the log

Updated

4 years ago
Attachment #789590 - Attachment is patch: true

Comment 16

4 years ago
Created attachment 789593 [details]
Part 5 - Content API to set the requested latency

Comment 17

4 years ago
Part 1 till Part 5 are Padenot's original latency patches that has been made to work  on the latest m-c. These have been tested these patches on OSX and Windows 8.1
I can generate latency graphs once i understand the use-cases that needs to be verified to help debug this issue
(Assignee)

Updated

4 years ago
Depends on: 904617

Updated

4 years ago
Attachment #789585 - Attachment is obsolete: true
Attachment #789585 - Attachment is patch: false

Updated

4 years ago
Attachment #789586 - Attachment is obsolete: true
Attachment #789586 - Attachment is patch: false

Updated

4 years ago
Attachment #789587 - Attachment is obsolete: true
Attachment #789587 - Attachment is patch: false

Updated

4 years ago
Attachment #789590 - Attachment is obsolete: true
Attachment #789590 - Attachment is patch: false

Updated

4 years ago
Attachment #789593 - Attachment is obsolete: true
Attachment #789593 - Attachment is patch: false

Comment 18

4 years ago
Moved all the attachments to 904617
Depends on: 907817
(Assignee)

Updated

4 years ago
No longer depends on: 907817
(Assignee)

Updated

4 years ago
Depends on: 908834
(Assignee)

Updated

4 years ago
Depends on: 919215
(Assignee)

Updated

4 years ago
Depends on: 920325
(Assignee)

Updated

4 years ago
Depends on: 920328
(Assignee)

Updated

4 years ago
Depends on: 920329
Assignee: snandaku → rjesup

Updated

3 years ago
Blocks: 970426
(Assignee)

Updated

3 years ago
No longer blocks: 970426

Updated

2 years ago
backlog: --- → webRTC+
Rank: 14
Whiteboard: [WebRTC], [blocking-webrtc-]
This is a meta, which typically don't prioritize.  We prioritize the dependent bugs (except at the very beginning - prior to bug breakdown).
backlog: webRTC+ → ---
Rank: 14
Priority: P1 → --
Comment hidden (spam)
tracking-b2g: backlog → ---
Do we consider this fixed? This has improved so much over the last few years that it might make sense to retire this bug and file more focused issues (this was pre-latency optimizations, and pre-MSG refactoring, and of course pre-full-duplex).
You need to log in before you can comment on or make changes to this bug.