1393689 - audio delay 3-4 seconds after 15 hours (and other faster cubeb audio drift issues)

Status: RESOLVED FIXED

(Core :: Audio/Video: cubeb, defect, P2)

Description

[xpeng]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:55.0) Gecko/20100101 Firefox/55.0
Build ID: 20170814072924

Steps to reproduce:

case 1.we use the demo https://webrtc.github.io/samples/src/content/getusermedia/audio/ 
 to play local audio  to test the audio delay.

case 2: we use the demo https://webrtc.github.io/samples/src/content/peerconnection/audio/  to play local audio  to test the audio delay.



Actual results:

1.we find that the audio delay 3 seconds after 15 hours in case 1
2.we find that the audio delay 4 seconds after 15 hours in case 2

Comment 1

[Paul Adenot (:padenot)]

xpeng, are you on windows when doing those tests? I've written a patch for windows yesterday that should make things better.

Comment 2

[xpeng]

(In reply to Paul Adenot (:padenot) from comment #1)
> xpeng, are you on windows when doing those tests? I've written a patch for
> windows yesterday that should make things better.

yes , we test it on windows, how can i get the patch for testing?

Comment 3

[Paul Adenot (:padenot)]

It'll be in Firefox Nightly soon, I'll let you know in this bug.

Comment 4

[xpeng]

(In reply to Paul Adenot (:padenot) from comment #3)
> It'll be in Firefox Nightly soon, I'll let you know in this bug.

OK We will test it as soon as possible while the patch is ready.

Comment 5

[xpeng]

Can we test it using the current firefox nightly ?

Comment 6

[Paul Adenot (:padenot)]

Nope, sorry, not yet. I'll let you know.

Comment 7

[Maire Reavy [:mreavy]]

Likely this will be duped, but prioritizing it separately for now.

Comment 8

[Bulk Bug Changes for mreavy's org]

Mass change P1->P2 to align with new Mozilla triage process

Comment 9

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

We've had a couple of bug reports on audio drift. We're picking this one to track work Paul is doing to avoid long-term drift in cubeb here:

 issue https://github.com/kinetiknz/cubeb/issues/353
 pr is https://github.com/kinetiknz/cubeb/issues/354

Comment 11

[Randell Jesup [:jesup] (needinfo me)]

Mirroring comment I made on github, about the "right" way to really solve this.  Likely not that surprising...

FYI, from chatting on Slack:
The "right" way to do it is to a) resample to match rates, b) keep enough buffer to avoid underruns, but no more. This obviously can fail if the environment/load changes, which is why it has to be adaptive (basically it has to be a form of jitter buffer). In most cases, it should run at "no extra buffering" (i.e. we haven't seen underruns) which means max one capture "frame" (typ 10ms or less - and if you're not having to resample (no drift), it should be 0 in that case. When the buffer starts to grow, you slew the resample rate to stop the growth and get it back near your target depth. If you see underruns, you glitch, and you slew the resample rate to increase the amount of buffering (and bump the target for buffering probably).

This is classic jitter-buffer design. For "extra" fancy, you can minimize pitch warbles when slewing (and slew faster) by using some of the magic in NetEq to add or remove cycles of the dominant frequency (TDC - Time Distortion Compensation IIRC)

IMHO

Comment 12

[Paul Adenot (:padenot)]

xpeng, this is now in Nightly, please have a look, and tell us what you think! Thanks!

Comment 13

[xpeng]

(In reply to Paul Adenot (:padenot) from comment #12)
> xpeng, this is now in Nightly, please have a look, and tell us what you
> think! Thanks!

OK , we will test it soon.

Comment 14

[xpeng]

we have checked it with demo below for 21 hours, It works fine:
https://webrtc.github.io/samples/src/content/peerconnection/audio/ 
https://webrtc.github.io/samples/src/content/getusermedia/audio/

Comment 15

[Paul Adenot (:padenot)]

xpeng, I'm marking this one as fixed. Thanks for reporting and testing the fix!

Comment 16

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Updated tracking flag to match bug 1405258 comment 11.