1143673 - GetUserMedia delay, loopback stops happening

Status: NEW

(Core :: WebRTC: Audio/Video, defect, P3)

Description

[c.schoene]

Attachment: loopback.html

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Firefox/36.0 Iceweasel/36.0.1
Build ID: 20150309005701

Steps to reproduce:

This little loopback audio demo connects the microphone to a ScriptProcessor which is connected to the speaker. See attached file loopback.html



Actual results:

The demo runs as aspected, delay depends on buffer size.
However, over time the delay gets bigger and after about 15 - 20 min it stops working completely. No errors or warning are thrown.

Chrome doesn"t show this behaviour, it runs the loopback forever (only tested a few hours).

I tested on Linux and Windows 7, disabling noise and aec didn't show any difference.


Expected results:

Loopback with (kinda) constant delay.

Comment 1

[c.schoene]

Just to clarify:
Firefox doesn't crash, it just isn't doing the audio loopback anymore.
After a page reload it will work for another 15 or so minutes.

Comment 2

[[:Aleksej]]

I haven't noticed the delay growing, it just silenced in less than 15 minutes.

> I tested on Linux and Windows 7,

Does the bug also happen on Windows 7?

Comment 3

[[:Aleksej]]

(In reply to [:Aleksej] from comment #2)
> I haven't noticed the delay growing, it just silenced in less than 15
> minutes.

That's with 2015-03-22-03-02-16-mozilla-central-firefox-39.0a1.ru.linux-x86_64

Comment 4

[c.schoene]

Yes, I mainly tested Windows 7 32bit on a laptop running Firefox 36.
The delay has been about 0.16 secs at the beginning. After 5 minutes it's been 0.25 secs.

I'd like to know if this delay is further increasing over time, that's primarily the reason I made this test for. In an application I have this growing to 2-3 seconds after 30 minutes. Unfortunately the test doesn't run that long.

Comment 5

[Paul Adenot (:padenot)]

This is likely to be a gUM bug, in which case, WebRTC: Video/Audio is a better fit.

Comment 6

[c.schoene]

With Nightly (41) loopback is working now correctly.
I tested several hours with Linux (64bit) and Windows 7 (32bit).

However, I still see the delay built-up.

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

For local gUM streams, if there is a drift between the microphone and the speakers on your system (with the mic running 'faster'), then delay can build up locally.  This will not happen to data sent over a peerconnection - the receiving side of a peerconnection reclocks audio to deal with drift, since two systems will never have identical clocks.

We do eventually plan to a) switch to full-duplex audio capture/playback, which will in most cases remove the input/output drift (via the OS), and b) implement a drift-compensating resampler for cases where the input isn't locked to the output (and the data is being processed via the MediaStreamGraph or <video>/<audio> elements.

Comment 8

[c.schoene]

Hm, ok..

In my case, I need to synchronize audio with a button (push to talk) and record audio samples which are fed to a voice recognizer. If this is not in sync some syllables or even words are missing and the recognizer runs into problems (miss recognitions).

Currently after about 30 minutes the delay is pretty big and the recorded audio isn't from between the button pressed and depressed events anymore (usually off a second or 2).
I would appreciate some sort of local drift compensation here.

Comment 9

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.