Report AEC echo levels (ERLE/etc)

RESOLVED FIXED in mozilla31

Status

()

Core
WebRTC: Audio/Video
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: jesup, Assigned: jib)

Tracking

(Blocks: 2 bugs)

unspecified
mozilla31
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [p=5, 1.5:p1, ft:webrtc])

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

4 years ago
Created attachment 8393802 [details] [diff] [review]
aec_erle.patch

We should report echo levels in the AEC.  Note that calculating them can cost CPU; so we don't want to blindly enable this all the time.  We need to consider what the options are.  Also, it can take a while to get results after enabling.

Right now, I've made it depend on NSPR_LOG_MODULES=mediamanager:5 -- we likely should change this.  Currently the patch I have dumps the stats once every few second or so.

We should add it to about:webrtc (note: this will be complicated by the cost of measuring it/enabling/etc)
(Reporter)

Updated

4 years ago
Blocks: 970426
(Reporter)

Comment 1

4 years ago
I did some profiling, and on a debug build it used <0.1% of the CPU used in an audio-only call on linux ... i.e. very little, so we can likely leave it on.

On the downside: it is *very* hard to interpret ERLE in a vacuum; it depends heavily on the input signals, noise, volume, etc.  So telemetry on ERLE may be very hard to interpret/misleading.  Also, if used with telemetry we'd want to segregate it so headphone and speakerphone uses don't get mixed (we could segregate roughly based on ERL as a stand-in for knowing if it's a headset or not).

Jean-Marc - do you know any good automated metrics beyond ERL/ERLE we could use?  I'll also consult my Voice Quality for mobile book by Perry.
Flags: needinfo?(jmvalin)
I think ERLE is probably a good metric here. OTOH, ERL is completely meaningless. The problem is that turning the volume up by X dB on your external speakers while turning it down X dB in the mixer will increase the ERL by X dB even though nothing actually changed in practice. The only metric I could think of would be a signal-to-echo ratio (SER). Since the signal power is fixed, then the ERLE also happens to be the improvement in SER.
Flags: needinfo?(jmvalin)
(Reporter)

Updated

4 years ago
Assignee: nobody → jib
(Reporter)

Comment 3

4 years ago
Created attachment 8394951 [details] [diff] [review]
Monitor AEC echo levels (ERLE/etc) in gUM
(Reporter)

Updated

4 years ago
Attachment #8393802 - Attachment is obsolete: true
(Reporter)

Updated

4 years ago
Attachment #8394951 - Flags: review?(jib)
Comment on attachment 8394951 [details] [diff] [review]
Monitor AEC echo levels (ERLE/etc) in gUM

Review of attachment 8394951 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm, with the caveat that I haven't seen the patch that this patch depends on (there's no update_echo or mStarted in the tree), so bear in mind when landing.

::: content/media/webrtc/MediaEngineWebRTCAudio.cpp
@@ +589,5 @@
>  
> +#ifdef PR_LOGGING
> +  mSamples += length;
> +  if (mSamples > samplingFreq) {
> +    mSamples -= samplingFreq;

Is length ever > 2*samplingFreq? If so then you fall behind here.

Maybe a while(>) -=, an assert, or mSamples %= samplingFreq?
Attachment #8394951 - Flags: review?(jib) → review+
(Reporter)

Comment 5

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/cb894b5d342f
Target Milestone: --- → mozilla31
(Reporter)

Comment 6

3 years ago
Backed out for perma-orange on b2g emulator M10 unrelated to this patch
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=cb894b5d342f

http://hg.mozilla.org/integration/mozilla-inbound/rev/c0eaa844f2f0
https://hg.mozilla.org/mozilla-central/rev/cb894b5d342f
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 8

3 years ago
Backed out of central:
https://hg.mozilla.org/mozilla-central/rev/ac6cbaa47f34
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Reporter)

Comment 9

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/daf5df0306b2
(Reporter)

Comment 10

3 years ago
backed out:
https://hg.mozilla.org/integration/mozilla-inbound/rev/206169eef995
(Reporter)

Comment 11

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/78de1d480f29
https://hg.mozilla.org/mozilla-central/rev/78de1d480f29
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED

Updated

3 years ago
Whiteboard: [s=fx32]
Whiteboard: [s=fx32] → [p=5, 1.5:p1, ft:webrtc]
You need to log in before you can comment on or make changes to this bug.