1381850 - Investigate skipping webrtc.org audio device initialization

Status: RESOLVED WONTFIX

(Core :: WebRTC: Audio/Video, enhancement, P3)

Description

[Dan Minor [:dminor]]

We do a lot of work to initialize audio devices in webrtc.org code that we might be able to skip. For instance, on Linux:
(audio_device_impl.cc:91): Create
(audio_device_buffer.cc:73): AudioDeviceBuffer::ctor
(audio_device_impl.cc:138): AudioDeviceModuleImpl
(audio_device_impl.cc:146): CheckPlatform
(audio_device_impl.cc:163): current platform is Linux
(audio_device_impl.cc:190): CreatePlatformSpecificObjects
(audio_device_impl.cc:1901): PlatformAudioLayer
(audio_device_impl.cc:310): attempting to use the Linux PulseAudio APIs...
(audio_device_impl.cc:316): Linux PulseAudio APIs will be utilized
(audio_device_impl.cc:396): AttachAudioBuffer
(audio_device_buffer.cc:189): SetRecordingSampleRate(0)
(audio_device_buffer.cc:196): SetPlayoutSampleRate(0)
(audio_device_buffer.cc:213): SetRecordingChannels(0)
(audio_device_buffer.cc:220): SetPlayoutChannels(0)
(audio_device_impl.cc:1509): RegisterEventObserver
(audio_device_impl.cc:1522): RegisterAudioCallback
(audio_device_buffer.cc:88): RegisterAudioCallback
(audio_device_impl.cc:524): Init
(audio_device_impl.cc:1253): SetPlayoutDevice(0)
(audio_device_impl.cc:573): InitSpeaker
(audio_device_impl.cc:1348): SetRecordingDevice(0)
(audio_device_impl.cc:583): InitMicrophone
(audio_device_impl.cc:1062): StereoPlayoutIsAvailable
(audio_device_impl.cc:1072): output: 1
(audio_device_impl.cc:1081): SetStereoPlayout(1)
(audio_device_buffer.cc:220): SetPlayoutChannels(2)
(audio_device_impl.cc:948): StereoRecordingIsAvailable
(audio_device_impl.cc:958): output: 0
(audio_device_impl.cc:967): SetStereoRecording(0)
(audio_device_buffer.cc:213): SetRecordingChannels(1)

In gdb, 62 threads are started as this runs. This work may be unnecessary, we should see if we can skip it.

I had noticed something similar on Android when doing the branch 57 update, but there the impact did not seem as severe.

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

For GUM: (linux opt nightly)

MediaManager
CamerasIPC
VideoCaptureThread
inotifyEventThread (video)
voe Process thread
AudioDeviceBufferTimer TaskQueue thread
pulseaudio threaded-ml thread
ptrThreadRec thread (ADM)
ptrThreadPlay (ADM)
threaded-ml
<give permission>
cubebOperation#1
LatencyLogger thread (we should either fix this, or consider removing it)
MediaStreamGrph
CubebOperation#2
<audio runs>
<stop>
CubebOperation#3

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

one-way audio-only call in pc_test, no identity

call_worker_queue
module_process__thread
ProcessThread
SubtleCrypto#1 (DTLS?)
call_worker_queue (probably receiver)
ProcessThread (receiver)
ProcessThread (?)
WorkerThread (?)
ProcessThread (EnumerateAudioDevices)
AudioDeviceBufferTimer
threaded-ml
ptrThreadRec
ptrThreadPlay
<allow audio>
CubebOperation#4
MediaStreamGrph
CubebOperation#5
ProcessThread (AudioConduit)
AudioDeviceBufferTimer
threaded-ml
ptrThreadRec (receiver)
ptrThreadPlay
ProcessThread (receiver)
AudioDeviceBufferTimer
threaded-ml
ptrThreadRec (?)
ptrThreadPlay
AudioProxy#1 (MediaPipeline)
threaded-ml

So, I don't see near 62 (even though a couple might be skipped because I called GUM before).
And this has extra threads because there both ends of the one-way call.

We could use a timer to try to avoid some re-inits of the VoiceEngine used for enumeration, or try to re-use that for capture, at the cost of some extra complexity.  (We don't want to leave it initialized while we're idle, since it keeps a timer-tickled thread running which can chew battery.)

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

Enumeration initializes the entire voice engine currently; if that could be skipped we'd save some.  Not a ton.

Comment 4

[Paul Adenot (:padenot)]

not 62 resident, 62 that are being created and probably stopped.

Comment 5

[Dan Minor [:dminor]]

I set _platformAudioLayer to kDummyAudio here [1] and was able to make a successful appear.in call. In the log, I can see the choice to use the dummy audio APIs, so I think this is working as expected. I'm going to investigate further and see if using the dummy APIs saves us anything.

[1] http://searchfox.org/mozilla-central/source/media/webrtc/trunk/webrtc/modules/audio_device/audio_device_impl.cc#133

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Paul Adenot (:padenot) from comment #4)
> not 62 resident, 62 that are being created and probably stopped.

That was with a breakpoint on pthread_create, so it should have caught every thread creation.  Note I didn't list StreamTrans#N, dns, etc threads that aren't anything directly to do with us; I'd guess there were at least 10 of them.  I broke on each create, and then used 'up' until I found what it was.

Comment 7

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.

Comment 8

[Dan Minor [:dminor]]

We no longer use the webrtc.org audio devices.