Closed Bug 591976 (microphone) Opened 10 years ago Closed 5 years ago

Expose raw audio from the microphone to web content

Categories

(Core :: WebRTC, enhancement)

enhancement
Not set

Tracking

()

RESOLVED FIXED

People

(Reporter: humph, Unassigned)

Details

Now that we have an audio data API, it would be great to be able to work with audio coming out of the mic on the local machine.  I've seen some reference to this idea on lists, but could find no bug.

In addition to being able to stream the audio from the mic into the browser to be played, I want to be able to manipulate it, send it to remote boxes, store it locally, whatever.  If we can get the raw audio data to the media element's audio data API, this is pretty easy now.

From talking with khuey on irc,

< khuey|class> we can just plug into the same framework we get camera data from
< khuey|class> and then you can do "moz-device:mic" to get a stream

Hopefully this bug can serve as a starting point for discussion on doing this after ff4 timing.
Security is the tricky bit here.  For the camera work we're using a <video> with src="moz-device:camera" but that's not exposed to content directly.
Flash does something similar to what we are doing for our geolocation stuff, namely, prompting for access.
Would it make sense to make this block ‘webapi’ and become part of the WebAPI work?
Yes and no. It's device access, and so falls under the WebAPI umbrella, so blocking 'webapi' makes sense.

It's also a prerequisite for the MediaStream API proposal which is part of the larger Audio API and WebRTC/Conferencing work going on. There's quite a bit of overlap with projects outside the broader mission of the WebAPI group, as I understand it.
A MediaStream looks like the way to go IMHO, like in this audio recorder example from:

http://dev.w3.org/2011/webrtc/editor/webrtc-20111004.html#examples-1

Or example 4 from:

https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/webrtc-integration.html

Anyways, I am not sure if I am on the correct mozilla bug or not, but I am trying to implement an audio recorder app for gaia/b2g/firefoxOS and trying to figure out the current blockers, looks like it would be possible to create such an app once the different borwsers implement/fix some webrtc bugs.

I am gathering hints and references here for now https://kippt.com/fabricio/audio-recorder-webapp-blockers
Component: Audio/Video → WebRTC
Covered by the getUserMedia API
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.