855568 - Implement MediaElementAudioSourceNode

Status: RESOLVED FIXED

(Core :: Web Audio, defect)

Description

[Matthew Gregan [:kinetik]]

MediaElementAudioSourceNode is defined here: <https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#MediaElementAudioSourceNode>.

Comment 1

[(no longer active)]

Matthew, did you want to work on this?  :-)

FWIW I think we should first implement MediaStreamAudioSourceNode (bug 856361) and then use mozCaptureStream to implement this node type.

Comment 2

[Matthew Gregan [:kinetik]]

I don't think I've got time right now sorry, I just wanted to file it for the bug 855570 deps.

Comment 3

[(no longer active)]

Mass moving Web Audio bugs to the Web Audio component.  Filter on duckityduck.

Comment 4

[(no longer active)]

Demo: http://people.mozilla.org/~eakhgari/webaudio/mediaelement-webaudio.html

Comment 5

[(no longer active)]

Attachment: WIP

Comment 6

[(no longer active)]

Attachment: WIP

Added a test, it doesn't pass and it crashes.  But Bobby wanted to see it.

Comment 7

[Robert O'Callahan (:roc) (email my personal email if necessary)]

The test originally attached to this patch won't work since playback of the media element may introduce silence (especially at the start) while the media element decoding pipeline spins up.

Comment 8

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Attachment: patch

Basically you already had, which I've reviewed/tweaked a bit, plus a new test which is just test_mediaStreamAudioSourceNodeResampling.html but using MediaElementAudioSourceNode instead of MediaStreamAudioSourceNode. Nice and simple implementation :-).

Comment 9

[(no longer active)]

Comment on attachment 780774 [details] [diff] [review]
patch

Review of attachment 780774 [details] [diff] [review]:
-----------------------------------------------------------------

It would also be really great if you could add a test similar to test_mediaStreamAudioSourceNode.html.  r=me with that.

Comment 10

[Robert O'Callahan (:roc) (email my personal email if necessary)]

You mean where the track sample rate is equal to the context sample rate? I don't think that's really necessary given this is a very thin wrapper around MediaStreamAudioSourceNode.

Comment 11

[(no longer active)]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #10)
> You mean where the track sample rate is equal to the context sample rate? I

Yes.

> don't think that's really necessary given this is a very thin wrapper around
> MediaStreamAudioSourceNode.

Hmm that's a fair point given our knowledge about the implementation, but I'd still like us to test the API surface there.  If you want, I'm fine with us doing this in another bug though.

Comment 12

[Robert O'Callahan (:roc) (email my personal email if necessary)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/911a11878522

Comment 13

[Ed Morley [:emorley]]

Push backed out for mochitest-1 failures:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=f7496fddb076

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/6946b8d86502
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/31453209a88a
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/c43900741e9a
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/3f150133a4de
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/ebd005b9e9d7
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/6e72a4e894e1
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/0f30f29ffa1f
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/fab0e9b04d8d
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/4382f83efcaf
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/47f46080685f

Comment 14

[Robert O'Callahan (:roc) (email my personal email if necessary)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/27f384522675

Comment 15

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/27f384522675

Comment 16

[(no longer active)]

Comment on attachment 780774 [details] [diff] [review]
patch

This is needed for Web Audio in Firefox 25.

Comment 17

[(no longer active)]

checkin-needed for Aurora, a=webaudio.

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/c0397331a1b7

Comment 19

[Ryan VanderMeulen [:RyanVM]]

Backed out for mochitest failures.
https://hg.mozilla.org/releases/mozilla-aurora/rev/918a847c0018

https://tbpl.mozilla.org/php/getParsedLog.php?id=26380446&tree=Mozilla-Aurora

Comment 20

[Robert O'Callahan (:roc) (email my personal email if necessary)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/64b3c05e6d58

Comment 21

[u279076]

Assuming this doesn't need QA.

Comment 22

[Jean-Yves Perrier [:teoli]]

Doc is up-to-date: https://developer.mozilla.org/en-US/docs/Web/API/MediaElementAudioSourceNode