944924 - Web Audio API: Playback, HTML5 Audio UI instability when using AudioContext createMediaElementSource()

Status: RESOLVED WORKSFORME

(Core :: Web Audio, defect, P2)

Description

[Scott Schiller]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:27.0) Gecko/20100101 Firefox/27.0 (Beta/Release)
Build ID: 20131129004001

Steps to reproduce:

Numerous playback issues were encountered when attempting to use the Web Audio API to get visualization data from an <audio> element.

With apologies for the multi-layered bug report, demo + details follow.

Method 1: AudioContext() + createMediaElementSource() + createScriptProcessor() → audioprocess event
http://isflashdeadyet.com/tests/web-audio-visualization/

Method 2: AudioContext() + createMediaElementSource() + createAnalyser() / getByteFrequencyData() method
http://isflashdeadyet.com/tests/web-audio-visualization/index-analyser.html


Actual results:

Using both methods, Initial playback (autoplay-based) freezes at 0:01. Clicking within the <audio> UI to seek causes playback to resume (and audio data is returned), but <audio> UI shows inconsistent position behaviour. Play/pause triggers position issues as well. Clicking the volume control updates the UI, but has no effect on the audio output.


Expected results:

Audio playback should not freeze. Seeking and position should be reflected accurately in the <audio> UI and via JS properties on the object during playback, seek and pause/resume. Volume control should work.

Comment 1

[Karl Tomlinson (:karlt)]

I haven't reproduced the freezing at 0:01, but I see bad position behavior in the UI and the mute button is ineffective.

Bug 938022 should help the UI, I hope, but I don't know that that will help the mute button.

Comment 2

[Scott Schiller]

Thanks for the detail.

FWIW, I'm getting the initial playback stall at both demo URLs, when using Aurora 27.0a2 (2013-12-01) on both Windows XP Pro SP3 and Mac OS X 10.9.

I don't think this issue is specific to the type of sound being played back (the demo uses MP3 or OGG depending on canPlayType() support), but I'll follow up if I find evidence suggesting otherwise.

Comment 3

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Please retest with the next nightly now that bug 938022 has been fixed.

Comment 4

[Scott Schiller]

Playback still initially stalls, but I can confirm improved position tracking and seeking during playback on Nightly on Win7 and WinXP once playback starts by seeking to a position.

Comment 5

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Bug 943461 will make a lot of improvements to this code. Please retest after it lands.

Comment 6

[Scott Schiller]

The audioprocess() method appears to work better now on my demo page and channel data is shown, but there is a very brief loud "click" of digital noise when seeking on Nightly (31.0a1 / 2014-03-29) on OS X 10.9.2. Don't play this when wearing on/in-ear headphones, to be safe.

http://isflashdeadyet.com/tests/web-audio-visualization/index.html

Testing quickly with Nightly on Windows 7, the audio plays perhaps the first second and then stops. On seek, no click is heard, but playback resumes and channel data is shown.

For the analyser + getByteFrequencyData() method, the behaviour is slightly different.

http://isflashdeadyet.com/tests/web-audio-visualization/index-analyser.html

Aurora on OS X: Playback starts, channel data shown, loud click on seek. After seek, data sticks at 255, 255, 255.

Aurora on Windows 7: Playback of first second of audio with data, then playback stops. Seeking results in playback resuming, data shown, no click noise.

Comment 7

[Karl Tomlinson (:karlt)]

(In reply to Scott Schiller from comment #6)
> The audioprocess() method appears to work better now on my demo page and
> channel data is shown, but there is a very brief loud "click" of digital
> noise when seeking on Nightly (31.0a1 / 2014-03-29) on OS X 10.9.2.

Bug 944924 perhaps.

Comment 8

[Paul Adenot (:padenot)]

This has been fixed by all the work we've done on MediaElementSourceNode.