Closed Bug 944924 Opened 6 years ago Closed 5 years ago

Web Audio API: Playback, HTML5 Audio UI instability when using AudioContext createMediaElementSource()

Categories

(Core :: Web Audio, defect, P2)

25 Branch
x86
macOS
defect

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: spam, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:27.0) Gecko/20100101 Firefox/27.0 (Beta/Release)
Build ID: 20131129004001

Steps to reproduce:

Numerous playback issues were encountered when attempting to use the Web Audio API to get visualization data from an <audio> element.

With apologies for the multi-layered bug report, demo + details follow.

Method 1: AudioContext() + createMediaElementSource() + createScriptProcessor() → audioprocess event
http://isflashdeadyet.com/tests/web-audio-visualization/

Method 2: AudioContext() + createMediaElementSource() + createAnalyser() / getByteFrequencyData() method
http://isflashdeadyet.com/tests/web-audio-visualization/index-analyser.html


Actual results:

Using both methods, Initial playback (autoplay-based) freezes at 0:01. Clicking within the <audio> UI to seek causes playback to resume (and audio data is returned), but <audio> UI shows inconsistent position behaviour. Play/pause triggers position issues as well. Clicking the volume control updates the UI, but has no effect on the audio output.


Expected results:

Audio playback should not freeze. Seeking and position should be reflected accurately in the <audio> UI and via JS properties on the object during playback, seek and pause/resume. Volume control should work.
I haven't reproduced the freezing at 0:01, but I see bad position behavior in the UI and the mute button is ineffective.

Bug 938022 should help the UI, I hope, but I don't know that that will help the mute button.
Status: UNCONFIRMED → NEW
Depends on: 938022
Ever confirmed: true
Blocks: 855568
Priority: -- → P2
Thanks for the detail.

FWIW, I'm getting the initial playback stall at both demo URLs, when using Aurora 27.0a2 (2013-12-01) on both Windows XP Pro SP3 and Mac OS X 10.9.

I don't think this issue is specific to the type of sound being played back (the demo uses MP3 or OGG depending on canPlayType() support), but I'll follow up if I find evidence suggesting otherwise.
Please retest with the next nightly now that bug 938022 has been fixed.
Playback still initially stalls, but I can confirm improved position tracking and seeking during playback on Nightly on Win7 and WinXP once playback starts by seeking to a position.
Bug 943461 will make a lot of improvements to this code. Please retest after it lands.
Depends on: 943461
The audioprocess() method appears to work better now on my demo page and channel data is shown, but there is a very brief loud "click" of digital noise when seeking on Nightly (31.0a1 / 2014-03-29) on OS X 10.9.2. Don't play this when wearing on/in-ear headphones, to be safe.

http://isflashdeadyet.com/tests/web-audio-visualization/index.html

Testing quickly with Nightly on Windows 7, the audio plays perhaps the first second and then stops. On seek, no click is heard, but playback resumes and channel data is shown.

For the analyser + getByteFrequencyData() method, the behaviour is slightly different.

http://isflashdeadyet.com/tests/web-audio-visualization/index-analyser.html

Aurora on OS X: Playback starts, channel data shown, loud click on seek. After seek, data sticks at 255, 255, 255.

Aurora on Windows 7: Playback of first second of audio with data, then playback stops. Seeking results in playback resuming, data shown, no click noise.
(In reply to Scott Schiller from comment #6)
> The audioprocess() method appears to work better now on my demo page and
> channel data is shown, but there is a very brief loud "click" of digital
> noise when seeking on Nightly (31.0a1 / 2014-03-29) on OS X 10.9.2.

Bug 944924 perhaps.
This has been fixed by all the work we've done on MediaElementSourceNode.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.