937962 - sound from MediaElementAudioSourceNode is a distorted buzz sometimes

Status: RESOLVED FIXED

(Core :: Web Audio, defect, P1)

Description

[Karl Tomlinson (:karlt)]

I thought this had been resolved (see bug 932400 comment 5), but I still sometimes get this on http://www.scirra.com/labs/audioeffects/ without reliable steps to reproduce.  I'll attach a testcase that does reproduce reliably.

Bug 932400 comment 2 suggest removing createMediaElementSource until we have this working reliably.

Comment 1

[Karl Tomlinson (:karlt)]

Attachment: media-element-source-1.html

1. load any page
2. load this attachment and hear sound
3. press back
4. press forward
5. if sound is not awful, repeat from 3.

Comment 2

[Karl Tomlinson (:karlt)]

Attachment: ting-44.1k-1ch.ogg

Comment 3

[Karl Tomlinson (:karlt)]

Attachment: testcase

See comment 1 for STR.

Comment 4

[Karl Tomlinson (:karlt)]

The testcase fails many of these assertions even when the sound is playing as expected:

###!!! ASSERTION: Samples missing: 'startTicks == TimeToTicksRoundUp(inputTrackRate, outputStart)', file /home/karl/moz/dev/content/media/AudioNodeExternalInputStream.cpp, line 379

Comment 5

[ashley]

Still reproduces on Firefox 26 (stable) and Nightly (29.0a1) on Windows 8.1 x64.

In particular:
http://www.scirra.com/labs/audioeffects/

1. Play SFX1
2. Play music (uses MediaElementAudioSourceNode)
3. Play SFX1 (note music now cuts out, it should keep playing)
4. Play music again (starts playing again but extremely distorted)
5. Play SFX1 (all audio cuts out, horrible buzzing)

I'd recommend not to test this on headphones. If they're up loud, step 5 could be painful.

Comment 6

[ashley]

Note we've disabled music playback via MediaElementAudioSourceNode in the Construct 2 engine until this can be fixed.

Comment 7

[Paul Adenot (:padenot)]

(In reply to ashley from comment #6)
> Note we've disabled music playback via MediaElementAudioSourceNode in the
> Construct 2 engine until this can be fixed.

Can I get a sane testcase (not mangled), or at least know what code is executed (when are the node disconnected, stopped, the media element paused, etc.) ? I've stepped through the code, but it is super painful to read.

Comment 8

[ashley]

Clicking SFX1 just creates a buffer source and starts it. Clicking 'Play music' creates a MediaElementAudioSourceNode (only on the first click) and calls play() on the underlying HTMLAudioElement.

The code's messy even unminified since we switch between 4 different audio APIs for our engine. Is that enough info or do you need a minimal test case still?

Comment 9

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Retest when 943461 is fixed.

Comment 10

[Maire Reavy [:mreavy]]

Moving to Paul per our meeting yesterday.

Comment 11

[ashley]

Still reproduces in Nightly 29.0a1 (2014-01-08). Any progress on this? It's causing us compatibility headaches since our workaround makes Chrome and Firefox work differently in some cases.

Comment 12

[Paul Adenot (:padenot)]

(In reply to ashley from comment #11)
> Still reproduces in Nightly 29.0a1 (2014-01-08). Any progress on this? It's
> causing us compatibility headaches since our workaround makes Chrome and
> Firefox work differently in some cases.

So, I added debug statements directly in gecko because I could not make sense of the code. It appears that your code uses createMediaElementSource on the media element  every time you press "Play Music", and also seeks the media element to 0.0. This adds a new copy of the media element to the mix, and it understandably saturates after some time (because, of course, the sound is perfectly in phase). You should probably disconnect the first MediaElementSourceNode when playing the new one if you don't want this to happen.

Blink/Webkit don't have this behaviour, but I'm pretty sure they are wrong. At best this is unspecified, and I'm going to propose to fix the spec to match our behaviour, because you limit the possible use cases otherwise (think multiple processing out of the same HTMLMediaElement, using Web Audio).

Comment 13

[Paul Adenot (:padenot)]

(In reply to :Karl Tomlinson (back Jan 28) from comment #4)
> The testcase fails many of these assertions even when the sound is playing
> as expected:
> 
> ###!!! ASSERTION: Samples missing: 'startTicks ==
> TimeToTicksRoundUp(inputTrackRate, outputStart)', file
> /home/karl/moz/dev/content/media/AudioNodeExternalInputStream.cpp, line 379

I'll now focus on this.

Comment 14

[Paul Adenot (:padenot)]

This happens when seeking (either manually, or because of the loop attribute). Not sure why yet, but it sounds fine anyways.

Comment 15

[Maire Reavy [:mreavy]]

Paul -- How far along are you on this?  If you're not too far along, I'd like to reassign this so you can start bug 848954.  Thanks!

Comment 16

[Paul Adenot (:padenot)]

Comment 12, 13, 14 sums it up:
- This looks like a bug in Chrome ;
- I see assert in our code, but the audio output seems correct to me nonetheless.


 I could not repro comment 0, though.

Comment 17

[Karl Tomlinson (:karlt)]

With attachment 831235 [details] and STR in comment 1, I no longer hear the awful buzz, and things work as expected more often, but occasionally the sound doesn't resume on returning to the test page.  When that happens, repeating back and forward doesn't make sound resume; a reload makes the sound play again.

Comment 18

[Karl Tomlinson (:karlt)]

The "Samples missing" assertion stops failing when the sound does not resume.

Comment 19

[Paul Adenot (:padenot)]

(In reply to Karl Tomlinson (:karlt) from comment #17)
> With attachment 831235 [details] and STR in comment 1, I no longer hear the
> awful buzz, and things work as expected more often, but occasionally the
> sound doesn't resume on returning to the test page.  When that happens,
> repeating back and forward doesn't make sound resume; a reload makes the
> sound play again.

I think you're seeing bug 960057.

Comment 20

[Maire Reavy [:mreavy]]

Karl -- I know you have several P1s on your plate, but I think you're likely to get to this before Paul finishes with the MSG refactor.

Comment 21

[Karl Tomlinson (:karlt)]

Comment 17 still reproduces in today's build, with bug 960057 fixed.

Comment 22

[ashley]

I still have audio quality issues as mentioned in comment 1 in Firefox Nightly 32.0a1.

Comment 23

[ashley]

Sorry, meant to reference comment 0.

Comment 24

[Karl Tomlinson (:karlt)]

(In reply to ashley from comment #23)
> Sorry, meant to reference comment 0.

Comment 0 doesn't have steps to reproduce and http://www.scirra.com/labs/audioeffects/ was playing multiple MediaElementAudioSourceNodes concurrently, which was confusing things.

Are you able reproduce distortion reliably?
Would you be able to file a new bug with steps to reproduce, please?

Comment 25

[Paul Adenot (:padenot)]

This is fixed. The difference between Chrome and Firefox is because Chome is not spec compliant and does not add a second MediaElementAudioSourceNode to the output when creating it a second time.