Closed Bug 937718 Opened 6 years ago Closed 5 years ago

Silent output with cross-origin createMediaElementSource and Access-Control-Allow-Origin: *

Categories

(Core :: Web Audio, defect, P2)

28 Branch
x86
All
defect

Tracking

()

RESOLVED FIXED
mozilla37

People

(Reporter: ian.kerr, Assigned: padenot)

References

Details

Attachments

(5 files, 5 obsolete files)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0 (Beta/Release)
Build ID: 20131112030204

Steps to reproduce:

Run the following JSFiddle, which uses Web Audio's createMediaElementSource() to route audio from an <audio> tag into a Web Audio context:

http://jsfiddle.net/ikerr/WcXHK/

To see debug output, open the Web Console (Tools > Web Developer > Web Console).  Note that the fiddle checks the audio context's sample rate and selects a matching 44100 Hz or 48000 Hz audio file (I haven't seen any other sample rates in my testing).

I tested this fiddle in Firefox 25, 27 (Aurora) and 28.0a1 (2013-11-12) on Mac OS X 10.9.  My system specs are as follows:

MacBook Pro, 15-inch, Mid 2012
Processor: 2.3 GHz Intel Core i7
Memory: 8 GB 1600 MHz DDR3
Graphics: NVIDIA GeForce GT 650M 512 MB


Actual results:

When this fiddle loads, I do not hear any music playing (i.e., silence).  When I run the fiddle, I get the following output in my Web Console (no errors):

10:02:28.005 "Context sample rate: 44100"
10:02:28.006 "Using audio asset: http://assets-staging.verold.com/entities/52812b86a9aff5020000000b/3f6b2c8c8996e748a753cf26c6f9406a/06+Old+Man-44100.mp3"
10:02:28.006 "Using audio asset: http://assets-staging.verold.com/entities/52812b86a9aff5020000000b/3f6b2c8c8996e748a753cf26c6f9406a/06+Old+Man-44100.ogg"
10:02:28.006 "Created MediaElementAudioSourceNode."


Expected results:

When this fiddle loads, music should start playing.  This works in the Chrome 30.0.1599.101, for example.  Documentation for createMediaElementSource() can be found here:

https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#MediaElementAudioSourceNode-section
Component: Untriaged → Web Audio
Product: Firefox → Core
Attached file audio element testcase (obsolete) —
This plays as expected.
This is reduced from http://jsfiddle.net/ikerr/WcXHK/
It doesn't produce sound.
Sorry.  This version, still without createMediaElementSource plays.
Attachment #831072 - Attachment is obsolete: true
Attached file reduced testcase
Assignee: nobody → karlt
Attachment #831070 - Attachment is obsolete: true
Attachment #831073 - Attachment is obsolete: true
Status: UNCONFIRMED → NEW
Ever confirmed: true
Blocks: 855568
OS: Mac OS X → All
Summary: Silent output with createMediaElementSource → Silent output with cross-origin createMediaElementSource and Access-Control-Allow-Origin: *
Priority: -- → P2
I am experience the same problem. jsfiddle that fails: http://jsfiddle.net/7bJUU/ 
In the above example, the ogg file from wikipedia is returned with "Access-Control-Allow-Origin: *" but the
output is silent. The same example works if 1) createMediaElementSource is not used and file is played directly
2) when a local, same origin file is played
Additional test case: http://jsfiddle.net/xqhWY/2/

When the createMediaElementSource is created the output of the audio stops. The source is connected to the destination of the context, so I would expect output.
I think I found the problem. -> http://jsfiddle.net/xqhWY/4/

The sample rate of the Audio Context is different then the sample rate of the audio stream. It appears in the Web Audio API spec that the AudioContext determines which sample rate to use <http://www.w3.org/TR/webaudio/#Sample-rate> and it assumes that other nodes will use the same rate <http://www.w3.org/TR/webaudio/#dfn-sampleRate>. This would explain the unexpected behaviour I'm seeing from Firefox when I attempt to create a media source from the audio element.
In the original bug report, I observed the problem even when the sample rate of the Audio Context matched the file's sample rate.  But I agree that when the sample rates do not match, the output should be silent.

(In reply to rbarber from comment #10)
> I think I found the problem. -> http://jsfiddle.net/xqhWY/4/
> 
> The sample rate of the Audio Context is different then the sample rate of
> the audio stream. It appears in the Web Audio API spec that the AudioContext
> determines which sample rate to use
> <http://www.w3.org/TR/webaudio/#Sample-rate> and it assumes that other nodes
> will use the same rate <http://www.w3.org/TR/webaudio/#dfn-sampleRate>. This
> would explain the unexpected behaviour I'm seeing from Firefox when I
> attempt to create a media source from the audio element.
See Also: → 996685
Duplicate of this bug: 1054120
I can replicate this bug, with video elements as well.

However, it's not that the audio is silent, but rather that it doesn't play at all. Here's a modified version of the jsfiddle that displays the audio element with controls. You'll see that the timeline does not progress even though it is "playing" ... or at least, not paused.

http://jsfiddle.net/WcXHK/37/
sorry i dupe'd the bug again .. did not see this in my activity area till today :)
ps.. my context sample returns 'undefined'
Duplicate of this bug: 1060150
So, this is kind of what <canvas> does for drawImage with cross-origin I think.
I can see audio frames being appended to the MediaStream in
MediaDecoderStateMachine::SendStreamData, but then something else is broken and
it does not work, so a bit more work is needed, but I'd like to validate the
approach here.
Attachment #8537911 - Flags: review?(roc)
Assignee: karlt → padenot
Status: NEW → ASSIGNED
This is the rest of the work, it works, now. I'll r? when the mochitest is
ready, it kind of works but it too flaky.
The silence was caused by the fact that the track was not enabled, because at
the MediaStream level, there was another origin check.

By teaching the existence of CORS to DOMMediaStream, and wiring that down to
MediaStreamAudioSourceNode::PrincipalChanged, we can track the cross-origin
status of the media that gets fed to the MSG, and mute/unmute accordingly.
Attachment #8541235 - Flags: review?(roc)
Attachment #8540254 - Attachment is obsolete: true
And the test, using an .sjs file to add the header. Green on try.
Attachment #8541236 - Flags: review?(roc)
Attachment #8537911 - Attachment is obsolete: true
https://hg.mozilla.org/mozilla-central/rev/0c9f97ebf7ea
https://hg.mozilla.org/mozilla-central/rev/bcbc64d97fbf
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla37
Duplicate of this bug: 1111382
This still isn't working for me. I'm trying it on Nightly 38.0a1 (2015-02-09) on OSX 10.9.5. I tried in both an e10s and Non-e10s window with no success in either.

Can we reopen this?
(In reply to brian from comment #25)
> This still isn't working for me. I'm trying it on Nightly 38.0a1
> (2015-02-09) on OSX 10.9.5. I tried in both an e10s and Non-e10s window with
> no success in either.
> 
> Can we reopen this?

Open a new bug, please. Bonus points if you have a test case!
The example from bug 1111382 still doesn't work in the nightly: http://www.jlescure.com/samples/firefox-bug.html
(I was using 38.0a1 (2015-02-19) on OSX 10.10.1)
This example from further up in the bug does not work either: http://jsfiddle.net/7bJUU/
i can confirm that this still wont work. even though soundcloud has access control allow origin on each of its mp3s, output is still silent in released firefox 37.0.1.
update: it just skips a bit at the start, but will play if audio tag has crossorigin = 'anonymous' property!
disregard previous comment. in the jlescure.com example you should add audio.crossOrigin = 'anonymous'; and it also works :) thanks for fixing this!
You need to log in before you can comment on or make changes to this bug.