If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Silent output with cross-origin createMediaElementSource and Access-Control-Allow-Origin: *

RESOLVED FIXED in mozilla37

Status

()

Core
Web Audio
P2
normal
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: Ian Kerr, Assigned: padenot)

Tracking

28 Branch
mozilla37
x86
All
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(5 attachments, 5 obsolete attachments)

(Reporter)

Description

4 years ago
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0 (Beta/Release)
Build ID: 20131112030204

Steps to reproduce:

Run the following JSFiddle, which uses Web Audio's createMediaElementSource() to route audio from an <audio> tag into a Web Audio context:

http://jsfiddle.net/ikerr/WcXHK/

To see debug output, open the Web Console (Tools > Web Developer > Web Console).  Note that the fiddle checks the audio context's sample rate and selects a matching 44100 Hz or 48000 Hz audio file (I haven't seen any other sample rates in my testing).

I tested this fiddle in Firefox 25, 27 (Aurora) and 28.0a1 (2013-11-12) on Mac OS X 10.9.  My system specs are as follows:

MacBook Pro, 15-inch, Mid 2012
Processor: 2.3 GHz Intel Core i7
Memory: 8 GB 1600 MHz DDR3
Graphics: NVIDIA GeForce GT 650M 512 MB


Actual results:

When this fiddle loads, I do not hear any music playing (i.e., silence).  When I run the fiddle, I get the following output in my Web Console (no errors):

10:02:28.005 "Context sample rate: 44100"
10:02:28.006 "Using audio asset: http://assets-staging.verold.com/entities/52812b86a9aff5020000000b/3f6b2c8c8996e748a753cf26c6f9406a/06+Old+Man-44100.mp3"
10:02:28.006 "Using audio asset: http://assets-staging.verold.com/entities/52812b86a9aff5020000000b/3f6b2c8c8996e748a753cf26c6f9406a/06+Old+Man-44100.ogg"
10:02:28.006 "Created MediaElementAudioSourceNode."


Expected results:

When this fiddle loads, music should start playing.  This works in the Chrome 30.0.1599.101, for example.  Documentation for createMediaElementSource() can be found here:

https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#MediaElementAudioSourceNode-section
(Reporter)

Updated

4 years ago
Component: Untriaged → Web Audio
Product: Firefox → Core
Created attachment 831070 [details]
audio element testcase

This plays as expected.
Created attachment 831072 [details]
source element child of audio element testcase

This is reduced from http://jsfiddle.net/ikerr/WcXHK/
It doesn't produce sound.
Created attachment 831073 [details]
source element child of audio element testcase

Sorry.  This version, still without createMediaElementSource plays.
Attachment #831072 - Attachment is obsolete: true
wget -S http://assets-staging.verold.com/entities/52812b86a9aff5020000000b/3f6b2c8c8996e748a753cf26c6f9406a/06+Old+Man-48000.ogg
indicates that "Access-Control-Allow-Origin: *" is returned.
Created attachment 831130 [details]
reduced testcase
Assignee: nobody → karlt
Attachment #831070 - Attachment is obsolete: true
Attachment #831073 - Attachment is obsolete: true
Status: UNCONFIRMED → NEW
Ever confirmed: true
Blocks: 855568
OS: Mac OS X → All
Summary: Silent output with createMediaElementSource → Silent output with cross-origin createMediaElementSource and Access-Control-Allow-Origin: *
Created attachment 831219 [details]
testcase with crossOrigin attribute without createMediaElementSource

Plays as expected.
Created attachment 831223 [details]
testcase with crossOrigin attribute and createMediaElementSource

No output.
Priority: -- → P2

Comment 8

4 years ago
I am experience the same problem. jsfiddle that fails: http://jsfiddle.net/7bJUU/ 
In the above example, the ogg file from wikipedia is returned with "Access-Control-Allow-Origin: *" but the
output is silent. The same example works if 1) createMediaElementSource is not used and file is played directly
2) when a local, same origin file is played

Comment 9

4 years ago
Additional test case: http://jsfiddle.net/xqhWY/2/

When the createMediaElementSource is created the output of the audio stops. The source is connected to the destination of the context, so I would expect output.

Comment 10

4 years ago
I think I found the problem. -> http://jsfiddle.net/xqhWY/4/

The sample rate of the Audio Context is different then the sample rate of the audio stream. It appears in the Web Audio API spec that the AudioContext determines which sample rate to use <http://www.w3.org/TR/webaudio/#Sample-rate> and it assumes that other nodes will use the same rate <http://www.w3.org/TR/webaudio/#dfn-sampleRate>. This would explain the unexpected behaviour I'm seeing from Firefox when I attempt to create a media source from the audio element.
(Reporter)

Comment 11

4 years ago
In the original bug report, I observed the problem even when the sample rate of the Audio Context matched the file's sample rate.  But I agree that when the sample rates do not match, the output should be silent.

(In reply to rbarber from comment #10)
> I think I found the problem. -> http://jsfiddle.net/xqhWY/4/
> 
> The sample rate of the Audio Context is different then the sample rate of
> the audio stream. It appears in the Web Audio API spec that the AudioContext
> determines which sample rate to use
> <http://www.w3.org/TR/webaudio/#Sample-rate> and it assumes that other nodes
> will use the same rate <http://www.w3.org/TR/webaudio/#dfn-sampleRate>. This
> would explain the unexpected behaviour I'm seeing from Firefox when I
> attempt to create a media source from the audio element.
See Also: → bug 996685
Duplicate of this bug: 1054120

Comment 13

3 years ago
I can replicate this bug, with video elements as well.

However, it's not that the audio is silent, but rather that it doesn't play at all. Here's a modified version of the jsfiddle that displays the audio element with controls. You'll see that the timeline does not progress even though it is "playing" ... or at least, not paused.

http://jsfiddle.net/WcXHK/37/

Comment 14

3 years ago
sorry i dupe'd the bug again .. did not see this in my activity area till today :)

Comment 15

3 years ago
ps.. my context sample returns 'undefined'
Duplicate of this bug: 1060150
(Assignee)

Comment 17

3 years ago
Created attachment 8537911 [details] [diff] [review]
Output frames to the MediaElementAudioSourceNode when an <audio> has labeled the resource has CORS-cross-origin. r=

So, this is kind of what <canvas> does for drawImage with cross-origin I think.
I can see audio frames being appended to the MediaStream in
MediaDecoderStateMachine::SendStreamData, but then something else is broken and
it does not work, so a bit more work is needed, but I'd like to validate the
approach here.
Attachment #8537911 - Flags: review?(roc)
(Assignee)

Updated

3 years ago
Assignee: karlt → padenot
Status: NEW → ASSIGNED
Attachment #8537911 - Flags: review?(roc) → review+
(Assignee)

Comment 18

3 years ago
Created attachment 8540254 [details] [diff] [review]
Output frames to the MediaElementAudioSourceNode when an <audio> has labeled the resource has CORS-cross-origin. r=

This is the rest of the work, it works, now. I'll r? when the mochitest is
ready, it kind of works but it too flaky.
(Assignee)

Comment 19

3 years ago
https://tbpl.mozilla.org/?tree=Try&rev=63849cd43f96
(Assignee)

Comment 20

3 years ago
Created attachment 8541235 [details] [diff] [review]
Output frames to the MediaElementAudioSourceNode when an <audio> has labeled the resource has CORS-cross-origin. r=

The silence was caused by the fact that the track was not enabled, because at
the MediaStream level, there was another origin check.

By teaching the existence of CORS to DOMMediaStream, and wiring that down to
MediaStreamAudioSourceNode::PrincipalChanged, we can track the cross-origin
status of the media that gets fed to the MSG, and mute/unmute accordingly.
Attachment #8541235 - Flags: review?(roc)
(Assignee)

Updated

3 years ago
Attachment #8540254 - Attachment is obsolete: true
(Assignee)

Comment 21

3 years ago
Created attachment 8541236 [details] [diff] [review]
Test for MediaElementAudioSourceNode and cross-origin resources. r=

And the test, using an .sjs file to add the header. Green on try.
Attachment #8541236 - Flags: review?(roc)
(Assignee)

Updated

3 years ago
Attachment #8537911 - Attachment is obsolete: true
Attachment #8541235 - Flags: review?(roc) → review+
Attachment #8541236 - Flags: review?(roc) → review+
(Assignee)

Comment 22

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/0c9f97ebf7ea
https://hg.mozilla.org/integration/mozilla-inbound/rev/bcbc64d97fbf
https://hg.mozilla.org/mozilla-central/rev/0c9f97ebf7ea
https://hg.mozilla.org/mozilla-central/rev/bcbc64d97fbf
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla37
(Assignee)

Updated

3 years ago
Duplicate of this bug: 1111382

Comment 25

3 years ago
This still isn't working for me. I'm trying it on Nightly 38.0a1 (2015-02-09) on OSX 10.9.5. I tried in both an e10s and Non-e10s window with no success in either.

Can we reopen this?
(Assignee)

Comment 26

3 years ago
(In reply to brian from comment #25)
> This still isn't working for me. I'm trying it on Nightly 38.0a1
> (2015-02-09) on OSX 10.9.5. I tried in both an e10s and Non-e10s window with
> no success in either.
> 
> Can we reopen this?

Open a new bug, please. Bonus points if you have a test case!

Comment 27

3 years ago
The example from bug 1111382 still doesn't work in the nightly: http://www.jlescure.com/samples/firefox-bug.html

Comment 28

3 years ago
(I was using 38.0a1 (2015-02-19) on OSX 10.10.1)

Comment 29

3 years ago
This example from further up in the bug does not work either: http://jsfiddle.net/7bJUU/

Comment 30

3 years ago
i can confirm that this still wont work. even though soundcloud has access control allow origin on each of its mp3s, output is still silent in released firefox 37.0.1.

Comment 31

3 years ago
update: it just skips a bit at the start, but will play if audio tag has crossorigin = 'anonymous' property!
disregard previous comment. in the jlescure.com example you should add audio.crossOrigin = 'anonymous'; and it also works :) thanks for fixing this!
You need to log in before you can comment on or make changes to this bug.