Open Bug 1434362 Opened 6 years ago Updated 9 months ago

Stereo upmix to 5.1 not happening when using Web Audio configured for stereo output

Categories

(Core :: Audio/Video: Playback, defect, P3)

Product:
Core
Component:
Audio/Video: Playback
Version:
59 Branch
Type:
defect

Tracking

()

People

(Reporter: thomas_jenkinson, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.119 Safari/537.36

Steps to reproduce:

user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0

On a system with a surround sound setup:

1) Load https://jsbin.com/renawuveyo/edit?html,js,output and click play.
2) Load https://jsbin.com/qawocax/edit?html,js,output



Actual results:

The output from 1 is upmixed to 5.1.

The output from 2 is only send from the front L and R speaker, and the bass is missing.

See https://soundcloudcommunity.com/desktop-230066/since-today-2018-01-15-music-only-plays-in-one-speaker-7396612#post18022543 for more context



Expected results:

The output from 1 and 2 should be identical.

In a 5.1 setup if 1 is upmixed to 5.1, then the second one should also be upmixed to 5.1.

In the example we are explicitly setting the `channelCount` to 2 on the destination, so I think the OS should only be seeing 2 channels from the browser. I am wondering if firefox is actually filling everything but the first 2 channels with silence and sending that?

The problem does not happen with the test pages in chrome.

Here is the users configuration: https://gist.githubusercontent.com/tjenkinson/9150c8da4a5178305bc9e84f62e734ae/raw/af914a051c4ec13daa17e8374f1b8800baa853ed/DxDiag.txt
Component: Untriaged → Audio/Video
Product: Firefox → Core
Sorry I just realised my examples are the wrong way around, and I can't see an edit button.


1 (web audio) is only output from front L and R
2 (standard audio element) is upmixed to 5.1
Component: Audio/Video → Audio/Video: Playback
Thanks for the report. At first glance it doesn't seem unreasonable that setting the AudioDestinationNode.channelCount to 2 would result in stereo output instead of a surround upmix. But then I didn't realize we upmixed at all.

Paul, which behaviour is correct?
Flags: needinfo?(padenot)
Priority: -- → P3
Not up-mixing is the correct behaviour in both cases. Jean-Yves is currently overhauling our multichannel support, this should work "soon". Work is happening in bug 1431221.
Depends on: 1431221
Flags: needinfo?(padenot)
Great!

Does this mean that the standard audio element will also only output in front L and R?

Or does this mean that Firefox will only send 2 channels, and this opens up the possibility for the OS/audio hardware to upmix?

One concern is that if an upmix doesn't happen, a stereo track out of just front L and R without any subwoofer might be missing most of the low end, because surround speakers won't have the full range as 2 standard ones.

Also with Chrome the user reported that the upmix does seem to happen for them. Is it intentional that Firefox is behaving differently (or maybe Chrome also outputs 2 channels and the OS is upmixng. I don't have surround speakers to test)?
Flags: needinfo?(padenot)
(In reply to thomas_jenkinson from comment #5)
> Great!
> 
> Does this mean that the standard audio element will also only output in
> front L and R?

If we're talking about a stereo media, yes.

> Or does this mean that Firefox will only send 2 channels, and this opens up
> the possibility for the OS/audio hardware to upmix?

As always, it depends. Say you're playing a stereo media (say a music track). A user that has, say, a 2.1 kit can have any combination of:
- Full range speakers / Speakers with only medium and tweeter
- For full range speakers, crossover enabled or disabled (to cut the low end, to leave some room for the subwoofer)
- Exposing 2.1 or 2.0 (with the sub automatically playing downmixed, low-passed audio when it receives a stereo stream)

Firefox cannot determine what to do in most cases, so I think we'll just send stereo (again, we're playing a stereo track here). The user will have to have its software and hardware properly configured. 

To summarize, it's unlikely we'll do the downmix/lowpass ourself.

> One concern is that if an upmix doesn't happen, a stereo track out of just
> front L and R without any subwoofer might be missing most of the low end,
> because surround speakers won't have the full range as 2 standard ones.

My take on that is that this is on the user.

> Also with Chrome the user reported that the upmix does seem to happen for
> them. Is it intentional that Firefox is behaving differently (or maybe
> Chrome also outputs 2 channels and the OS is upmixng. I don't have surround
> speakers to test)?

I don't think Chrome does anything special here. Firefox just has a bug.

All this requires a bit more testing on our end, I'll get some hardware and report back.
Flags: needinfo?(padenot)
Great thanks. I agree that the browser shouldn't do upmixing by default, and just pass through the channels it has.

If the audio context destination `channelCount` was set to 6, then I'd expect firefox to be sending 6 channels with 4 silent. This would be different to `channelCount` being 2, where it would only send 2 channels (and then maybe the OS/audio hardware would upmix).

Looking forward to see if this fixes the problem, or at least makes the output of playing a stereo track with an audio element vs audio element through web audio consistent.
Hello, I am the user Thomas is talking about. I think it's important to clarify a few things here.

I am a Windows 10 user with a Creative Sound Blaster X-Fi Xtreme Audio PCI Express 7.1 Sound Card and Creative 7.1 speaker system. It's an older card, but it does have Windows 10 drivers.

The card is set to "stereo surround" in the 3D expand section of the Creative control panel. The Windows OS "spatial sound" option is not available with this card, and is greyed out.

Chrome does NOT upmix any media sources, nor does Opera. Any media source, Soundcloud, Mixcloud, Youtube, etc. plays in 2.something sound, which is to say that the subwoofer outputs some bass, but it is very weak and thin sounding. Chrome sounds really bad, so Firefox has always been my browser of choice for streaming media.

Firefox does upmix most media sources into glorious 7.1 surround stereo with full bass. That is, all except Soundcloud. Mixcloud, Youtube, etc. all play perfectly fine. However, a recent change to Soundcloud has it playing just as it does in Chrome, in flimsy 2.1 sound.

Microsoft Edge behaves exactly the same way as Firefox and most media sources upmix perfectly, but unfortunately not Soundcloud.

All locally played media, including files downloaded from Soundcloud, play in 7.1 through media players like AIMP.

So, as far as I am concerned, Firefox is generally working well for upmixing most media sources, just not Soundcloud.
For me nothing is upmixed in FF 63. 

Which is a pitty, because AMD drivers (most users use their HDMI Audio for computer sound) don't offer an upmix function. In addition, I can't use my receiver (because it already gets 5.1 sound from the computer, even if 4 of them are quiet). I guess for most users, there's currently neither a SW nor a HW solution to upmix. That's why almost all audio and video players offer some upmix filters.

Is there an option in FF to enforce upmix of stereo sources? In https://bugzilla.mozilla.org/show_bug.cgi?id=1432779 it's mentioned that the same config as FFMPEG is used. Can I somehow install FFMPEG and configure it to upmix?

Would be really cool to enjoy youtube & spotify in surround.

BTW: I know there are around 0.5% audiophiles who use 10000€ speakers who propably want to enjoy stereo sources in stereo.
Don't know what changed, but after Firefox updated itself today the sound on Soundcloud is back to being upmixed to 7.1 and sounds amazing.
Unfortunately this is because we disabled web audio temporarily in Firefox 65 due to https://bugzilla.mozilla.org/show_bug.cgi?id=1515099

I think this bug might still exist with web audio.

Yep, it's back again. Audio has stopped upmixing from stereo to 5.1 and 7.1 on some sites like Soundcloud. This is a real shame because it was working great for the last three weeks. Why can't you just leave web audio disabled? It works better off :D

And now sites that used to upmix perfectly fine, like MixCloud, Bandcamp, and YouTube no longer do so. This is horrible.

(In reply to Brad from comment #13)

And now sites that used to upmix perfectly fine, like MixCloud, Bandcamp, and YouTube no longer do so. This is horrible.

This is unexpected. MixCloud is broken for me right now in Firefox and Chrome, but bandcamp and youtube don't use the Web Audio API at all. What version of Firefox are you using? Would you mind trying with Firefox Nightly (which is about 12 weeks more recent), which is available at 1.

I'm looking into fixing the case where the Web Audio API is used for playback, for example of stereo content, where AudioDestinationNode.channelCount is set to 2 (for stereo, that is), however.

Status: UNCONFIRMED → NEW
Ever confirmed: true

Firefox version used above was 65.0b10 (64-bit). Tested Nightly 66.0a1 (2019-01-15) (64-bit)and can confirm the problem exists across the board. No stereo upmix on Soundcloud, MixCloud, Bandcamp or YouTube.

Have played audio files locally in AIMP and upmix is working as expected.

As far as Chrome goes, I tried reporting this problem years ago and was rebuked for expecting my web browser to play back in stereo surround. I gave up and have used Firefox for media playback ever since.

Thanks for looking into this.

Any progress on this issue?

I'm not sure if I have this exact problem, but my sound also comes only from the left speaker when playing from soundcloud or the example in the OP (and it's very quiet in the right speaker) on FF 66.0.3 64-bit ubuntu 18.04.2

however mixcloud, youtube and bandcamp work

Here is another report showing what is possibly the same issue: https://community.soundcloud.com/desktop-230066/mono-stereo-issue-6601542#post18079121

You can see on youtube where webaudio is not being used 2 channels are shown, but on soundcloud (with webaudio) there are 4 for some reason, and left is mapped to 'left front' and right is mapped to 'front centre'.

I can report that upmixing with Firefox is working fine for the moment in Youtube, Mixcloud, Bandcamp, Spotify and music retailers like Traxsource, Beatport, Juno, etc.

Unfortunately, it's not working in Soundcloud still.

Please stop spreading rumors that FF is upmixing anything in Youtube. FF is not upmixing anything at all:

  1. Connect your video card hdmi to an Hifi Receiver
  2. Enable the video hdmi as primary sound in Windows --> this way you get the most clear, uncompressed, digital signal in the best quality out of you computer
  3. Deactivate any upmix in you Hifi receiver (dolby etc.)
  4. Test you setup with https://www2.iis.fraunhofer.de/AAC/multichannel.html (HE-AAC Bitstream)

With that test all speakers should output the correct sound for the correct box. Then you are correctly set up to play the original sound as it was intended initially. Afterwards you can try to load a youtube video and you will realize that it will only play on the front speakers.

There is no upmix in FF.

(In reply to sgl4kn from comment #20)

I can assure you that Firefox upmixes in all of the media outlets I listed above, including Youtube. The only outlet that it doesn't upmix in, which is the reason this bug exists in the first place, is Soundcloud.

The funny part about all this is the Frauenhofer test above works for me in Chrome on my setup, along with Firefox, yet not one single media outlet will upmix in Chrome for me at all normally.

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.