1155769 - html5 audio/video fails to play any audio on N-Computing X350 X550 in Windows 7

Status: RESOLVED WONTFIX

(Core :: Audio/Video: cubeb, defect, P3)

Description

[Russell Valentine]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0
Build ID: 20150403141925

Steps to reproduce:

Run firefox on a N-Computing X550 or X350 in windows 7. Open any URL that uses html5 audio or video (example: http://hpr.dogphilosophy.net/test/ )

No audio will play.

This was tried with many different Windows 7 hosts, all experience this problem on the NComputing client.


Actual results:

In windows 7 using a X350 or X550 N-Computing device audio fails to play on the client devices. Firefox gives warning messages as if it doesn't know how to decode the audio file (ogg, opus, etc) even though it is a supported codec and works otherwise on the host or other machines.

For video files, the video plays but there is no audio to be heard.

Why this is a firefox bug is that audio works for all other applications except firefox. This include Chrome's html 5 audio.

Also the flash plugin audio works. Just not firefox html5 audio/video.

I've contacted the N-Computing technical support to try to resolve the issue. They suggested I file a bug here. Below is my slightly-edited correspondence.
---------------------------------------
** From me:
Hello, we are having compatibility problems with Mozilla Firefox.

We have an issue where audio is not heard when using html5 audio/video (as opposed to flash) on the clients. We hear it fine on the host. Also on the client html5 audio works fine in chrome. However in firefox html5 audio we can not hear on the clients.

Do you have any suggestions to work around this issue?

When Windows XP 32 bit is the host everything is working fine. But with Windows 7 32/64 bit hosts using X350/X550, the clients can not get html5 audio out of firefox.

Recap:

    X350/X550 with Windows 7 32 or 64 bit host
    Firefox html5 audio/video can not be heard on the clients
    Chrome html5 audio/video can be heard on the clients
    Flash audio/video can be heard on the clients
    Firefox html5 audio/video can be heard on the host

    Windows XP 32 bit host, clients hear audio/video.

** From NComputing
Russell,

Thank you for contacting NComputing Technical Support.

This could be related to an IRQ conflict that is caused by port management on the system. It may not be this issue because the sound works fine in Chrome but it is worth a shot. Please go to the Windows Device Manager and disable all items under the "Ports" category and then perform a reboot of the host. Let us know if this resolves the issue.

Another possible fix:

Finally, after hundreds of hours I fixed it and Firefox plays sound in HTML5 in everywhere, YouTube (HTML5 mode), SoundCloud, Bandcamp and etc.

You have to: Right-click on your sound/speakers icon in the system tray and select Playback Devices, go to Speakers properties (the one which is green-ticked, also disable the red ones, if any), then go to Enhancements and tick Disable all sound effects. And that's it, restart the browser and enjoy.

In short: Volume mixer -> Playback devices -> Playback tab -> Speakers properties -> Enhancements -> Disable all sound effects -> Restart browser.

If you use extensions (Firefox/Tools > Add-ons > Extensions) that can block content (e.g. Adblock Plus, NoScript, Flash Block, Ghostery) then make sure that such extensions aren't blocking content.

** From Me

We tried the IRQ conflict suggestion and Disable all sound effects. Both have been reported to not solve this issue.

** Later I ran a debugging tool that they wanted me to run

Russell,

Thank you for contacting NComputing Technical Support.

I had a chance to review the NCDebug and found nothing out of the ordinary that would be particular to the sound issues.

The fact that sound does not work with HTML5 in Firefox but it does in Chrome leads me to believe the issue is specific to Firefox and you may want to see if they have a support option to get assistance.

Here is another article that may be of some use to you:

https://support.mozilla.org/en-US/kb/html5-audio-and-video-firefox



Expected results:

We should hear audio when trying to play supported html5 audio/video files on the X350 and X550 clients.

There are several schools that use these N-Computing devices, this is the main reason why I care about this issue.

Comment 1

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Is this still an issue?

Comment 2

[Russell Valentine]

As far as I know yes. Did something change with the audio code that I should verify it again? Is there something I can do to help with this fix, like maybe send some hardware somewhere? Just really confused because every audio thing I tried works great except firefox/xulrunner.

Comment 3

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

(In reply to Russell Valentine from comment #2)
> As far as I know yes. Did something change with the audio code that I should
> verify it again?

Much has changed. If it is still an issue then you should make sure pulse-audio is installed and running.

Comment 4

[Russell Valentine]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #3)
> (In reply to Russell Valentine from comment #2)
> > As far as I know yes. Did something change with the audio code that I should
> > verify it again?
> 
> Much has changed. If it is still an issue then you should make sure
> pulse-audio is installed and running.

Understood will see if I can try it again. However please note, this is a problem for Windows. I filed the bug on a Linux machine, so that user agent showed up. Sorry for the confusion.

Comment 5

[Russell Valentine]

The problem still exists in latest 43.0.3 as described. If I can help by sending hardware to someone, or trying another build let me know.

Comment 6

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

(In reply to Russell Valentine from comment #5)
> The problem still exists in latest 43.0.3 as described. If I can help by
> sending hardware to someone, or trying another build let me know.

We will look into it if you send one to me in Auckland https://www.mozilla.org/en-US/contact/spaces/auckland/

Comment 7

[Matthew Gregan [:kinetik]]

Given that audio works in Firefox on XP, my first guess is that there's a problem with the WASAPI audio backend in Firefox (which is only available in Vista upwards) and/or it's not available in the vSpace virtualized desktop and the code is not falling back to the old WinMM backend correctly.

An interesting test would be to play audio in Chrome on one of the machines Firefox is failing on, then open chrome://media-internals in a new tab, select the audio tab inside that tab and click the controller and stream listed (there should be one of each if there's only one tab playing audio).  If the frames_per_buffer value there is 2048, that suggests that Chrome is falling back to the WinMM backend.  If it's some other (lower) value, that suggests Chrome is using WASAPI.

I can also give you a test build with some logging in the audio initialization code to try so that we can narrow down where the init (and fallback, if necessary) is going wrong.

Comment 8

[Matthew Gregan [:kinetik]]

Russell, there's a special build with logging available at http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-7a6a917f11722e971ef12a02042461549e214695/try-win32/

It writes to firefox-audio.log in the current directory. Please note that you must disable "Enable multi-process Nightly" in about:preferences under General and restart before testing as the sandboxing used by multi-process Firefox prohibits the filesystem access necessary for the crude logging I added.

Comment 9

[Russell Valentine]

In chrome: chrome://media-internals  The controller has 882 frames_per_buffer value.  In the stream it has 2048 frames_per_buffer value.

I will get back to you with the log from the build.

Comment 10

[Russell Valentine]

More detailed info from chrome's chrome://media-internals

*Controller*

channel_layout: STEREO
channels: 2
component_id: 2
component_type: 1
device_id: default
device_type: pcm_low_latency
effects: NO_EFFECTS
frames_per_buffer: 882
owner_id: 8
sample_rate: 44100
status: started
render_process_id: 14
web_contents_title: Which audio formats are supported?

*Stream*

channel_layout: STEREO
channels: 2
component_id: 1
component_type: 2
device_id: 
device_type: pcm_linear
effects: NO_EFFECTS
frames_per_buffer: 2048
owner_id: 1
sample_rate: 44100
status: started
volume: 1

Comment 11

[Russell Valentine]

From firefox-audio.log:

cubeb_init
trying backend '659835E9'
backend 'wasapi' initialized, context '668058C8'
wasapi: initializing audio client failed: 80004003
wasapi: stream '0A14F94C' initialized

Comment 12

[Matthew Gregan [:kinetik]]

Thanks Russell, much appreciated!

If I understand Chrome's audio architecture correctly (and I don't claim to be familiar with it, so I may be wrong!) the stream with a device_type of pcm_linear, a frames_per_buffer value of 2048, and an empty device_id (WASAPI includes a UUID here) strongly suggests the audio output is via the older WinMM backend.

The logging I added shows that we're not falling back from WASAPI because the tests we perform for API availability in wasapi_init all succeed and then the actual stream creation fails later (where we won't fall back to the older API) with a slightly confusing error.  Chrome's code to probe API availability looks very similar, except it also tests the Windows version and whether audioses.dll can be loaded... I don't expect either of those tests would be failing in this case anyway, so I doubt that difference matters, but I'll add similar tests to the next batch of logging to double check.  I'm confused though, because it looks as if Chrome is falling back to WinMM.

Once we reach stream initialization, the logging shows IAudioClient::Initialize fails with an
HRESULT of 0x80004003 which is E_POINTER.  That is a documented as a return code for IAudioClient::Initialize when the pFormat parameter is null... which it can't be in this case, because we dereference the pointer we pass in before calling Initialize, so you'd expect a NULL deref crash.  Perhaps we hit that error for some other (undocumented) reason?

I've pushed a new build with more logging, it should show up at http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-0ce9cd0d477fe6807b860e84011f040d6bc70a9b/try-win32/ within the next couple of hours.  If you could try testing again with that, it might give us a new clue.

In this build, I've also added a (very temporary and hacky) way to force fallback to the older audio API.  To do this, set the environment variable GECKO_FORCE_WASAPI_FAILURE to 1.  You should see an entry in the log saying "wasapi: WASAPI force disabled" if it worked.

Comment 13

[Russell Valentine]

Audio works when GECKO_FORCE_WASAPI_FAILURE is set to 1

*Without GECKO_FORCE_WASAPI_FAILURE*

cubeb_init
trying backend '6725F700'
wasapi: is_vista_up test: 1
wasapi: load_audioses_dll test: 1
wasapi: GetDefaultAudioEndpoint(eRender, eMultimedia) failed: 8000ffff
wasapi: get_default_endpoint failed: 8000ffff
trying backend '6725DF85'
backend 'winmm' initialized, context '680E7D48'
winmm: waveOutOpen failed: 2
winmm: stream '0D8EFAF4' initialized
winmm: waveOutOpen failed: 2
winmm: stream '0BC0FA84' initialized
winmm: waveOutOpen failed: 2
winmm: stream '0B86F904' initialized
winmm: waveOutOpen failed: 2
winmm: stream '0BEEF9E4' initialized
winmm: waveOutOpen failed: 2
winmm: stream '0BC0F964' initialized
winmm: waveOutOpen failed: 2
winmm: stream '0DADFCC4' initialized
winmm: waveOutOpen failed: 2
winmm: stream '12CBF80C' initialized
winmm: waveOutOpen failed: 2
winmm: stream '0D8CF6CC' initialized



*With GECKO_FORCE_WASAPI_FAILURE*

cubeb_init
trying backend '5E41F700'
wasapi: WASAPI force disabled
trying backend '5E41DF85'
backend 'winmm' initialized, context '5F2A7D48'
winmm: stream '0ED7F610' initialized
stream '1142AB40' destroyed
winmm: stream '0A25FAC0' initialized

Comment 14

[Matthew Gregan [:kinetik]]

Thanks again Russell.

(In reply to Russell Valentine from comment #13)
> Audio works when GECKO_FORCE_WASAPI_FAILURE is set to 1

Consistently over multiple Firefox restarts/client restarts/etc.?

I ask because the failure below has changed from the earlier log (which could be a result of my changes, but it's unexpected in that case):

> *Without GECKO_FORCE_WASAPI_FAILURE*
> 
> cubeb_init
> trying backend '6725F700'
> wasapi: is_vista_up test: 1
> wasapi: load_audioses_dll test: 1
> wasapi: GetDefaultAudioEndpoint(eRender, eMultimedia) failed: 8000ffff

So here we fail during WASAPI init and will try to fall back.  The earlier log showed this succeeding and a later failure during stream creation.

HRESULT 0x8000ffff is E_UNEXPECTED, which isn't a documented return code for GetDefaultAudioEndpoint, so I'm not sure what's happening there.

It's possible the act of loading audioses.dll has changed the behaviour.  That'd be a positive change if it allowed us to fall back, but that requires understanding why the winmm backend fails when falling back.

> wasapi: get_default_endpoint failed: 8000ffff
> trying backend '6725DF85'
> backend 'winmm' initialized, context '680E7D48'
> winmm: waveOutOpen failed: 2
> winmm: stream '0D8EFAF4' initialized

MMRESULT 2 is MMSYSERR_BADDEVICEID.  We only ever pass WAVE_MAPPER as the device ID, so I'm not sure why this error would occur.

Given the failure when trying to fall back, but success when immediately using the older API, it seems like the act of trying (and failing) to initialize WASAPI also breaks WinMM, which seems very unusual.

I'd like to understand this failure so we can fall back automatically, but I don't have any ideas for how to make progress debugging this right now.  In the worst case it seems like we could provide a hidden preference (or something) to force use of the old backend.

Comment 15

[Matthew Gregan [:kinetik]]

Another test build: http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-20ee9fc648f2f90fed791b00833ba2463b9ed443/try-win32/

This adds two more env vars:

GECKO_FORCE_WASAPI_SKIP_AUDIOSES - to see if loading audioses.dll is changing the init behaviour
GECKO_FORCE_WASAPI_ROLE_CONSOLE - to force the role to eConsole instead of eMultimedia

If you wouldn't mind running another round of tests with various combinations of the env vars set, the logs might give some additional clues... but I'm a bit stuck at this point.

Comment 16

[Russell Valentine]

Attachment: firefox-audio-08-01-2016.zip

Attached log files:

    Two times with no special environment variables set.
    audio test with just GECKO_FORCE_WASAPI_SKIP_AUDIOSES=1
    audio test with just GECKO_FORCE_WASAPI_ROLE_CONSOLE=1
    audio test with both GECKO_FORCE_WASAPI_SKIP_AUDIOSES=1 and GECKO_FORCE_WASAPI_ROLE_CONSOLE=1

Comment 17

[Matthew Gregan [:kinetik]]

Thanks Russell.  Sadly, the new logs don't give me any further clues, so I'm stuck now.  I'm not even sure what the next step to debug this would be if I had the hardware in front of me.

Out of curiosity, if you go back to the first test build, does that still fail with:

  wasapi: initializing audio client failed: 80004003

rather than:

  wasapi: get_default_endpoint failed: 8000ffff

like all of the later builds do?  That particular change in behaviour seems to have occurred between test builds, and I don't understand why.

Comment 18

[Russell Valentine]

Yes it does same as before. Log from running again with first build log below.

What about an prefs flag to be built in as a toggle? Is it hard to get one of those commited to be in official builds? A flag for forcing WASAPI failure.

After I originally posted this bug report I used ctypes to play audio using pyglet for these cases, however it doesn't help me for video as that still wouldn't work.

I know you don't know if it would help, but I can try to see if I can send hardware somewhere, to you, to Anothony? If willing to give it more of a shot.


cubeb_init
trying backend '563435E9'
backend 'wasapi' initialized, context '571C58C8'
wasapi: initializing audio client failed: 80004003
wasapi: stream '0B85FC70' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0E49F624' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0B82FC24' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0D38F730' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0D4FF804' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0E6BFC78' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0A4DF7C8' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0A4DF5EC' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0E68FB30' initialized
wasapi: initializing audio client failed: 80004003
wasapi: stream '0E6BFD08' initialized

Comment 19

[Matthew Gregan [:kinetik]]

Thanks Russell.  The behaviour definitely changed between those builds, but I don't understand why.  It's most likely from a change I made in the test builds, but having reviewed the code again, I don't see what could have made the behaviour of the system's IMMDeviceEnumerator::GetDefaultAudioEndpoint go from succeeding to failing.  Since I based the builds on different base revisions on Firefox (first change on base 26e9a666179b, second change on base 1fc8fc0fa9f4) it's possible a change elsewhere in the codebase is responsible, but that'd be pretty surprising.

Adding some type of flag to force fallback to the older WinMM may be possible as a last resort, but while it does solve your immediate issue it doesn't help anyone else with the same problem since they'd have to discover the cause and enable the workaround.  And since Chrome and other players work, it's possible they'd simply switch instead of investigating and reporting the problem.

For sending hardware, Anthony and I are in the same location, so that's still a possibility.

Do you think it's worth approaching N-Computing tech support again with the info we've got in the logs to see if they can give us any clues?

I could also build a standalone binary for testing based on libcubeb (the audio library we use) for you or them to test against, if you think that would help.

Comment 20

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

(In reply to Russell Valentine from comment #18)
> I know you don't know if it would help, but I can try to see if I can send
> hardware somewhere, to you, to Anthony? If willing to give it more of a
> shot.

I have contacted N-Computing through their web site in the hope that they will help us to figure out the issue. I'm concerned that setting up a device here may be a time consuming endeavour.

Comment 21

[Russell Valentine]

Great, thank you. In the mean time I'll work on sending a X550 PCI Card as well as a interface box to plug into the card.

Comment 22

[Matthew Gregan [:kinetik]]

After digging around more, I discovered that Chrome will fall back to the old audio API when opening the stream with the new API fails (relevant code is in AudioOutputResampler::SetupFallbackParams), where it will fall back to a "high latency" (aka WinMM) stream if the "low latency" (aka WASAPI) stream fails to open).

My suggestion is to implement the same type of fallback in our code, making it possible to switch from the WASAPI libcubeb backend to the WinMM backend not only on cubeb_init failure (where we are testing for WASAPI availability) but also on cubeb_stream_init failure.  I'm busy with other work at the moment, but I'll try to take a look at making this change in the near future.

Comment 23

[Russell Valentine]

Sent a package your way with some X550 gear in it. Emailed the tracking and customs form.

Comment 24

[Maire Reavy [:mreavy]]

Matthew -- Looks like you are already working on this;  so I'm just making it official by assigning this to you.

Comment 25

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.

Comment 26

[Matthew Gregan [:kinetik]]

We stopped supporting Windows 7 quite some time ago, so we won't ever fix this, sorry!