1124981 - Support getUserMedia input rates >48000Hz

Status: REOPENED

(Core :: WebRTC: Audio/Video, defect, P4)

Description

[Trevor Rowbotham [:rowbot]]

When trying to use Firefox Hello in both the latest nightly and Fx35, I get the following output in the browser console when clicking on Start a conversation:

22:06:14.734 "OT.Publisher.onStreamAvailableError NotFoundError: Unknown Error while getting user media" sdk.js:910:14

22:06:14.758 "OT.exception :: title: Unable to Publish (1500) msg: Publisher failed to access camera/mic: Unknown Error while getting user media" sdk.js:910:14

22:06:14.764 1500 "Unknown Error while getting user media" sdk.js:910:14

22:06:14.765 "OT.exception :: title: Unable to Publish (1500) msg: Unknown Error while getting user media" sdk.js:910:14

22:06:14.883 no element found ClientEvent:1:1
22:06:14.972 no element found ClientEvent:1:1
22:06:15.054 no element found ClientEvent:1:1

People cannot hear me, but I can hear them.  Microphone works perfectly fine in mumble, teamspeak, and skype, as well as listening to it loop back through the audio control panel.

My soundcard is a Sound Blaster X-Fi XtremeGamer Fatal1ty Pro with the latest drivers, version 2.30.0004.  Using Windows 7.  Microphone is set as both the default device and default communications device.  I do have the nVidia HD audio driver installed as well for my video card, but I don't know if that has anything to do with this.  Onboard sound is disabled in the BIOS.

Comment 1

[Trevor Rowbotham [:rowbot]]

Moving to a more appropriate place since I couldn't find the loop component when I filed the bug.

Comment 2

[Mark Banner (:standard8)]

Do you have a camera installed/connected?

Comment 3

[Trevor Rowbotham [:rowbot]]

No, I do not have a camera installed or connected.

Comment 4

[Mark Banner (:standard8)]

Ok, thanks. That's a known issue we're working on, so I'll mark it as a duplicate of that bug.

Comment 5

[Trevor Rowbotham [:rowbot]]

Unfortunately, bug 1106941 did not resolve my issue.  Audio input in Firefox Hello still does not work for me.  I still see the same errors as in comment #1 in the browser console.  If I remember correctly, the last time that I tried out Hello and it actually worked was when Firefox 34 was in the nightly channel, although it would disconnect me from the conversation within 15 seconds at that time. I'll see if I can find a regression range.

Could this have something to do with the fact that I'm using the 64-bit nightly?
Could this have something to do with the fact that my audio drivers are using the Windows Driver Model (WDM) instead of the Unified Audio Architecture (UAA), which is what Vista and higher prefers?

Comment 6

[Trevor Rowbotham [:rowbot]]

Well, finding a regression range didn't go too well.  None of the builds worked :(  Looking in the browser console in some of the older builds I get the follwing:

"OT.Publisher.onStreamAvailableError NoDevicesFoundError: No voice or video input devices are available on this machine." sdk.js:913

"OT.exception :: title: Unable to Publish (1500) msg: Publisher failed to access camera/mic: No voice or video input devices are available on this machine." sdk.js:913

1500 "No voice or video input devices are available on this machine." sdk.js:913

"OT.exception :: title: Unable to Publish (1500) msg: No voice or video input devices are available on this machine." sdk.js:913

1500 "Session.publish :: No voice or video input devices are available on this machine." sdk.js:913

"OT.exception :: title: Unable to Publish (1500) msg: Session.publish :: No voice or video input devices are available on this machine." sdk.js:913

no element found ClientEvent:1

Looks like the errors generated by the SDK got more generic over time.  I would bet that these errors are the same as the ones in comment #1.  So my guess is that this is either a driver issue or Firefox is doing something wrong when looking for input devices.  If either of these are the case, then this is probably a webRTC bug rather than a Hello bug.  But, I suppose the SDK could be doing something wrong somewhere as well.

Creative's audio drivers are notoriously bad and I would like to avoid having to change drivers if possible since its a headache with Creative, but I'd be willing to give it a try if you think that is the root cause here.

Comment 7

[Mark Banner (:standard8)]

Hi Trevor,

Can you confirm you're definitely using the latest nightly (via help -> about)?

Also, could you try accessing your microphone and/or camera from this page:

http://mozilla.github.io/webrtc-landing/gum_test.html

That will help eliminate if this is an issue in Hello or the core of Firefox.

Comment 8

[Trevor Rowbotham [:rowbot]]

Build ID: 20150312030248

The About Nightly window also reports that Nightly is up to date with 39.0a1 (2015-03-12).

Testing the audio only option on that page reports Success! in green, however, I cannot hear any sound coming from the audio player on the page when I speak into the mic.  The browser console doesn't report any errors.

Disabling e10s didn't make a difference either.

Comment 9

[Trevor Rowbotham [:rowbot]]

I assume since the tests on that page don't work for me that this is a core bug rather than a bug in Hello.  Is there any sort of advanced logging that I can enable to perhaps try and figure out what is going wrong?

Comment 10

[Trevor Rowbotham [:rowbot]]

So, I tried installing the UAA version of the drivers today since I ran out of things to try.  After installing and rebooting, the test at [1] worked.  I thought everything was good to go.  I then went to try Hello, but that still didn't work.  Then I went back to [1] to test it again and that wasn't working anymore either.  No idea whats going on here.

I tested both gUM and Hello in some VMs using the problem computer as the host machine and all the VM's ran both gUM and Hello perfectly fine.  So, I'm pretty confident that this is not a Hello bug, but its difficult to say whether this is a bug in Firefox's core or a driver bug.

Any other thoughts on this, Mark?

[1] http://mozilla.github.io/webrtc-landing/gum_test.html

Comment 11

[Mark Banner (:standard8)]

Hi Trevor, I'm going to pass you to the core team. As they have logging, but I'm not sure which one is best for your case.

Maire/Randell can you try and help Trevor please? (note hardware details in comment 0).

Comment 12

[Maire Reavy [:mreavy]]

Thanks for flagging this to us.  I agree.  This is a core bug.

Trevor -- Can you get getUserMedia and Media logs for us?  https://wiki.mozilla.org/Media/WebRTC/Logging tells you how.  (We don't need ICE/STUN/TURN or Signaling (SDP offer/answer handling) logs, which are also described on that page.)

Once you have the logs, please upload them to this bug and needinfo me.  We'll then go from there.  Thanks!

Comment 13

[Trevor Rowbotham [:rowbot]]

Hi Maire,

I must be doing something wrong as I can't get the logging to work.  However, I did discover the underlying issue here.  My playback device's sampling rate is set to 24 bit, 96000 Hz in the Windows audio control panel.  Changing the sampling rate to 48000 Hz or lower resolves the issue with both gUM and Hello.  So, it would appear that the webRTC code has an issue handling audio sampling rates > 48000 Hz.

If you would still like some logs, I would be more than happy to provide them, but I would require some additional information on how to get the logging working.

Comment 14

[Maire Reavy [:mreavy]]

Hi Trevor,  Given that 48KHz works, the problem you're having is that we (Firefox) don't yet support 96KHz/24-bit sampling.  So, rather than have you jump through hoops to get us logs, I think it makes sense to turn this bug into "Add support for a 24-bit, 96kHz sampling rate".

Just out of curiosity, does your test work in Chrome?

Comment 15

[Trevor Rowbotham [:rowbot]]

Hi Maire,

Chrome already seems to support sampling rates > 48Khz.  The test at [1] works in Chrome, but does not work in Firefox when I have my sampling rate set at 96Khz.  Additionally, if I create a Hello conversation in Firefox and join it in Chrome, I can hear myself in Firefox when talking in Chrome, but I cannot hear myself in Chrome when talking in Firefox when I have my sampling rate set at 96Khz.  I tested all the options that are available to me below:

16 bit, 44100 Hz (works)
16 bit, 48000 Hz (works)
16 bit, 88200 Hz (broken)
16 bit, 96000 Hz (broken)
16 bit, 176400 Hz (broken)
16 bit, 192000 Hz (broken)
24 bit, 44100 Hz (works)
24 bit, 48000 Hz (works)
24 bit, 96000 Hz (broken)
24 bit, 192000 Hz (broken)

The bit part seems to be irrelevant, just anything > 48Khz is a problem from what I can tell.  If it is helpful, :padenot fixed a similar issue having to do with the sampling rate in Bug 893307.  Based on Bug 893307 comment #33, I would have expected this to be a non-issue in WebRTC.

[1] http://webrtc.github.io/samples/src/content/getusermedia/audio/

Comment 16

[Trevor Rowbotham [:rowbot]]

This bug is somewhat confusing.  Although changing the sample rate of the playback device "resolves" the issue, this bug screams problems with audio input in Firefox.  The YouTube HTML5 player plays audio perfectly fine with the sampling rate set to 96Khz as does audio input coming from Chrome or another browser.  So, from what I can tell Firefox already supports 96Khz audio playback, but there is an issue with audio input.  Perhaps an issue with sample rate conversion on audio input? I think Bug 893307 only dealt with audio output.

Comment 17

[Maire Reavy [:mreavy]]

The audio input sample rates are independent of the sample rates supported for output.  Supporting other input rates (on Windows, and maybe Linux) would be useful; I believe Mac already resamples before we see the samples.

Comment 18

[chris.rudmin]

When attempting to use a 96000 input sample rate, the navigator.mediaDevices.getUserMedia API returns success and a stream with null data. The expected behaviour for an unsupported sample rate would be an error callback with an appropriate error message.

Comment 19

[Dietrich Ayala (:dietrich)]

Marking DevAdvocacy as I've seen several references to this bug while using Node.js libraries for audio processing.

Eg: https://www.npmjs.com/package/opus-recorder

Comment 20

[Bulk Bug Changes for mreavy's org]

Mass change P3->P4 to align with new Mozilla triage process.

Comment 21

[Trevor Rowbotham [:rowbot]]

I can still reproduce this using the latest nightly.