Closed Bug 1797362 Opened 4 months ago Closed 3 months ago

Microphone input distorted

Categories

(Core :: Audio/Video: Recording, defect)

Firefox 108
defect

Tracking

()

RESOLVED FIXED
109 Branch
Tracking Status
firefox109 --- fixed

People

(Reporter: dvoracekjirkas, Assigned: padenot)

References

Details

Attachments

(7 files)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0

Steps to reproduce:

  1. Open mic-test.com (but not relevant to this website, unusable in any other recording via browser)
  2. Allow integrated microphone (mic array, Realtek HDA drivers, HP)
  3. Start recording
  4. Listen to recording

Windows 10, 22H2
Sound enhancements enabled, slightly better with enhancements disabled but still not usable.

I used mozregression and narrowed the issue to be (possibly) in changeset between a35461d1fc07e6e855d064453363a46d32bb5a3d and bd7c35b7e234712f94f6dbb755ad983c1db8b07b.

Present on stable v106 and nightly v108.

Captured profile while capturing distorted audio: https://share.firefox.dev/3zeOdWb

Actual results:

  1. Distorted audio, sound like clipping.
  2. Not happening in MS Edge or Google Chrome or MS Teams.

Expected results:

Clean audio.

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core
Component: Audio/Video: Playback → Audio/Video: Recording
Flags: needinfo?(padenot)

Reporter, would you mind attaching here the "raw" data from about:support, in the same configuration (same audio devices plugged in, etc.)? There's something interesting in the range you provided, I'm investigating, but it's hardware (or configuration) dependant.

Flags: needinfo?(padenot) → needinfo?(dvoracekjirkas)
Assignee: nobody → padenot
Status: UNCONFIRMED → NEW
Ever confirmed: true

Specifically, we might want to add a 1/input channel count scaling here: https://searchfox.org/mozilla-central/source/dom/media/webrtc/MediaEngineWebRTCAudio.cpp#927. My theory is that the microphone has 4 channels on this machine.

Attached file supportRAWdata.txt

(In reply to Paul Adenot (:padenot) from comment #2)

Reporter, would you mind attaching here the "raw" data from about:support, in the same configuration (same audio devices plugged in, etc.)? There's something interesting in the range you provided, I'm investigating, but it's hardware (or configuration) dependant.

Hi, attaching the RAW input as requested. And you're right, the input device in question has 4 channels:

  {
    "name": "Mikrofon (Realtek(R) Audio)",
    "groupId": "HDAUDIO\\FUNC_01&VEN_10EC&DEV_0285&SUBSYS_103C8898&REV_1000\\5&6878457&0&0001",
    "vendor": "",
    "type": 1,
    "state": 2,
    "preferred": 5,
    "supportedFormat": 4112,
    "defaultFormat": 4096,
    "maxChannels": 4,
    "defaultRate": 48000,
    "maxRate": 48000,
    "minRate": 48000,
    "maxLatency": 480,
    "minLatency": 144
  }

I tested it now also with my earphones (1 ch) and the issue wasn't present:

  {
    "name": "Headset (Galaxy Buds Pro (4811) Hands-Free AG Audio)",
    "groupId": "BTHHFENUM\\BthHFPAudio\\9&1edb9e0b&0&97",
    "vendor": "",
    "type": 1,
    "state": 1,
    "preferred": 0,
    "supportedFormat": 4112,
    "defaultFormat": 4096,
    "maxChannels": 1,
    "defaultRate": 8000,
    "maxRate": 8000,
    "minRate": 8000,
    "maxLatency": 0,
    "minLatency": 0
  }

Thanks for assistance. :)

Flags: needinfo?(dvoracekjirkas)

Thank you very much for the very speedy answers, I'll get a patch up hopefully tomorrow (it's getting late here) and I'll probably ask you to report if it's working well, as I don't think I have a machine with a quad input mic around at the moment.

Attachment #9302529 - Attachment description: Bug 1797362 - Test that downmixing is down appropriately (or not performed) in pass through and non-pass through audio input processing mode. r?pehrsons → Bug 1797362 - Test that downmixing is done appropriately (or not performed) in pass through and non-pass through audio input processing mode. r?pehrsons
Pushed by padenot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3a8f0fce898a
When downmixing audio input that has more than two channels before passing it to webrtc::AudioProcessing, scale by the number of channels to avoid clipping. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/48fab6b48e09
Test that downmixing is done appropriately (or not performed) in pass through and non-pass through audio input processing mode. r=pehrsons

Backed out for causing build bustages on TestAudioInputProcessing.cpp

  • Backout link
  • Push with failures
  • Failure Log
  • Failure line: /builds/worker/checkouts/gecko/dom/media/gtest/TestAudioInputProcessing.cpp:372:34: error: no viable conversion from 'nsTArray<float>' to 'nsTArray<mozilla::AudioDataValue>'
Flags: needinfo?(padenot)
Pushed by padenot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/277a48916295
When downmixing audio input that has more than two channels before passing it to webrtc::AudioProcessing, scale by the number of channels to avoid clipping. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/f2a77659167f
Test that downmixing is done appropriately (or not performed) in pass through and non-pass through audio input processing mode. r=pehrsons

Backed out 2 changesets (Bug 1797362) for causing gtest failures on TestAudioInputProcessing.cpp and AudioSegment.h.
Backout link
Push with failures <--> GTest-1proc
Failure Log
Also

Pushed by padenot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/07b6b9a7b101
When downmixing audio input that has more than two channels before passing it to webrtc::AudioProcessing, scale by the number of channels to avoid clipping. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/f130aa968d7e
Test that downmixing is done appropriately (or not performed) in pass through and non-pass through audio input processing mode. r=pehrsons
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → 109 Branch
Attached file AudioSamples.zip

Hi, thanks for the fix. It looks like it helped but only in single case. :/ But it may be harder to fix than anticipated.

I attached archive with 4 audio samples.

MSEdgeStable - control, this is how it should sound like; noise cancelling from the laptop's array and software ON
firefoxStable107 - recording from ff stable v107.0; noise cancelling ON
firefoxNightly_2022-11-16 - still sounds bad :(; noise cancelling ON.
firefoxNigtly_NoiseCancellingOff - looks like the clipping is gone; noise cancelling OFF.

I also tried to make some recordings with audacity (noise cancelling ON) but the recordings were unfortunately unusable exactly the same way as the recordings made with firefox with noise cancelling ON. In audacity I tried Windows DirectSound, WIndows WASAPI and MME with 2 (or 4 with WASAPI) ch.

Built-in windows voice recorder also sounded bad the same way.

Maybe Chromium has different way to capture sound? I am trying to find diagnostic data in Edge or Chrome but no luck so far.

Hope this helps a bit...

Ok, having those audio clips helps a lot, thanks. Can you try (in the Nightly build) to go to about:config, and type media.cubeb.wasapi-raw in the text box, creating a "Number" configuration option, and then setting it to 1 (disable processing for input) ? You can also try 3 (disable processing for input and output).

I'm attaching a short video clip that shows how to do it, just to make sure.

Essentially, this instructs Firefox to ask Windows to not use any audio input processing in the audio stack, and simply pass the audio from the microphone to the program. This is probably what we want here because otherwise we'd be stacking various audio processing passes: one from your mic array, and then one from Firefox, and then (depending on the service you use for video conferencing), maybe another one inserted by the website itself.

I don't have a Windows machine handy at the moment, but I can double-check when I'm back in the office, next week.

Additionally, would you mind sharing the brand and model of your computer, so I can maybe find a machine that has this type of hardware, and do something a bit more durable than asking users to flip preferences?

Flags: needinfo?(padenot) → needinfo?(dvoracekjirkas)

I've opened bug 1801066 to continue investigating this (with the backstory and some ideas).

See Also: → 1801066
Attached file aboutConfigTest.zip

Ok, I tried it and it sound great with the processing disabled (value 1 or 3). See attachement.

Laptop: HP Elitebook 845 G8, product number: 48R68EA

Flags: needinfo?(dvoracekjirkas)

This is nice to hear, thanks for following up. I'm not going to change this configuration option any time soon, so you can keep it like this. I'll be doing a more durable solution in bug 1801066.

Also, thanks for providing the laptop model, I'll see if I can find a computer with a similar microphone array, and I can probably find a better way to do this once I have the machine in front of me.

QA Whiteboard: [qa-109b-p2]
You need to log in before you can comment on or make changes to this bug.