1739505 - After some time (75 min in this case), audio quality degrades considerably

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect)

Description

[Andrew Overholt [:overholt]]

Today I attended a virtual conference using hopin.com using Nightly on an M1 Mac. About 1.25 hours (75 minutes) into the session of 10 audio/video streams, audio from all participants got very garbled/choppy and laggy with an almost robotic sound.

Comment 1

[David Bolter [:davidb] (NeedInfo me for attention)]

Same thing, same conference. 2017 MBP. The similar sound problems happened about 10 minutes in. I rejoined in Chrome and it was OK, and later rejoined back on FF Nightly and it was fine for that second session.

Comment 2

[Michael Froman [:mjf]]

padenot, do you have any thoughts on what might be happening here? Would a profile be helpful if this happens again?

Comment 3

[Paul Adenot (:padenot)]

Yes, always.

Please use the "Media" profiler preset, that will automatically select all the right threads and settings. Recording for about 10s is enough, no need for a long profile. Please also mention if this was a Nightly from after the WebRTC uplift or not: if this has been opened four days ago and Nightly was not up to date, it might have been an older build.

Tomorrow's Nigthly (2021-11-11) will be better, please update if you can, I've added more threads to the profiler.

Comment 4

[David Bolter [:davidb] (NeedInfo me for attention)]

Thanks I haven't had the opportunity to experience the problem since this report. I'll catch a profile with the media preset if it happens again.

Comment 5

[Andrew Overholt [:overholt]]

Same.

Comment 6

[Paul Adenot (:padenot)]

One last thing, what audio devices were you both using in this call? For example:

• Builtin speakers, builtin-mic ?
• Analog headset, plugged into the jack socket on the macbook
• Bluetooth headset ?
• USB headset ?

Comment 7

[Andrew Overholt [:overholt]]

(In reply to Paul Adenot (:padenot) from comment #6)

One last thing, what audio devices were you both using in this call? For example:

External USB microphone, analog headphones plugged into an external USB speaker or maybe analog headphones plugged directly into 2020 MacBook Air headphone jack (my Jabra speakerphone thing sometimes doesn't work and I can't remember if it worked with hopin.com).

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:jib, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 9

[Jim Mathies [:jimm]]

Mathew, any thoughts here? Might be something in cubeb?

We're going to try to reproduce through QA.

Comment 10

[Jim Mathies [:jimm]]

I ran a test to see if I could reproduce -

1. windows laptop + zoom web client + latest nightly

connected to

2. a non-M1 2018 mac + zoom web client + latest nightly

This test has been running for about 1.5 hours and thus far no issues with audio.

This might be an M1 issue, which makes reproducing a bit harder since there are very few M1s available in engineering. Will do some additional poking around for those tomorrow.

Comment 11

[Jim Mathies [:jimm]]

No issue with that simple test case after about 2.5 hours. Will test with blue tooth connected devices, and I'll try to track down an M1 tomorrow.

Comment 12

[Jim Mathies [:jimm]]

Hey Eric, could you please post your about support text from the device + browser version that exhibits this issue? Thanks.

Comment 13

[Jim Mathies [:jimm]]

(In reply to David Bolter [:davidb] (NeedInfo me for attention) from comment #1)

Same thing, same conference. 2017 MBP. The similar sound problems happened about 10 minutes in. I rejoined in Chrome and it was OK, and later rejoined back on FF Nightly and it was fine for that second session.

^ Note, this may not be Mac M1 specific, based on David's report.

Comment 14

[Andrei Purice]

QA has tested this issue using different scenarios like (M1 mini (11.6) and MacBook Pro / M1 and Lenovo Yoga / M1 and Windows 10) with conference times of (30/50/75 mins) using zoom on Firefox Nightly 97.0a1 (2021-12-16).
We also tested it on a zoom webinar of over 100 minutes using the Firefox Nightly 97.0a1 (2021-12-16).
We tested with headphones and with M1's own audio mini speakers.
We did not encounter any issues (garbled/choppy audio, robotic sound or any other issue) on any of the devices.
If there are anymore details that could help us in our investigation or you wish us to provide more info, let us know.

Comment 15

[Jim Mathies [:jimm]]

So Zoom seems to be the best chance at reproducing this since to reproduce on hopin you have to be broadcasting. We'll do more extensive runs this week in the hopes of catching this in a profile.

Comment 16

[Jim Mathies [:jimm]]

I ran into a problem with Zoom where my audio output on the mac claimed it was sending, but other clients could not hear me. It stopped working randomly in the middle of a long zoom meeting. Not exactly the same issue (no garbled audio) but close.

https://share.firefox.dev/3pczCFQ

Comment 17

[Andrei Purice]

Hey Jim,
I will test this again tomorrow on the latest version of Nightly and I have a couple of questions that maybe could help me.

1. Did it occur only for the M1 mac or should we take the "Mac M1 specific" off the table and not primarily focus on m1 when testing?
2. Did you have any specific scenario when it occurred or just normal zoom call?
3. After how many minutes did it occur for you since you said it was a long zoom meeting? (I also want to look out for a more specific timeframe)

Re-tested on the latest version of Firefox Nightly 97.0a1 (2021-12-22) with M1 11.6 and MacBook 11.6 in a zoom conference and didn't encounter any issues.

Comment 18

[Jim Mathies [:jimm]]

(In reply to Andrei Purice from comment #17)

Hey Jim,
I will test this again tomorrow on the latest version of Nightly and I have a couple of questions that maybe could help me.

1. Did it occur only for the M1 mac or should we take the "Mac M1 specific" off the table and not primarily focus on m1 when testing?

I was able to get this to happen on a 2018 Mac laptop, not an M1.

2. Did you have any specific scenario when it occurred or just normal zoom call?

No, normal usage of the Zoom web client, sitting in normal meetings.

3. After how many minutes did it occur for you since you said it was a long zoom meeting? (I also want to look out for a more specific timeframe)

Maybe 15 - 20 minutes or so and then it started to cut out.

Re-tested on the latest version of Firefox Nightly 97.0a1 (2021-12-22) with M1 11.6 and MacBook 11.6 in a zoom conference and didn't encounter any issues.

Comment 19

[Paul Adenot (:padenot)]

As far as Firefox is concerned, it looks like it's still recording and processing microphone data.

Ideally, we'd need some more tooling to be 100% sure, but it's somewhat unlikely that suddenly it doesn't work, only in Zoom. I'm also unable to reproduce myself on my 20218 macbook pro 15".

Comment 20

[Darkspirit]

Possibly related: Discord in Nightly, Debian Testing (pipewire-pulse), USB microphone:
My friend told me I would often be sending noise (crackling) in Discord although my avatar doesn't have a green circle around it. The noise would stop when I actually say a word. I assume (!) it occurs roughly since bug 1654112.
Sometimes I also have audio playback degradation (crackling) simply by scrolling websites.

Comment 21

[Paul Adenot (:padenot)]

This bug is strictly a macOS issue afaik.

That said, when you can repro, please take a profile with the Media preset and attach it to a bug, along with about:support's raw data. Please try to include a period of time when you're not talking in Discord and the cracklings can be heard, and then say a word so that they go away, hopefully we'll be able to catch what's up.

Comment 22

[Andrei Purice]

Hello,
QA has ran a few more tests with M1 versus other devices on the latest Firefox Nightly 98.0a1 (2022-01-10) and found out the following :
While connected with Apple earpods (using their own microphone 3.5mm jack) there were no issues with the sound or image capture.
When switching the microphone to the Microsoft HD webcamera (usb jack), after some time (5-10 minutes) robotic/garbled sounds were heard by the others in the conference (to the point where they couldn't understand what I was saying).
This was happening on zoom, linkello, talky, google hangouts and cisco - but this is probably a global issue on all sites.
Also occurred on Firefox release and Firefox beta using M1 mac mini. After some time when switching the microphone from earpods to the camera one the sound has issues.

Attaching a Firefox profiler : https://share.firefox.dev/3ng6o7D -profiler from zoom and https://share.firefox.dev/3zMBZ5V - profiler from talky.
Let us know if further tests are needed.

Comment 23

[Paul Adenot (:padenot)]

Andrei, are you saying that it works well on non-M1 macbooks, but there is a problem on M1 macbooks, with external microphone (a webcam in your case) ?

I'm going to start investigating this on my x86 mac and see if I can repro nonetheless.

Comment 24

[Vicky Chin [:vchin]]

I have experienced this with Google Meet using an external microphone (Yeti) after about 45 minutes on a M1 Max Macbook Pro. If I switch to using the internal microphone (without dropping from the meeting) then the problem is resolved.

Comment 25

[Andrei Purice]

Hey Paul,
Might look like it. Today I have tested on MacBook PRO (non M1) with the same OS version as m1 mini (11.6). The fact is that on this Mac I could only test with earpods and Mac's own microphone input and there weren't any issues. Could not connect the Microsoft HD camera since this doesn't have an usb port.
We tested Facebook/Discord and videolink2me. There were no issues with the MacBook Pro (non M1).
There were issues as well when using Chrome and switching from earpods to the Microsoft HD cam on Mac Mini M1 (on Discord after 15 mins). So even if it took more time this might not be a Firefox issue and maybe an M1 specific one.
There were issues on Firefox Nightly 98.0a1 (2022-01-12) when switching from earpods to the Microsoft HD cam on Mac Mini M1 (same 3 sites usually after 2-3 minutes) as the sound became distorted/garbled/robotic.
Here are 2 new Firefox Nightly profilers : https://share.firefox.dev/3fiCkEg - FB / https://share.firefox.dev/3tiGPXB - Discord

Comment 26

[Paul Adenot (:padenot)]

I know of audio issue on Monterey, but I haven't update my work nor personal machines to this release, they are on Big Sur. Hopefully I can repro on x86, otherwise I'll get an M1. My work laptop is due for a refresh in a few months anyways.

Comment 27

[Paul Adenot (:padenot)]

Andrei, are you running Monterey on any of those machines, if yes, which ones ? Thanks for the detailed testing in any case.

Comment 28

[Andrei Purice]

Unfortunately it's Big Sur on both of them. Version 11.6 on both machines.

Comment 29

[Paul Adenot (:padenot)]

Andrei, can you get me logs for this issue ?

MOZ_LOG=cubeb:5 /path/to/firefox

should show the issue. They can be quite big though. I think I can repro here, but I'm not sure.

Comment 30

[Andrei Purice]

Attachment: new log.rar

Attaching the log

Comment 31

[Paul Adenot (:padenot)]

Huh, this doesn't show any error, was it glitching out? I'm interested in this issue:

There were issues on Firefox Nightly 98.0a1 (2022-01-12) when switching from earpods to the Microsoft HD cam on Mac Mini M1 (same 3 sites usually after 2-3 minutes) as the sound became distorted/garbled/robotic.

Comment 32

[Andrei Purice]

I don't even need to switch from earpods to the Microsoft cam microphone anymore. Just have the cam plugged in directly and the sound will break eventually.

Comment 33

[Paul Adenot (:padenot)]

Ok. Can you make me a recording like so:

MOZ_DUMP_AUDIO=1 MOZ_DISABLE_CONTENT_SANDBOX=1 path/to/firefox --no-remote -profile path/to/test-profile

and get me all the files that this creates, that are called GraphDriverInput-xxx.wav or GraphDriverOutput-xxx.wav ? They can be big as well.

No need to talk, I just need any kind of noise in the mic when it's glitching out, ideally continuous.

I need to understand if what I can reproduce is what you're experiencing.

Comment 34

[Paul Adenot (:padenot)]

Also can you get me the "raw" info of about:support, with the webcam that shows the issue attached ??

Comment 35

[Andrei Purice]

Attachment: raw data.txt

Attaching the raw data from about:support. All files sent.
Today's testing was weird. From time to time the audio broke then it came back and kept on reverting between these states.

Comment 36

[Paul Adenot (:padenot)]

Attachment: Screenshot 2022-01-21 at 16.29.27.png

We were in a call with Andrei today, and I was able to hear what's up. The issue is caused by discontinuities every 1538 frames. 1538 is a sum of power of two (6 * 256), and the discontinuity is precisely 6 frames long.

I'll add some logging and check myself.

Comment 37

[Paul Adenot (:padenot)]

Just an update here to say that this has been delayed by various another macOS issues or other urgent stuff that took precedence, but I have patches working locally.

Comment 38

[Jim Mathies [:jimm]]

We have work in the process of merging currently. Should have fixes in Nightly in the next week.

Comment 39

[Paul Adenot (:padenot)]

(In reply to Andrew Overholt [:overholt] from comment #7)

(In reply to Paul Adenot (:padenot) from comment #6)

One last thing, what audio devices were you both using in this call? For example:

External USB microphone, analog headphones plugged into an external USB speaker or maybe analog headphones plugged directly into 2020 MacBook Air headphone jack (my Jabra speakerphone thing sometimes doesn't work and I can't remember if it worked with hopin.com).

Andrew, can you get me screenshots of the following:

• Open the program "Audio MIDI Setup" on your mac, with the devices plugged in (the samson mic, the jabra mic/speaker), your audio devices should appear in a list:
• We can have two types of setup
  • Two entries in the left-hand side of the panel for a particular device, OR
  • A single entry for a particular device, with a tab selector labeled "Input | Output" on the right-hand side
• Take a screenshot of the windows (either two screenshot per device in the first setup, a single screenshot in the second setup), so that I can look at various info available on this app and make sure I'm fixing what you're experiencing.

I'm interested in whether either of those are considered by macOS by a single audio device with both input and output side, or two devices. I've got a Jabra device locally, but it's a bit older than yours, so I'd like to make sure.

Comment 40

[Andrew Overholt [:overholt]]

Attachment: Screen Shot 2022-03-09 at 11.09.32 AM.png

Comment 41

[Andrew Overholt [:overholt]]

Attachment: Screen Shot 2022-03-09 at 11.09.43 AM.png

Comment 42

[Andrew Overholt [:overholt]]

Attachment: Screen Shot 2022-03-09 at 11.09.50 AM.png

Comment 43

[Andrew Overholt [:overholt]]

Attachment: Screen Shot 2022-03-09 at 11.09.54 AM.png

Comment 44

[Andrew Overholt [:overholt]]

Are those what you need, Paul?

Comment 45

[Paul Adenot (:padenot)]

This is precisely what I needed, thanks, my suspicion was that the Samson mic was in/out, and that's effectively the case, thanks! I should be able to reproduce a setup roughly similar.

Comment 47

[Jim Mathies [:jimm]]

Work here is mostly complete. Most of the changes are landing upstream on github and then will be pulled in. Changes should merge this week.

Comment 49

[Paul Adenot (:padenot)]

Bug 1760774 has merged, so Nightly should behave properly now, let me know.

Comment 50

[georg.hartmann]

Following up on https://bugzilla.mozilla.org/show_bug.cgi?id=1757425 here.
Unfortunately, I can't detect any improvement on Firefox Nightly 100.0a1 (2022-03-24).

The only change can detect is that switching audio sources no longer resets the degradation.
Audio recording will be attached

Comment 51

[georg.hartmann]

Attachment: online-voice-recorder.com on firefox nightly 100.0a1 (2022-03-24)

Comment 52

[Jim Mathies [:jimm]]

(In reply to georg.hartmann from comment #50)

Following up on https://bugzilla.mozilla.org/show_bug.cgi?id=1757425 here.
Unfortunately, I can't detect any improvement on Firefox Nightly 100.0a1 (2022-03-24).

The only change can detect is that switching audio sources no longer resets the degradation.
Audio recording will be attached

Thanks for posting the recording. I think we're dealing with two different issues so I've unduped and reopened bug 1757425. We can carry forward over there.