Closed Bug 1524193 Opened 5 years ago Closed 5 years ago

Flaky sound on youtube and overcast.fm

Categories

(Core :: Audio/Video: Playback, defect, P2)

Product:
Core
Component:
Audio/Video: Playback
Version:
67 Branch
Type:
defect

Tracking

()

Status:
VERIFIED FIXED
Milestone:
mozilla67
Tracking Flags:
Tracking Status
firefox-esr60 --- unaffected
firefox65 --- unaffected
firefox66 --- unaffected
firefox67 --- verified

People

(Reporter: rinkevics, Assigned: mconley)

References

Details

(Keywords: regression)

Attachments

(4 files)

trobleshoot_data.txt
22.03 KB, text/plain
Details
systeminfo.txt
8.50 KB, text/plain
Details
Bug 1524193 - Make audio wakelocks put background tabs into the normal priority bucket. r?gsvelto
47 bytes, text/x-phabricator-request
Details | Review
about_rawdata.json
45.27 KB, application/json
Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

Listen to sound on https://overcast.fm/+FgGOsF_P8 and youtube.com
Using firefox nightly

Actual results:

When CPU usage was high due to activity by other applications, sound was intermittently disappearing/appearing.

Expected results:

There should not be interruptions in the sound.

Attached file trobleshoot_data.txt —

Is this something new with the nightly or do you have the same problem with Firefox64/65 release builds ?

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core

This happens only on nightly. Release 65 works fine.

Could you try to find the change in Firefox that caused this with the mozregression tool ?
That tool downloads a few nightly builds and let you test them.

Flags: needinfo?(rinkevics)

will try that

Flags: needinfo?(rinkevics)

Any ideas Paul?

Flags: needinfo?(padenot)

Regression tool does not work because of proxy connection issues.

Can repro, mozregression shows that the problem is bug 1476981

With dom.ipc.processPriorityManager.enabled set to false the problem dissapears

Blocks: 1476981
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: regression
Depends on: 1523974
Flags: needinfo?(padenot)

Setting priority here although it looks like the fix will happen in Bug 1523974.

Rank: 15
Priority: -- → P2

Hi rinkevics - we just landed a patch in Nightly that slightly raises the process priority of background tabs that are playing audio. Do you still hear the dropped audio if you try to reproduce in today's Nightly?

Flags: needinfo?(rinkevics)

Build ID 20190210213844
User Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0

I can still reproduce the issue.

Thanks, Alice. How about with this try build?:

32-bit: https://queue.taskcluster.net/v1/task/a6_aMdKgR-CROx3z48zPsA/runs/0/artifacts/public/build/target.zip

64-bit: https://queue.taskcluster.net/v1/task/QLycC4IDSPCAFyqM2xVi4w/runs/0/artifacts/public/build/target.zip

padenot ^-- perhaps you can see if these also pass the test you ran earlier?

Flags: needinfo?(padenot)
Flags: needinfo?(alice0775)

(In reply to Mike Conley (:mconley) (:⚙️) from comment #14)

Thanks, Alice. How about with this try build?:

32-bit:
https://queue.taskcluster.net/v1/task/a6_aMdKgR-CROx3z48zPsA/runs/0/
artifacts/public/build/target.zip

64-bit:
https://queue.taskcluster.net/v1/task/QLycC4IDSPCAFyqM2xVi4w/runs/0/
artifacts/public/build/target.zip

padenot ^-- perhaps you can see if these also pass the test you ran earlier?

Yes, the try build fixed the issue.

Flags: needinfo?(alice0775)

The issue persists in 67.0a1 (2019-02-11)
But the issue is also observed in the latest public release.
I sometimes have the same issue (sound disappearing intermittently) in Chrome but much much less.

Flags: needinfo?(rinkevics)

(In reply to rinkevics from comment #17)

This build is the same:
32-bit: https://queue.taskcluster.net/v1/task/a6_aMdKgR-CROx3z48zPsA/runs/0/artifacts/public/build/target.zip
64-bit: https://queue.taskcluster.net/v1/task/QLycC4IDSPCAFyqM2xVi4w/runs/0/artifacts/public/build/target.zip

Will try to run same workload on another, more powerful PC tonight.

What is your machine? On a release build, sound should be absolutely flawless, so this indicate another possible issue.

Flags: needinfo?(padenot) → needinfo?(rinkevics)
Attached file systeminfo.txt —
Flags: needinfo?(rinkevics)

I attached systeminfo output. Do you need anything else?

I also have Kaspersky antivirus is quite heavy.

(In reply to Mike Conley (:mconley) (:⚙️) from comment #14)

Thanks, Alice. How about with this try build?:

32-bit: https://queue.taskcluster.net/v1/task/a6_aMdKgR-CROx3z48zPsA/runs/0/artifacts/public/build/target.zip

64-bit: https://queue.taskcluster.net/v1/task/QLycC4IDSPCAFyqM2xVi4w/runs/0/artifacts/public/build/target.zip

padenot ^-- perhaps you can see if these also pass the test you ran earlier?

As I mentioned briefly on IRC, my slow machine that I used to test this doesn't boot anymore, so I'm afraid I can't test this. I hope to have another machine soon.

However, it should fix it, based on reading some documentation.

It's perfect, thanks. (In reply to rinkevics from comment #20)

I attached systeminfo output. Do you need anything else?

No, it's perfect, thanks.

Pushed by mconley@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/cea0ec7278c1
Make audio wakelocks put background tabs into the normal priority bucket. r=gsvelto

https://hg.mozilla.org/mozilla-central/rev/cea0ec7278c1

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox67: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Assignee: nobody → mconley
Flags: qe-verify+

Managed to reproduce the issue; however when trying to verify the fix, the flaky-ness reappeared on macOS(10.13).
Fwiw, on Ubuntu the issue did not seem to appear.
As suggested by :mconley on Slack, will open a separate bug for the above mentioned issue(bug 1546064).

Fix verified with 67.0b12 Windows 10 - 20 minutes session of trying to reproduce.

Status: RESOLVED → VERIFIED
status-firefox67: fixed → verified
Flags: qe-verify+

I'm experiencing this issue on Firefox Nightly 75.0a1 (2020-03-02) (64-bit) on SoundCloud. This particularly hits when the host machine is under load (eg building .net solution)

Opening process explorer and finding the process for this tab, I can see that it's set to low-priority. Raising to "High" makes music buttery-smooth again.

(In reply to Davyd McColl from comment #28)

I'm experiencing this issue on Firefox Nightly 75.0a1 (2020-03-02) (64-bit) on SoundCloud. This particularly hits when the host machine is under load (eg building .net solution)

Opening process explorer and finding the process for this tab, I can see that it's set to low-priority. Raising to "High" makes music buttery-smooth again.

Davyd, this is windows I assume? Mike, anything changed on your side of things?

Flags: needinfo?(mconley)
Flags: needinfo?(davydm)

Paul, correct, Windows 10, build 1909 (OS Build 18364.657)

Flags: needinfo?(davydm)

Hi David,

Do you have Fission enabled?

Flags: needinfo?(mconley) → needinfo?(davydm)

Not on my windows machine at the moment, but
(a) only just learned about (non-nuclear) Fission
(b) doesn't seem to be enabled by default on my Gentoo box
(C) there are about 10 flags I see in about:config when I search for Fission - what should I enable?

Flags: needinfo?(davydm) → needinfo?(mconley)

If you've not enabled Fission manually yourself, then I don't think we need to worry about it - I wanted to check if you'd opted into testing it.

Are you able to reproduce the same issue on Firefox Beta or Release?

Flags: needinfo?(mconley) → needinfo?(davydm)

I've just installed Firefox 73.0.1 (64-bit) and I'll see how that goes today. A question though: Firefox nightly shows the PID of the tab in the tooltip for the tab, so it's easy to find in, eg Process Explorer (which is where I saw that my soundcloud tab was set to low priority). Firefox current doesn't do this -- is there a way to figure it out? about:performance doesn't seem to show me, and, with one tab open, Process Explorer shows 4 Firefox processes, so I'm not sure which one relates to the tab?

Flags: needinfo?(davydm)

Ok, so, after running for a while, and figuring out I could just watch process explorer to see which new process a new tab belonged to:

  1. I have noticed stuttering under heavy load with firefox stable
  2. I see that process for the new tab I opened is running backgrounded (https://postimg.cc/mPzKqHQ5)

Would you like me to install and test with beta too?

Blocks: 1621684

Has anyone had a moment to look into this? Listening to SoundCloud today was really frustrating, whilst WebStorm and Rider were eating up CPU. The only resolution was to manually bump the tab's process priority via ProcessExplorer, but that doesn't stick, so I have to periodically bump process priority.

Please can this get some <3 ?

(In reply to Davyd McColl from comment #36)

Has anyone had a moment to look into this? Listening to SoundCloud today was really frustrating, whilst WebStorm and Rider were eating up CPU. The only resolution was to manually bump the tab's process priority via ProcessExplorer, but that doesn't stick, so I have to periodically bump process priority.

Please can this get some <3 ?

Mike, I thought you had fixed this and written a test, why are we still seeing this? Davyd, please attach the result of going to about:support and getting the raw data with the button for this at the top of the page.

Flags: needinfo?(mconley)
Flags: needinfo?(davydm)
Attached file about_rawdata.json —

about:support data, as requested.

Flags: needinfo?(davydm)

(In reply to Paul Adenot (:padenot) from comment #37)

Mike, I thought you had fixed this and written a test

I had, and I did.

why are we still seeing this?

Unsure. Davyd, do you have some reliable steps-to-reproduce where the background tab playing audio has lower process priority?

Flags: needinfo?(mconley) → needinfo?(davydm)

Power just went out in my area, so please bear with a short response from my phone.

I just noticed sound stuttering today, opened process explorer and set priority to real-time -stuttering stopped (big hammer, I know). Switched away, and a little time later looked again at process explorer - the set-priority submenu marks the current process priority and I noticed it was back down to idle/background, so I boosted it again.

Flags: needinfo?(davydm)

And this background tab was running SoundCloud?

Flags: needinfo?(davydm)

Correct, with audio playing.

Flags: needinfo?(davydm)

Hey Rares,

Is it possible for someone at SV to help try to reproduce Davyd's issue / generate some STR? I'm having no luck reproducing locally. If we can find some STR, then let's get a new bug on file and get this sorted.

Flags: needinfo?(rares.bologa)
Flags: needinfo?(rares.bologa) → needinfo?(clara.guerrero)

Hi Mike

I think I may have steps to repro (at least, two or three runs though, this seemed to work for getting the reduced priority, however, after a few more attempts, it stopped, so perhaps there's some kind of race condition I was "lucky" to encounter earlier, and which I manage to encounter at least once or twice a day?)

  1. SoundCloud is running pinned, playing music
  2. Open one or more new, empty tabs
  3. Navigate a new tab to www.google.com
  4. Switch away to another window (in my case, it was WebStorm)
  5. Switch back
  6. Close the new tab with www.google.com in it

after doing this, the process priority for my soundcloud tab had been bumped to background / idle

It's not a perfect repro tho -- like I say, this worked around 3 times in a row, and then I couldn't get it to happen again ): Is there perhaps some code which, upon closing a tab, attempts to figure out what else could be cleaned up, and potentially adjusts process priorities?

All priority adjustments happen in the ProcessPriorityManager. It relies upon a bunch of events to decide when to lower or raise the priority of a given process. BTW how many tabs did you had open when you reproduced the issue?

About 7 pinned tabs and anywhere between 1 and 20 regular tabs. I often "close all to right" when tabs start being scrollable.

Alright so we might have more than one tab per process. The process priority manager was originally designed for Firefox OS and thus assumed that we had always only one tab per process. It might be that its logic is buggy when more than one tab is shared by the same process.

I'll reduce the number of content processes in my local setup and try again to reproduce.

Hey Davyd,

I've tried your STR, and a number of alternative configurations, and I'm still not able to reproduce this.

Would you be able to do and post a screen capture so that we can see you walking through your steps and reproducing the bug? That might make it easier for me to see what I'm missing here.

Flags: needinfo?(davydm)

Hi Mike

I'm not even sure that will help; as noted above, those steps to repro worked 3 times in a row, and then didn't.

If I find something that works every time, I'll post it. In the mean-time, since I also have a day-job, I have to leave my home-grown renice util in watch mode to bump the tab's priority on a regular basis. I think I'll add logging to that -- it might help to pinpoint when the issue occurs.

Flags: needinfo?(davydm)

something (perhaps related) which I've just observed & I found interesting:

I have FF running, some pinned tabs (incl SoundCloud). I have a tab open with this bug report in it (not pinned). That tab remains focused whilst I moused over the tab header for the SoundCloud tab to get the PID, with ProcessExplorer open. I notice that as I mouse over, the tab priority jumps from 4 to 8 (Background to Normal), and then the tooltip with the PID appears. If I mouse-out, the tab drops back down to 4 (Background). Could whatever is deciding on these changes be interfering with your fix?

(note that if sound is playing, I don't see the above; it's just an interesting behavior that I wouldn't have expected, and I have to wonder if that automatic drop-down isn't related in some way; I'm probably grasping at straws tho)

I know I'm probably being annoying now, but I'd like to propose that tabs playing audio should in fact have their priority bumped to above normal. Hear me out:

I've experienced a couple of times now where sound flakes out a bit when WebStorm / Rider starts cranking up to get stuff done -- when I'm seeing 100% core usage. Bumping the priority of a tab to above-normal solves this: the tab is never interfered with by a busy system, even one at 100% cpu. A lot of music playing programs have this option because it's pretty annoying to have music (which doesn't require a lot of processing, really) be interrupted by other hungry tasks.

In other words, I don't believe it's enough to simply not drop process priority for tabs playing media -- I think that Firefox should bump the priority for a better user experience when the system is under load. The user obviously chose to play music (or a music video, or whatever), and if they didn't want that getting priority, they could stop. Right now, I'm running my own cli app to bump priority and keep it bumped so that the times when my system is under load don't cause hiccups in music, much like that load doesn't interfere with other interactive tasks like the mouse cursor or display update, or sound from any other system.

Thoughts?

Flags: needinfo?(mconley)
Flags: needinfo?(gsvelto)

Raising priority above the normal level is impossible to do cross-platform because it requires privileges that the user doesn't have (e.g. root access to use negative nice values on Linux, admin access to use higher-than-normal priorities on Windows). Our audio decoding code already employs a few tricks to avoid glitches - given the priority of the process is not demoted - so that's the process we should fix. I apologize if I didn't have time to look yet but my swelling backlog became bigger with the recent round of layoffs.

Flags: needinfo?(gsvelto)

Understandable -- even as I was writing, I was wondering about privilege ): I'm quite sad about the recent round of Mozilla layoffs. Sorry for adding more on your plate )':

Flags: needinfo?(mconley)

Hi,
I'm also not being able to reproduce yet, following steps mentioned in comment 44, with dom.ipc.processPriorityManager.enabled both false and true. I'm using windows 10 pro, tested in nightly 81.0a1 (2020-08-20) (64-bit)
If you can manage to post a screen capture it would be very helpful (even though you were able to reproduce just those 3 times as you said).
Let me know if I can help in any other way.
Best,
Clara

Flags: needinfo?(clara.guerrero)

Yeah, frustratingly, I also haven't been able to repro since those 3 times. Perhaps I was imagining the whole thing ):

-d

This appears to be back, unfortunately: I noticed choppy audio from SoundCloud, looked up the pid of the tab and found that the process was running at Idle priority (win10). I manually boosted it, but I see that it's been popped back down to Background after alt-tabbing out of Firefox and then coming back in again to write this report.

This time it's much easier to reproduce: I made a recording here: https://recordit.co/cjpwNbjbrN

  • playing sound on SoundCloud: the tab starts out as background priority (even when focused)
  • I up the priority to Normal, switch to another tab, switch back to Process Explorer: tab priority is back at Background
  • I up the priority there to Normal again, switch back to the tab... and it's back as Background

When using Firefox to play music in the background, whenever my programming tooling starts getting a little active, sound gets really choppy - understandable since the tab has been prioritised as Background (and I've seen it drop down to Idle too)

Could be fallout from bug 1618547. Could you please open a new bug and needinfo? :mccr8 who made the last few changes to the process priority manager?

Flags: needinfo?(davydm)

I've created the new bug here: https://bugzilla.mozilla.org/show_bug.cgi?id=1715502 (hopefully I've mentioned people properly)

Flags: needinfo?(davydm)
See Also: → 1715502

(In reply to Davyd McColl from comment #61)

I've created the new bug here: https://bugzilla.mozilla.org/show_bug.cgi?id=1715502 (hopefully I've mentioned people properly)

Yes, thank you!

shameless plug for anyone this affects, on windows: I wrote a little "renice-like" app to work around this (wrote this around when I opened the original issue). It's available here: https://github.com/fluffynuts/renice if anyone wants to work around the problem until it's fixed. Yes, I know the binary release is huge - that's what standalone .net core binaries look like ;_;

hope this helps someone else.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: