Closed Bug 1682713 Opened 3 years ago Closed 3 years ago

Application not responsive after wake from sleep or screen lock, low CPU usage during unresponsiveness

Categories

(Core :: Widget: Cocoa, defect, P1)

Product:
Core
Component:
Widget: Cocoa
Version:
Firefox 85
Platform:
All
macOS
Type:
defect

Tracking

()

Status:
VERIFIED FIXED
Milestone:
88 Branch
Tracking Flags:
Tracking Status
relnote-firefox --- 86+
firefox86 + verified
firefox87 + verified
firefox88 --- verified

People

(Reporter: st3fan, Assigned: mstange)

References

(Blocks 1 open bug)

Details

(Whiteboard: [mac:hang])

Attachments

(15 files)

Sample1.txt
136.14 KB, text/plain
Details
Sample2.txt
137.17 KB, text/plain
Details
Sample3.txt
129.45 KB, text/plain
Details
sample1-openAndSavePanelService.txt
95.96 KB, text/plain
Details
sample1-FirefoxCPPrivelegedContent.txt
99.40 KB, text/plain
Details
sample1-FirefoxCPWebContent.txt
99.91 KB, text/plain
Details
sample2-FirefoxCPWebContent.txt
101.54 KB, text/plain
Details
sample3-FirefoxCPWebContent.txt
91.66 KB, text/plain
Details
sample4-FirefoxCPWebContent.txt
99.91 KB, text/plain
Details
sample5-FirefoxCPWebContent.txt
99.91 KB, text/plain
Details
sample1-FirefoxCPWebExtensions.txt
97.03 KB, text/plain
Details
sample1-QuickLooksUIService-com.apple.appkit.xpc.openAndSavePanelServiceFirefox.txt
85.76 KB, text/plain
Details
Bug 1682713 - Recreate the CVDisplayLink after display reconfigurations. r=mattwoodrow
48 bytes, text/x-phabricator-request
jcristau
: approval-mozilla-beta+
pascalc
: approval-mozilla-release+
Details | Review
testcase 1
420 bytes, text/html
Details
testcase 2
212 bytes, text/html
Details

M1 Mac Mini / macOS 11.1 / Firefox Beta 85b1

When the Mac wakes up from sleep, Firefox is unresponsive: windows do not accept input.

Interestingly, menu items work. I can open About Firefox for example.

STR is simple: run Firefox Beta 85b1. Select Sleep from the Apple menu. Wake up the machine.

Any reason to suspect this is specific to ARM?

This might just be a dupe of https://bugzilla.mozilla.org/show_bug.cgi?id=1567018

Component: General → Widget: Cocoa
Flags: needinfo?(sarentz)
Product: Firefox → Core

I suspected this has something to do with ARM because I was using an Intel build of v83 and did not see this issue at all. I updated to an ARM build of v84 and immediately started seeing this. That's also why I suspect this is not the same issue as https://bugzilla.mozilla.org/show_bug.cgi?id=1567018 but I don't know for sure.

Flags: needinfo?(sarentz)

https://bugzilla.mozilla.org/show_bug.cgi?id=1422855 has the same failure mode, from what I can tell.

Unfortunately that would mean the change also includes the v83->v84 update (and it's not stated, but Big Sur was also released in the same timeframe).

We have some other reports of hanging after restarting/resuming but those are on Intel, hence I'm still not sure what to think of this.

What do I do when Firefox hangs? (Beachballs) Are there instructions about how to profile?

I am fairly confident this is an issue only on the arm build.

  • When I run 84.0.1 on my M1 mac mini (ie, Apple Silicon/arm), I see the issue shown above, nearly every time my computer sleeps.
  • When I force the exact same build to run in x86 via Rosetta (in the 'Info' for the app, select 'Open Using Rosetta'), I do not see this issue any more. After the computer wakes up, the Firefox works as expected.

Thanks for the report.

(In reply to matthewjamesadam from comment #7)

I am fairly confident this is an issue only on the arm build.

  • When I run 84.0.1 on my M1 mac mini (ie, Apple Silicon/arm), I see the issue shown above, nearly every time my computer sleeps.
  • When I force the exact same build to run in x86 via Rosetta (in the 'Info' for the app, select 'Open Using Rosetta'), I do not see this issue any more. After the computer wakes up, the Firefox works as expected.

After resume, when the problem is occurring, you can use Activity Monitor to get a sample of the main Firefox process. Open Activity Monitor (from /Applications/Utilities) and double click on the Firefox process named "Firefox". Then click the "Sample" button. It will take a few seconds. Then click "Save..." to save the sample. If you could do this 2 or 3 times and send us the saved samples we may be able to figure out what's happening.

There are instructions for using the Firefox profiler here, but if Firefox is beachballing, you may not be able to turn on the profiler. If you're able to collect a Firefox profile, we'll definitely look at that and you can post the link here.

Can you say if this only happens when sleeping and being plugged into power vs battery?

If it occurs when plugged in, is "Wake for network access" enabled in the power System Preferences?

Flags: needinfo?(matthewjamesadam)

Stefan, if you're able to collect an Activity Monitor sample or profiler sample as described in comment 8, that may help here.

Flags: needinfo?(sarentz)
Attached file Sample1.txt —
Flags: needinfo?(matthewjamesadam)
Attached file Sample2.txt —
Attached file Sample3.txt —

I've attached 3 samples taken after the app stops responding. As you guessed, since the UI is not responsive I can't run the profiler.

This happens while plugged in (this is a mac mini, so no battery). I do have 'Wake for network access' checked in System Preferences.

(In reply to matthewjamesadam from comment #13)

I've attached 3 samples taken after the app stops responding. As you guessed, since the UI is not responsive I can't run the profiler.

This happens while plugged in (this is a mac mini, so no battery). I do have 'Wake for network access' checked in System Preferences.

Thanks. In all the samples, I don't see anything indicating a problem. I've asked some others to take a look too. None of the threads are busy and the main thread in each sample is mostly waiting for events. So we'll have to keep debugging this. I have not been able to reproduce the problem on an M1 MacBook Pro.

In Activity Monitor, would you be able to get Activity Monitor samples from the other Firefox processes e.g. "FirefoxCP Web Content" and others? Especially if any of those have CPU % above 1%. It might not tell us anything, but there's a chance it could be useful.

Does the problem eventually correct itself and Firefox become usable again? If so, you might be able to run the profiler (using the Firefox Platform settings option) and leave it running when you put the computer to sleep. When it wakes up and after Firefox becomes usable again, collect a profile.

Flags: needinfo?(matthewjamesadam)
Flags: needinfo?(matthewjamesadam)

OK I've attached samples for all of the firefox processes shown in the Activity Monitor. None of the processes showed any significant CPU usage.

Firefox does not eventually become usable, ever (I waited about 5 minutes which I assumed was sufficient). One thing I did notice is that when I switch away from the Firefox window, and switch back, sometimes the window content would refresh. I could scroll (via the mouse wheel), which would cause no updating in the window, but if I were to then focus on another window, and re-focus the Firefox window, the browser content would be updated to display wherever I had scrolled to.

This still happens for me on Firefox Beta 85b6.

I simplified the situation a bit and started Beta with a clean profile. No Sync, no data. All I have to do is open Beta, sleep my Mac and then wake it up.

The app still seems to run and is sort off responsive. I can for example open About Firefox from the Firefox menu. But I cannot interact with new tab or the location bar. However the cursor does properly update to a caret when I hoever over the location bar. So this is definitely not a full hang.

I tried to sample with Activity Monitor, but interestingly that only displays kernel_task for me! I cannot see any other processes. Not sure if that is related or if this is some Big Sur bug that I am running into. Will try again.

Flags: needinfo?(sarentz)

Trying Beta 85b6 again and I can't reproduce the same situation. Tried a few times.

Unclear if this is relevant, but for my first succesfull try it took quite some time for the Mac to actually go to sleep. Normally this happens instantly when you select the Sleep menu item. In my case I selected it multiple times and waited quite a long time. Not sure what that means. Could an app hold on to some lock that prevents the OS from sleeping? (I did not get an App Unresponsive warning in the dialog that you get with Option-Command-Escape.)

I have noticed that I am no longer seeing this issue any more either, after seeing it (nearly) every time my machine went to sleep.

I do notice that I am now using Firefox 84.0.2, I guess at some point Firefox auto-updated. Maybe the new build of Firefox fixed this?

I'm not aware of what could have fixed this, but since we cannot reproduce at the moment I will close it as such. Please feel free to reopen should this start happening again.

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → WORKSFORME
See Also: → 1686454

Other people are still seeing this. We should find out which versions of macOS and Firefox this reproduces on, and which ones it is fixed in.

Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---

My previous report was inaccurate -- I am seeing this happen much less, but every now and then this does still happen for me. I just reproduced it by selecting the 'Sleep' menu option, waiting 5 minutes, re-waking my machine and trying to use Firefox.

I am using Firefox 84.0.2 and macOS 11.1 on an M1 mac mini. This does sound like it might be the same issue as 1686454.

Let me know if there is any way for me to get more information for you on this.

I can confirm that I have the issue and it is always reproducible with the following configuration:

Firefox 85.0 64-bit (20210118153634)
macOS Big Sur 11.1 (20C69)
Mac mini (M1, 2020)

Steps to reproduce:

  1. Keep Firefox in focus
  2. Send the Mac mini to sleep
  3. Awake the machine
  4. Firefox does not refresh the entire window after waking up. The only way to refresh the view is to switch the focus after each change and focus the Firefox window again. Restarting Firefox helps

Technically, I can build Firefox locally, attach the debugger and try to figure out what is happening. Please, share hints If you know what to check.

Some additional information from logs. During the Firefox state when it refuses to redraw the window I see the following assertions in the logs:

Acquiring assertion targeting [app<application.org.mozilla.firefox.18192498.18307678(501)>:641] from originator [daemon<com.apple.coreservices.launchservicesd>:114] with description <RBSAssertionDescriptor| "frontmost:641" ID:193-114-4435 target:641 attributes:[
	<RBSDomainAttribute| domain:"com.apple.launchservicesd" name:"RoleUserInteractiveFocal" sourceEnvironment:"(null)">
	]>

Acquiring assertion targeting [app<application.org.mozilla.firefox.18192498.18307678(501)>:641] from originator [daemon<com.apple.WindowServer>:141] with description <RBSAssertionDescriptor| "FUSBFrontmostProcess" ID:193-141-4436 target:641 attributes:[
	<RBSDomainAttribute| domain:"com.apple.fuseboard" name:"Frontmost" sourceEnvironment:"(null)">,
	<RBSAcquisitionCompletionAttribute| policy:AfterApplication>
	]>

Acquiring assertion targeting [app<application.org.mozilla.firefox.18192498.18307678(501)>:641] from originator [daemon<com.apple.coreservices.launchservicesd>:114] with description <RBSAssertionDescriptor| "notification:641" ID:193-114-4437 target:641 attributes:[
	<RBSDomainAttribute| domain:"com.apple.launchservicesd" name:"LSNotification" sourceEnvironment:"(null)">
	]>

Acquiring assertion targeting [app<application.org.mozilla.firefox.18192498.18307678(501)>:641] from originator [daemon<com.apple.WindowServer>:141] with description <RBSAssertionDescriptor| "FUSBProcessWindowState: occluded" ID:193-141-4438 target:641 attributes:[
	<RBSDomainAttribute| domain:"com.apple.fuseboard" name:"Occluded" sourceEnvironment:"(null)">,
	<RBSAcquisitionCompletionAttribute| policy:AfterApplication>
	]>

Acquiring assertion targeting [app<application.org.mozilla.firefox.18192498.18307678(501)>:641] from originator [daemon<com.apple.WindowServer>:141] with description <RBSAssertionDescriptor| "FUSBProcessWindowState: visible" ID:193-141-4442 target:641 attributes:[
	<RBSDomainAttribute| domain:"com.apple.fuseboard" name:"Visible" sourceEnvironment:"(null)">,
	<RBSAcquisitionCompletionAttribute| policy:AfterApplication>
	]>

And these messages for each assertion mentioned above (when OS is trying to acquire them):

Calculated state for app<application.org.mozilla.firefox.18192498.18307678(501)>: running-active (role: UserInteractiveFocal)
[app<application.org.mozilla.firefox.18192498.18307678(501)>:641] Ignoring jetsam update because this process is not memory-managed

[app<application.org.mozilla.firefox.18192498.18307678(501)>:641] Ignoring suspend because this process is not lifecycle managed
[app<application.org.mozilla.firefox.18192498.18307678(501)>:641] Set darwin role to: UserInteractiveFocal
[app<application.org.mozilla.firefox.18192498.18307678(501)>:641] Ignoring GPU update because this process is not GPU managed
[app<application.org.mozilla.firefox.18192498.18307678(501)>:641] Skipping AppNap state - not lifecycle managed
[app<application.org.mozilla.firefox.18192498.18307678(501)>:641] Ignoring jetsam update because this process is not memory-managed

I have managed to reproduce this on my DTK.

Assignee: nobody → mstange.moz
Status: REOPENED → ASSIGNED

(In reply to Markus Stange [:mstange] from comment #34)

I have managed to reproduce this on my DTK.

See the progress made on bug 1415923.

See Also: → 1415923

Thanks! There are so many bug reports filed about similar issues, I wasn't aware of that one.

I think there might be two (or more) different bugs.
There's one case where the process is unresponsive because it is using 100% CPU, chugging through work that built up during the sleep. Firefox eventually recovers.
And then there's another case where there's ~0% CPU and we just don't paint.

I think bug 1415923 was initially about the 0% CPU case, but has now morphed into the 100% CPU case. Let's use this bug for the 0% CPU case.

Blocks: 1422855

I suspect Bug 1686454 is related to either this bug, or bug 1415923

bug 1672791 feels related, as well.

Severity: -- → S2
Priority: -- → P2

I can confirm that this is still happening to me also:
M1 Mac Mini
Big Sur 11.3 beta
Firefox 85.0.2 (64-bit)

Web pages are unresponsive after Mac wakes from sleep. Menu still works so quiting Firefox and opening it again brings Firefox back to life.

Quick question, does it make a difference if you put the Mac into sleep just for a moment and wake up immediately or would it have to stay in sleep mode for a longer time?

This just happened to me again on Nightly. I'm on an M1 Mini, macOS 11.2.1, Nightly of the 17th. I had to join a meeting so I was unable to sample the process. Same symptoms as originally reported:

  • When the Mac wakes up from sleep, Firefox is unresponsive: windows do not accept input.
  • Interestingly, menu items work. I can open About Firefox for example.
Whiteboard: [mac:hangs]
Whiteboard: [mac:hangs] → [mac:hang]

I found a fix for bug 1422855 and I'm hoping it will fix this bug as well. I will make a build soon so that everyone can test the fix.

Summary: Application not responsive after wake from sleep → Application not responsive after wake from sleep or screen lock, low CPU usage during unresponsiveness

Here's a build for testing, which contains the fix for bug 1422855: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NgT_yiiXRy2UnlByOWzF8w/runs/0/artifacts/public/build/target.dmg

For everyone who sees this bug, I'd love to hear if it fixes the problem!

Download and open the dmg, copy the "Firefox Nightly" app to some folder (for example your Desktop) so that you don't overwrite regular Firefox Nightly with it, then right-click the copy and choose Open to get around the warning about it being unsigned.

Update: It looks like the build doesn't actually run on M1 machines because it's unsigned. I'll try to get a signed build.

ok thanks can we expect a fix in firefox 86?

ok thanks can we expect a fix in firefox 86?

bug 1422855 was closed today and Firefox 86 will be released tomorrow. Therefore a fix in Firefox 86 is not possible.

arghhhhhh a fix for firefox 86.0.1? or 86.1...

The latest Firefox Nightly now includes the fix from bug 1422855. Please download it from https://www.mozilla.org/en-US/firefox/channel/desktop/#nightly (or updated your existing Nightly) and let me know if it helps!

If we can get multiple people to confirm that it fixes this bug, I'll try to make a case for uplifting it to 86 for a dot release.

(In reply to Markus Stange [:mstange] from comment #48)

The latest Firefox Nightly now includes the fix from bug 1422855. Please download it from https://www.mozilla.org/en-US/firefox/channel/desktop/#nightly (or updated your existing Nightly) and let me know if it helps!

If we can get multiple people to confirm that it fixes this bug, I'll try to make a case for uplifting it to 86 for a point release.

all language version are ok?

it's semmes to be work, wait and see, i test for 2/3 days before yell victory

at this time it's ok for me !

That's great news! I'd love to hear from more people still.

tracking-firefox86: --- → ?

[Tracking Requested - why for this release]: Now that we seem to have a fix (in bug 1422855), I think it's worth considering a dot release for this. The amount of reports of this bug has increased a lot over the last three weeks, possibly as the result of more people using M1 machines or maybe because a macOS update increased the rate at which this situation occurs.

for me, fix is working very well at this time in the nightly version ! i hope a 86 version with fix very soon .

No longer blocks: 1422855
Depends on: 1422855
OS: Unspecified → macOS
Priority: P2 → P1
Hardware: Unspecified → All
Whiteboard: [mac:hang] → [mac:hang][probably fixed by bug 1422855]
See Also: 1686454 →

is the fix ok in 87 beta version?

I experience the same issue again on Mac mini (Apple Silicon) on v87.0beta but it seemed fixed on the previous beta (no freezes for 3 or 4 days).

ok thanks, the nightly is ok i stay about fix in beta or release version

I'm not certain whether my problem was with high or low CPU usage, but the problem seems to be fixed on my M1 machine with 88.0a1.

me too i hop a version fixed very soon in release channel

(In reply to Artem Loenko from comment #59)

I experience the same issue again on Mac mini (Apple Silicon) on v87.0beta

Thanks, that is concerning. 87.0beta has the patch. So if the bug still happens there, it means it's not completely fixed. Please keep testing and let me know if it happens again.

i'm going to test the beta version (87.b02) and i'll tell you if it's good....or not :D

I'm on 87.b02 and it just happened to me. Not sure about CPU usage, but it froze.

(In reply to Ken W from comment #65)

I'm on 87.b02 and it just happened to me. Not sure about CPU usage, but it froze.

Sorry, M1 Mac mini.

I see. And this happened after sleep?

I'm also interested in the following:

  • Does Firefox eventually recover?
  • Does it beachball during the freeze?
  • Are tab hover effects displayed?
  • Does tab switching work?
  • When the mouse moves over the URL bar, does the cursor change to the text cursor?
  • Does the window respond to resizing?

Furthermore, it would be super helpful to capture a profile during the freeze: Go to https://profiler.firefox.com/ and enable the profiler button, and choose the "Firefox Platform" preset. Once the freeze occurs, start the profiler with Ctrl+Shift+1, let it run for a few seconds while interacting with the frozen browser, and then capture the profile using Ctrl+Shift+2. If the browser eventually unfreezes, the profile will open in a new tab and you can upload it from there. If Firefox stays frozen forever, then we can't get a profile.

Will do, Markus. I'll get a profile next time it happens.

I'm not sure about it eventually recovering. I usually will come back to Firefox after a long while, so I have no idea how long it had been in that state. But quitting it and reopening fixes it. During that frozen state, it actually seems to register activity, like clicking tabs, opening a new tab, etc., and executes that when I restart the browser. Nothing actually responds during the frozen state. I haven't tried the rest you mentioned, but will next time this happens.

Thank you!

i testes beta 87.b02 and at this time i can't see the bug... i'll test again, but for the moment, it's ok for me: mac mini m1

(In reply to Markus Stange [:mstange] from comment #67)

I see. And this happened after sleep?

I'm also interested in the following:

  • Does Firefox eventually recover?
  • Does it beachball during the freeze?
  • Are tab hover effects displayed?
  • Does tab switching work?
  • When the mouse moves over the URL bar, does the cursor change to the text cursor?
  • Does the window respond to resizing?

Furthermore, it would be super helpful to capture a profile during the freeze: Go to https://profiler.firefox.com/ and enable the profiler button, and choose the "Firefox Platform" preset. Once the freeze occurs, start the profiler with Ctrl+Shift+1, let it run for a few seconds while interacting with the frozen browser, and then capture the profile using Ctrl+Shift+2. If the browser eventually unfreezes, the profile will open in a new tab and you can upload it from there. If Firefox stays frozen forever, then we can't get a profile.

On 86, followed the instructions to setup the profiler for platform. When I saw the freeze happen, I attempted to start the profiler, but the only thing that I noticed was that the mouse cursor vanished for a fraction of a second, but no other signs of the profiler running. I interacted with the browser for about 15 seconds, and then attempted to capture the profile, but no luck.

(In reply to matthewjamesadam from comment #24)

OK I've attached samples for all of the firefox processes shown in the Activity Monitor. None of the processes showed any significant CPU usage.

Firefox does not eventually become usable, ever (I waited about 5 minutes which I assumed was sufficient). One thing I did notice is that when I switch away from the Firefox window, and switch back, sometimes the window content would refresh. I could scroll (via the mouse wheel), which would cause no updating in the window, but if I were to then focus on another window, and re-focus the Firefox window, the browser content would be updated to display wherever I had scrolled to.

I confirm that the window does appear to be responding to clicks, but that it's not redrawing it. If I have tab A open and focused on, and tab B open behind, if I click tab B in the tabbar, nothing happens. However, if I minimize or hide the Firefox window, and come back to it, tab B is the visible tab.

Had this issue multiple time the last days. Just upgraded to the latest version of Nightly 88.0a1 (2021-02-25). Still accruing.

Does Firefox eventually recover?

Let it stay frozen for a few hours, not recovering.

Does it beachball during the freeze?

No

Are tab hover effects displayed?

No, not from what I can see.

Does tab switching work?

You can switch tab, but the window in not redrawn until you do a focus change. So select a diffrent application and then back to firefox, the tab will change. This is also somewhat true for changes in the website and menus. Like if Firefox froze on a YouTube video, start and stop works. (Can hear the audio)

When the mouse moves over the URL bar, does the cursor change to the text cursor?

Yes

Does the window respond to resizing?

Yes, can even open new windows (but still broken)

Trying to get a profile: Can start and stop the profiler, but not able to get to the website to share it.

Got it! https://share.firefox.dev/2PdKiET

  1. Start Firefox with MOZ_PROFILER_SHUTDOWN=/Users/olemathias/profiler /Applications/Firefox\ Nightly.app/Contents/MacOS/firefox-bin
    (change profile path)
  2. Open a few tabs
  3. Wait for the freeze (this time around 15 min)
  4. Start the profiler with Ctrl+Shift+1
  5. Navigate around
  6. Close Firefox

(In reply to olemathias.aa.heggem from comment #73)

Apologies for asking a silly question, but according to the info in the top left corner, that's running on 10.15, not 11/10.16, which is the first M1-compatible public build. Is that a glitch in the profiler or is this a different bug? (Quite possible I'm in the wrong thread, but I checked and the reporter upthread is referring to Fx on ARM.)

(In reply to Varun from comment #74)

That must be a glitch in the profiler, running Mac Mini M1 on 11.2.1 (20D74)

% sw_vers
ProductName:	macOS
ProductVersion:	11.2.1
BuildVersion:	20D74

% system_profiler | grep "Model Identifier"
Model Identifier: Macmini9,1

Also tested by doing a new profiler, still showing Firefox 88 – macOS 10.15

That probably comes from the user agent string, which is capped at 10.15, see bug 1679929.

After 2 days, it's ok for me! mac mini m1, no freeze .

(In reply to Markus Stange [:mstange] from comment #67)

I see. And this happened after sleep?

I'm also interested in the following:

  • Does Firefox eventually recover?
  • Does it beachball during the freeze?
  • Are tab hover effects displayed?
  • Does tab switching work?
  • When the mouse moves over the URL bar, does the cursor change to the text cursor?
  • Does the window respond to resizing?

Furthermore, it would be super helpful to capture a profile during the freeze: Go to https://profiler.firefox.com/ and enable the profiler button, and choose the "Firefox Platform" preset. Once the freeze occurs, start the profiler with Ctrl+Shift+1, let it run for a few seconds while interacting with the frozen browser, and then capture the profile using Ctrl+Shift+2. If the browser eventually unfreezes, the profile will open in a new tab and you can upload it from there. If Firefox stays frozen forever, then we can't get a profile.

Ok, more info for you since it's currently frozen.

  • It doesn't seem to recover itself
  • No beachball
  • Tab hover effects do not work, but when clicking each tab, the title bar changes to that tab's page title. You don't actually get switched to the tab, though.
  • The cursor does change to a text cursor when hovering over the URL bar
  • Window resizing does work

Sorry I can't run the profiler. I didn't do the prerequisite for it beforehand, so nothing can be done with the window frozen.

Hope this helped!

update to macos 11.2.2 today, firefox 87.b03 and no freeze all is all right for me

Updated to macOS 12.2.2 and firefox 87.b03 and it is still freezing on me.

Sorry, that should be macOS 11.2.2. But it is still freezing.

Thanks for the reports everybody. Sounds like there's more work to do.

The observations, and the profile from comment 73 (nice work!), indicate that the CVDisplayLink is indeed permanently paused.
I am planning to land some logging code in Nightly, and then we can see if that sheds any light on this issue.

(In reply to Markus Stange [:mstange] from comment #82)

Thanks for the reports everybody. Sounds like there's more work to do.

The observations, and the profile from comment 73 (nice work!), indicate that the CVDisplayLink is indeed permanently paused.
I am planning to land some logging code in Nightly, and then we can see if that sheds any light on this issue.

I haven't been running nightlies, but if there's one that has the code you mentioned, I'm happy to run it to help track down the issue.

I have managed to reproduce this after all. I need to send my DTK back soon but for now I've been able to do some debugging. My steps to reproduce are:

  1. Open a page that paints periodically but leaves large enough gaps between the individual paints that we stop listening for vsync in between, i.e. at least 100-200ms. For example this page: data:text/html,<script>x=0;setInterval(() => document.body.style.backgroundColor = (x=1-x) ? "green" : "blue", 500)</script>
  2. Put your machine to sleep using the menu.
  3. Wait until you don't hear any noise from it anymore.
  4. Wait for 20 seconds. If it starts making noise again, wait until it stops again and wait another 20 seconds.
  5. Wake up the machine.

I've found a fix. It's very similar to the other one, except that I have to destroy and recreate the CVDisplayLink rather than just stopping and starting it.

Technical description of what happens:

During wake-up, there's a "display double-swap" similar to what happens during user switching: The regular display is swapped for a temporary display, and then that temporary display is swapped again for the regular display. Example:

First swap:
[1] DisplayReconfigurationCallback for 1: BeginConfiguration
[2] DisplayReconfigurationCallback for 21: BeginConfiguration
[3] DisplayReconfigurationCallback for 1: Remove Disabled DesktopShapeChanged
[4] DisplayReconfigurationCallback for 21: Moved SetMain SetMode Add Enabled DesktopShapeChanged
Second swap:
[5] DisplayReconfigurationCallback for 21: BeginConfiguration
[6] DisplayReconfigurationCallback for 1: BeginConfiguration
[7] DisplayReconfigurationCallback for 21: Remove Disabled DesktopShapeChanged
[8] DisplayReconfigurationCallback for 1: Moved SetMain SetMode Add Enabled DesktopShapeChanged

Here 1 is the regular display and 21 is the temporary display. These two swaps happen within one second of each other, during wake-up.

(Edited for correctness.)
During user switching, the first swap happens when switching away, and the second swap happens when switching back. In bug 1422855, we would create a new display link between [4] and [5], and then that CVDisplayLink would stay assigned to the temporary display and get stuck after [8].
In this bug, we try to create a CVDisplayLink just after sleep, but before the swaps happen. The CVDisplayLink object created before [0] does not assign itself to any display, and CVDisplayLinkGetCurrentCGDisplay returns 0 at that time. So we hit this fallback code that keeps creating new display links every 100ms until it succeeds.

It doesn't succeed before [1]. But once the first swap happens, it does succeed; the timer can fire between [2] and [3], or between [4] and [5], and create the display link at that point. However, now that display link's is assigned to the temporary display. So now we're in the same situation as in bug 1422855: The display link stays assigned to the temporary display, and is stuck with it even after [5]-[8] are done.

However, here's where the difference comes in: It seems that on Apple Silicon machines, stopping and starting the display link is not enough to get the it re-assigned to the regular display. So the display link stays assigned to the dead display and does not fire any vsync notifications. And, unlike on Intel machines, the stuck display link never recovers.
But creating a new display link object at this point works, and it correctly uses the regular display. So that's what I'm going to do.

For a proper fix, we should stop using CVDisplayLinkCreateWithActiveCGDisplays. Instead, we should create per-display CVDisplayLink objects, and keep track of which displays are active and only have CVDisplayLinks for those.

Another finding: It turns out that, on Apple Silicon machines, the patch from bug 1422855 didn't actually fix the user switching freeze. I can still reproduce the freeze with the steps from bug 1415923 comment 116 on Nightly builds with the fix from bug 1422855, both on the DTK and on my new M1 Macbook Pro. It seems that stopping and starting the display link just doesn't cause it to re-assign itself to an active display on Apple Silicon machines. I've edited the previous comment to mention that.

I've started using only my M1 Macbook Pro's 13" laptop screen for the last 4 days, and have not had this issue occur at all. However, before that it was always connected to an external Pro Display XDR, and it happened several times a day. (macOS 11.2.1, firefox 86.0). More details are in https://bugzilla.mozilla.org/show_bug.cgi?id=1684322

This just reproduced on 88.0a1 on a M1 Macbook Air. Exiting to the lock screen and back did not help, but exiting to the lock screen, then to "switch user", and back to the same user made Firefox responsive again. Indeed multiple attempts to open new tabs must have been queued up because several new tabs appeared at that time.

Still the case for Mac mini M1 and Firefox v87.0b3 (64-bit) with the symptoms as well. This is a Firefox process sample if it helps.

it's crazy, i haeven't the problem with macmini and 87.0b3

On M1 machines, the fix from bug 1422855 was not effective; stopping and starting the
CVDisplayLink is not enough to cause it to switch to an active display. So we create
a new display link, which will pick an active display.

This patch is written to be minimum impact and easy to uplift. More cleanup is needed in the future.

Whiteboard: [mac:hang][probably fixed by bug 1422855] → [mac:hang]
Attached file testcase 1 —
Attached file testcase 2 —

Thanks everyone for the reports. I don't need any further reports for now. Once the new patch lands in Nightly, you can test again.

Pushed by mstange@themasta.com:
https://hg.mozilla.org/integration/autoland/rev/ab3f57381f67
Recreate the CVDisplayLink after display reconfigurations. r=mattwoodrow

Further to my previous comment, I just experience the lock-up again on 88.0a1, but switching to the lock screen etc. didn't resolve the issue and I had to restart Firefox.

https://hg.mozilla.org/mozilla-central/rev/ab3f57381f67

Status: ASSIGNED → RESOLVED
Closed: 3 years ago → 3 years ago
status-firefox88: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 88 Branch

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

status-firefox87: --- → ?

I'll gladly use a test build as soon as the next build to try is announced. I'm following the ticket now so I'll get notification. Thanks!

The fix is now in Firefox Nightly. Please update and let me know if it works!

Comment on attachment 9206002 [details]
Bug 1682713 - Recreate the CVDisplayLink after display reconfigurations. r=mattwoodrow

Beta/Release Uplift Approval Request

  • User impact if declined: Unresponsive Firefox process on M1 Macs after fast user switching and system sleep
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: Scenario A: Fast user switching (reproducible on all M1 Macs, and before bug 1422855, also on Intel Macs):
  1. Make sure you have two user accounts configured in macOS, and both accounts have "Fast user switching" enabled.
  2. On one account, open Firefox, load attachment 9206004 [details], and keep it open in a foreground tab.
  3. Using the system status bar fast user switching menu in the top right corner of the screen, switch to the other account.
  4. Switch back to the original account the same way.

At this point, Firefox would be frozen.
Bug 1422855 fixed this bug for Intel Macs and this bug fixed it for M1 Macs.

Scenario B: After system sleep (reproducible on M1 Mac Mini, and for some users on M1 Macbook Pro with an external screen):

  1. Open Firefox, load attachment 9206004 [details], and keep it open in a foreground tab.
  2. Send the machine to sleep using [Apple menu] -> Sleep.
  3. Wait until the machine has become fully silent. Wait 20 more seconds. If the machine makes any noise whatsoever, wait until it's fully silent again and wait 20 more seconds.
  4. Wake the machine from sleep.

At this point, Firefox would be frozen, before this bug's fix.

  • List of other uplifts needed: bug 1422855 (already present on 87, but would need to be uplifted to 86)
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Very tightly-scoped fix.
  • String changes made/needed: none
Attachment #9206002 - Flags: approval-mozilla-release?
Attachment #9206002 - Flags: approval-mozilla-beta?
Flags: qe-verify+

Nightly is now working for me. Using 88.0a1 (2021-03-03) . Both scenario A and B worked. This is on a Mac Mini M1. Previous versions had always locked up doing Scenario B. (Hadn't tried scenario A before.) Tried scenario B now multiple times and not seeing problems.

I've installed and am using 88.0a1, I'll report back tomorrow.

Mac Mini M1 2020 running macOS Big Sur 11.2.1
I just tried using Nightly 88.0a1 (2021-03-03) (64-bit). It works for me: after the comp reawakens from sleep, the Nightly windows don't freeze.

QA Whiteboard: [qa-triaged]

Hello! We tried reproducing the issue but unfortunately, we can only reproduce scenario A from comment 100. Tried Scenario B multiple times on mac mini M1 machines but with no luck.

The issue has been verified on both scenarios from comment 100 on Mac mini M1 11.2.2 with Firefox 88.0a1 (20210303215027). The freeze is not occurring after switching the user with fast with enabled and after waking up firefox from sleep.

Jonathan Morgan, can you please confirm as well both scenarios from comment 100 using the latest Nightly 88.a01 to be extra sure? Thank you!

Flags: needinfo?(jonathan.morgan.007)

Comment on attachment 9206002 [details]
Bug 1682713 - Recreate the CVDisplayLink after display reconfigurations. r=mattwoodrow

approved for 87.0b6

Attachment #9206002 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

(In reply to Alexandru Trif, QA [:atrif] from comment #104)

Hello! We tried reproducing the issue but unfortunately, we can only reproduce scenario A from comment 100. Tried Scenario B multiple times on mac mini M1 machines but with no luck.

I'm curious, can you reproduce it if you try Scenario B with attachment 9206003 [details] (testcase 1) instead of attachment 9206004 [details] (testcase 2)?

Flags: needinfo?(alexandru.trif)

Release Note Request (optional, but appreciated)
[Why is this notable]: Fixes a rather severe bug that was reported by many users
[Affects Firefox for Android]: no
[Suggested wording]: macOS: Fixed an issue on Apple Silicon machines that caused Firefox to be unresponsive after system sleep.
[Links (documentation, blog post, etc)]: n/a

relnote-firefox: --- → ?

Good evening, I tested repeatedly, neither scenario A nor scenario B result in Firefox freezing up with Firefox Nightly 88.0a1 (2021-03-03).

I am on M1 Mac mini running macOS 11.2.2.

Flags: needinfo?(jonathan.morgan.007)

Also, no problems with 9206003 with Firefox Nightly 88.0a1 (2021-03-04).

(In reply to Markus Stange [:mstange] from comment #107)

I'm curious, can you reproduce it if you try Scenario B with attachment 9206003 [details] (testcase 1) instead of attachment 9206004 [details] (testcase 2)?

Nope, using this test case I cannot reproduce either of the scenarios. The test case from comment 100 reproduces the switch user scenario every time. I will change flags on verified for Nightly 88.0a1 based on comments 109, 104, 103, and 101. Thank you all for your verification.

I have also tried both scenarios today using 87.0b6 (20210304190020) and no freezes occur.

Jonathan could you please give another spin using the latest Firefox 87.0b6 as well on both scenarios? Thank you so much for your support!

status-firefox88: fixed → verified
Flags: needinfo?(alexandru.trif) → needinfo?(jonathan.morgan.007)

sure, will test now.

On Firefox 87.0b6 I tested scenarios A and B with attachment 9206004 [details], and then each scenario with attachment 9206003 [details] and had no problems. All looks good to me on 87.0b6. Please let me know if there is anything else I can do to help.

Flags: needinfo?(jonathan.morgan.007)

(In reply to Jonathan Morgan from comment #113)

On Firefox 87.0b6 I tested scenarios A and B with attachment 9206004 [details], and then each scenario with attachment 9206003 [details] and had no problems. All looks good to me on 87.0b6. Please let me know if there is anything else I can do to help.

Thank you again, Jonathan! For now, this is all.

Based on comment 113 and comment 111 the issue is verified fixed with 87.0b6. Leaving qe+ flag for now.

status-firefox87: fixed → verified

can we expect a patch for firefox 86?

Added to the 87.0beta release notes.

relnote-firefox: ? → 87+

Comment on attachment 9206002 [details]
Bug 1682713 - Recreate the CVDisplayLink after display reconfigurations. r=mattwoodrow

Approved for 86.0.1, thanks.

Attachment #9206002 - Flags: approval-mozilla-release? → approval-mozilla-release+

Added to the 86.0.1 relnotes.

relnote-firefox: 87+ → 86+

Verified Scenario A and B several times with Firefox 86.0.1 (20210310152336) on M1 mac mini 11.2.3 using steps from comment 100 and I cannot reproduce the issue by using the comment 100 test case and attached test case 1 and test case 2.

Jonathan given the fact that you could reproduce both scenarios from comment 100 could you please try to verify with Firefox 86.0.1 (link) to make sure that the fix is working as expected on release? Thank you in advance!

Flags: needinfo?(jonathan.morgan.007)

missed the notification, will test now.

On Firefox 86.0.1 I tested scenarios A and B with attachment 9206004 [details], and then each scenario with attachment 9206003 [details] and had no problems. All looks good to me on 86.0.1. Please let me know if there is anything else I can do to help.

Flags: needinfo?(jonathan.morgan.007)

(In reply to Jonathan Morgan from comment #124)

On Firefox 86.0.1 I tested scenarios A and B with attachment 9206004 [details], and then each scenario with attachment 9206003 [details] and had no problems. All looks good to me on 86.0.1. Please let me know if there is anything else I can do to help.

Nope, that will be all. Thank you so much for your help and your support.

Closing this. Thank you!

Status: RESOLVED → VERIFIED
status-firefox86: fixed → verified
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: