Bug 1883040 Comment 0 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

I've managed to reproduce an extremely specific hang scenario on our MacOS QoS work. To create this hang, the system must be operating at long-term very high load with extremely busy threads spinning at the highest priority. Due to the resource starvation, lower priority threads than the |User-interactive| level get scheduled >60 seconds to never from the time they receive work.

The hang scenario operates as such:
- The priority handler receives a signal to set QoS priority to background level. Because this thread handles tab switch IPC and therefore is run at the highest priority at all times, it is able to run this.
- An override is sent in low priority, overriding the main thread's existing priority to low. The main thread now may not be scheduled indefinitely.
- The command sent to the main thread to set its priority and remove the override is therefore scheduled at the lowest priority.
- This command is never scheduled to run, due to extreme resource starvation leading to the lowest priority work being postponed indefinitely.

In this scenario the main thread gets stuck setting itself. While this scenario is very rare and I had to make a conscious effort to produce it, I still would like to plug this hole. This is as easy as not setting an override when the intended priority is background-level, as we aren't dedicated to the timeliness of setting this priority. We only want to use the override to warm up the main thread, not to cool it down.
I've managed to reproduce an extremely specific hang scenario on our MacOS QoS work. To create this hang, the system must be operating at long-term very high load with extremely busy threads spinning at the highest priority. Due to the resource starvation, lower priority threads than the |User-interactive| level get scheduled >60 seconds to never from the time they receive work.

This is how I reproduce the hang and see the results:
- Spin up a bunch of high-priority threads running busy work (I use sqrt() on 50-100 at the |user-interactive| level)
- Wait for machine to start lagging behind, as it's trying to run all of this high-priority work
- Put a Firefox tab in the background by tabbing out
- Either wait, or just initiate a crash.
- You will see it’s never left RecvSetMainThreadQoS

In this scenario it looks like the main thread gets stuck setting itself. While this scenario is very rare and I had to make a conscious effort to produce it, I still would like to plug this hole. This is as easy as not setting an override when the intended priority is background-level, as we aren't dedicated to the timeliness of setting this priority. We only want to use the override to warm up the main thread, not to cool it down.
I've managed to reproduce an extremely specific hang scenario on our MacOS QoS work. To create this hang, the system must be operating at long-term very high load with extremely busy threads spinning at the highest priority. Due to the resource starvation, lower priority threads than the |User-interactive| level get scheduled >60 seconds to never from the time they receive work.

This is how I reproduce the hang and see the results:
- Spin up a bunch of high-priority threads running busy work (I use sqrt() on 50-100 at the |user-interactive| level)
- Wait for machine to start lagging behind, as it's trying to run all of this high-priority work
- Put a Firefox tab in the background by tabbing out
- Either wait, or just initiate a crash.
- You will see it’s never left RecvSetMainThreadQoS

In this scenario it looks like the main thread gets stuck setting itself. While this scenario is very rare and I had to make a conscious effort to produce it, I still would like to plug this hole. This is as easy as not setting an override when the intended priority is background-level, as we aren't dedicated to the timeliness of setting this priority. We only want to use the override to warm up the main thread.

Back to Bug 1883040 Comment 0