1675068 - Service workers delaying shutdown

Status: ASSIGNED

(Core :: DOM: Service Workers, defect, P2)

Description

[Yaron Tausky [:ytausky]]

After the nasty deadlock in bug 1588152 was resolved, it turns out that there are still reports of service workers delaying the shutdown. The top 3 async shutdown timeout reasons, accounting for 60% of the crashes, are:

1. {"shutdownStates":"parent process IPDL background thread, ","pendingPromises":1,"acceptingPromises":false}
2. {"shutdownStates":"","pendingPromises":0,"acceptingPromises":true}
3. {"shutdownStates":"content process main thread, ","pendingPromises":1,"acceptingPromises":false}

Comment 1

[Florian Quèze [:florian]]

(In reply to Yaron Tausky [:ytausky] from comment #0)

3. {"shutdownStates":"content process main thread, ","pendingPromises":1,"acceptingPromises":false}

I just encountered a crash today that falls in this category: https://crash-stats.mozilla.org/report/index/bac8b5b8-29da-487f-a42d-71cf40201104#tab-metadata

I have semi-reliable steps to reproduce. When I do startup profiling on Windows (start with MOZ_PROFILER_STARTUP=1 in the environment), sometimes when I attempt to collect the profile, the profiler tab doesn't load, and remain stuck with the animated loading indicator, and the profiler.firefox.com domain name as the tab title, but the tab doesn't actually load. The profiler website uses a service worker. When a profiler tab fails to load like this, Firefox shutdown never completes cleanly.

On a slow test laptop, I reproduce quite consistently when doing a cold startup profile. On my devlopment machine I could reproduce about half the time with my local build (so I might be able to test things there).

Comment 2

[Itiel]

I woke my laptop, immediately updated Nightly and after ~2-3 minutes of the updater running in the background I got this crash:
https://crash-stats.mozilla.org/report/index/0c366038-3f4a-42d8-bdbe-181e70201116

Comment 3

[alex_mayorga]

¡Hola!

Build ID 20201211213049 crashed here:

https://crash-stats.mozilla.org/report/index/7bb232a4-9947-4e39-acdf-a71050201216#tab-details

Updated flags per the crash data available.

¡Gracias!
Alex

Comment 4

[alex_mayorga]

¡Hola y'all!

Build ID 20210429214231 crashed here:

https://crash-stats.mozilla.org/report/index/56c30794-a886-45dd-9af5-b00c20210430#tab-details

Updated flags per the crash data available:

https://crash-stats.mozilla.org/signature/?product=Firefox&signature=AsyncShutdownTimeout%20%7C%20profile-change-teardown%20%7C%20ServiceWorkerShutdownBlocker%3A%20shutting%20down%20Service%20Workers

¡Gracias!
Alex

Comment 5

[alex_mayorga]

¡Hola y'all!

Build ID 20220129091708 crashed here:

https://crash-stats.mozilla.org/report/index/56c30794-a886-45dd-9af5-b00c20210430#tab-details

Updated flags per the crash data available:

https://crash-stats.mozilla.org/signature/?product=Firefox&signature=AsyncShutdownTimeout%20%7C%20profile-change-teardown%20%7C%20ServiceWorkerShutdownBlocker%3A%20shutting%20down%20Service%20Workers

¡Gracias!
Alex

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This signature started to get crash reports again at the end of last week (10-20 crash reports per Nighty). Regression range:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=720d5125a9b4aa6750806ed9b51fbd0811da10c4&tochange=59134b451eec1454be5f2e489176b4b59f1ddb5a

Yulia, could bug 1742438 be related?

Comment 7

[Yulia Startsev [:yulia]]

Yes, possibly. Hopefully I have a fix for this regression already.

Comment 8

[BugBot [:suhaib / :marco]]

The bug is marked as tracked for firefox104 (nightly). However, the bug still has low severity.

:jstutte, could you please increase the severity for this tracked bug? If you disagree with the tracking decision, please talk with the release managers.

For more information, please visit auto_nag documentation.

Comment 9

[Jens Stutte [:jstutte]]

(In reply to Yulia Startsev [:yulia] from comment #7)

Yes, possibly. Hopefully I have a fix for this regression already.

Yulia, could you please link the bug with the fix here?

Comment 10

[Yulia Startsev [:yulia]]

(discussed with jens offline)

I took a look, and this appears to still be happening. I believe this is due to the cache promise cleanup potentially sometimes being late. it is similar to the points made by Yaron above a few years ago, but now that it is done independently of the ScriptLoader shutdown it may be lagging or failing. I'll investigate.

Comment 11

[Yulia Startsev [:yulia]]

I think I found the underlying issue. Cancellation was changed into "informing" rather than triggering a kill of the worker. However, it is possible for the worker to be killed without going through cancellation, which means we don't do clean up. I am looking in to how to make this always go through the cancellation path / do cleanup.

Comment 12

[Jens Stutte [:jstutte]]

(removing myself as assignee as Yulia seems to be on top of it)

Comment 13

[BugBot [:suhaib / :marco]]

The bug is linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

:yulia, could you consider increasing the severity of this top-crash bug?

For more information, please visit auto_nag documentation.

Comment 14

[BugBot [:suhaib / :marco]]

Based on the topcrash criteria, the crash signature linked to this bug is not a topcrash signature anymore.

For more information, please visit auto_nag documentation.

Comment 15

[Yulia Startsev [:yulia]]

I am still tracking this bug, as there is a bit more work to do regarding the cancellation path. Will update once I am finished.

Comment 16

[Yulia Startsev [:yulia]]

I've reviewed the crashes in the last week, and they do not appear to be related to the ScriptLoader shutdown any more, they appear to be hanging in the JS interpreter, which looks unrelated.

Comment 17

[Jens Stutte [:jstutte]]

Indeed we are back to normal here. Unassigning Yulia but we keep it open for further monitoring.

Comment 18

[BugBot [:suhaib / :marco]]

The bug is linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

:jmarshall, could you consider increasing the severity of this top-crash bug?

For more information, please visit auto_nag documentation.

Comment 19

[Donal Meehan [:dmeehan]]

:jstutte this has started spiking in recent nightly builds.
Could someone take a look, is it related to Bug 1812490?

Comment 20

[Jens Stutte [:jstutte]]

The spike seemed to start with landing of bug 1811195, which has a remote worker related part. Then we landed the fix for bug 1812490 and numbers went down a lot, but seem to be still significantly higher than before. My assumption would be, that one of the two conditions we changed in bug 1811195 is not triggered anymore since bug 1812490 landed, but the other is?

:asuth, could it be that the bail out in BackgroundParentImpl::AllocPRemoteWorkerControllerParent is doing more harm than good?

Comment 21

[Jens Stutte [:jstutte]]

(In reply to Jens Stutte [:jstutte] from comment #20)

:asuth, could it be that the bail out in BackgroundParentImpl::AllocPRemoteWorkerControllerParent is doing more harm than good?

From here that looks very likely.

Comment 22

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Jens Stutte [:jstutte] from comment #20)

:asuth, could it be that the bail out in BackgroundParentImpl::AllocPRemoteWorkerControllerParent is doing more harm than good?

Yeah, I think this would probably break the state machine and so removing it again in https://phabricator.services.mozilla.com/D168264 makes sense.

Comment 23

[BugBot [:suhaib / :marco]]

This is a reminder regarding comment #8!

The bug is marked as tracked for firefox111 (nightly). We have limited time to fix this, the soft freeze is in 9 days. However, the bug still isn't assigned.

Comment 24

[Jens Stutte [:jstutte]]

(In reply to Release mgmt bot [:suhaib / :marco/ :calixte] from comment #23)

This is a reminder regarding comment #8!

The bug is marked as tracked for firefox111 (nightly). We have limited time to fix this, the soft freeze is in 9 days. However, the bug still isn't assigned.

We do not yet see data after bug 1813559 landed (which is tracked, too), the hope would be that we can untrack this bug again after that landed.

Comment 25

[BugBot [:suhaib / :marco]]

This is a reminder regarding comment #8!

The bug is marked as tracked for firefox111 (nightly). We have limited time to fix this, the soft freeze is in 8 days. However, the bug still isn't assigned.

Comment 26

[Donal Meehan [:dmeehan]]

Removing tracking for 111, the crash volume dropped after Bug 1813559 landed.

Comment 27

[phsaldanha]

I can reproduce this bug by opening all my 40+ bookmarks and closing the browser before all pages are loaded. Sometimes it happens and sometimes it doesn’t, but most of the time I get the same crash.
STR:
1 - Open Firefox Nightly 102.0a1
2 - Right-click bookmark bar -> Open all bookmarks
3 - Close the browser right after opening all the bookmarks without waiting for them to finish loading

Results: Firefox closes most of the processes, but a few particular processes remain open for a while until I get the crash notification.

Expected results: Firefox should close all the tabs.

Using Windows 11

Crash report: https://crash-stats.mozilla.org/report/index/205110cc-31e3-4c5c-ab0d-2df8a0230221

Comment 28

[BugBot [:suhaib / :marco]]

Based on the topcrash criteria, the crash signature linked to this bug is not a topcrash signature anymore.

For more information, please visit auto_nag documentation.

Comment 29

[BugBot [:suhaib / :marco]]

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

For more information, please visit auto_nag documentation.

Comment 30

[BugBot [:suhaib / :marco]]

Based on the topcrash criteria, the crash signature linked to this bug is not a topcrash signature anymore.

For more information, please visit BugBot documentation.

Comment 31

[BugBot [:suhaib / :marco]]

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

For more information, please visit BugBot documentation.

Comment 32

[Jens Stutte [:jstutte]]

Could it be that the bot is getting confused around merge day about how to query the nightly topcrash criteria (similar to https://github.com/mozilla/bugbug/issues/351) ?

Comment 33

[Suhaib Mujahid [:suhaib]]

(In reply to Jens Stutte [:jstutte] from comment #32)

Could it be that the bot is getting confused around merge day about how to query the nightly topcrash criteria

Thank you for reporting this! The bot account for the release day and includes the previous version in the first week after the release date. However, it seems that contradictions in the versions on the release day caused this. I filed an issue to follow up on this: https://github.com/mozilla/bugbot/issues/2100

(similar to https://github.com/mozilla/bugbug/issues/351) ?

I guess you mean https://github.com/mozilla/bugbug/issues/3516

Comment 34

[BugBot [:suhaib / :marco]]

Based on the topcrash criteria, the crash signature linked to this bug is not a topcrash signature anymore.

For more information, please visit BugBot documentation.

Comment 35

[BugBot [:suhaib / :marco]]

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

For more information, please visit BugBot documentation.

Comment 36

[BugBot [:suhaib / :marco]]

Based on the topcrash criteria, the crash signature linked to this bug is not a topcrash signature anymore.

For more information, please visit BugBot documentation.

Comment 37

[Jens Stutte [:jstutte]]

(In reply to Suhaib Mujahid [:suhaib] from comment #33)

(In reply to Jens Stutte [:jstutte] from comment #32)

Could it be that the bot is getting confused around merge day about how to query the nightly topcrash criteria

Thank you for reporting this! The bot account for the release day and includes the previous version in the first week after the release date. However, it seems that contradictions in the versions on the release day caused this. I filed an issue to follow up on this: https://github.com/mozilla/bugbot/issues/2100

It seems it is happening again.

Comment 38

[Suhaib Mujahid [:suhaib]]

:jstutte I will prioritize working on #2100 to prevent this from happening again.

Comment 39

[BugBot [:suhaib / :marco]]

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

For more information, please visit BugBot documentation.

Comment 40

[Worcester12345]

"Perf" key word?

Comment 41

[Dennis Clarke]

This crash may fall into the same category : 

https://crash-stats.mozilla.org/report/index/3e31cb79-abdb-46e7-83da-0f8680230819#tab-bugzilla

I see Signature: [@ AsyncShutdownTimeout | profile-change-teardown | ServiceWorkerShutdownBlocker: shutting down Service Workers ]

Comment 42

[BugBot [:suhaib / :marco]]

Based on the topcrash criteria, the crash signature linked to this bug is not a topcrash signature anymore.

For more information, please visit BugBot documentation.

Comment 43

[BugBot [:suhaib / :marco]]

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

For more information, please visit BugBot documentation.

Comment 44

[BugBot [:suhaib / :marco]]

Based on the topcrash criteria, the crash signature linked to this bug is not a topcrash signature anymore.

For more information, please visit BugBot documentation.

Comment 45

[Liz Henry (:lizzard)]

This seems to consistently be a topcrash in Nightly.

Comment 46

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Crash volume on Nightly multiplied over the past two weeks.

Comment 47

[Mr Beedell, Roke Julian Lockhart [:B.RJL]]

I've had crash-stats.mozilla.org/report/index/7322ab50-e64d-4e07-b671-430fa0260512 occur on firefox-nightly-152.0a1-20260511090215, merely after ydotool key 29:1 16:1 16:0 29:0, whilst discuss.kde.org/t/46941/9 was assign'd, if that's of any use. I observed no ANR, and Abrt and CoreDumpCtl did not register the SegV.

Comment 48

[Henrik Mitsch [:hmitsch]]

I am also experiencing this issue on Nightly. My system is macOS and the current Nightly version is: 153.0a1 (2026-06-15) (aarch64)

We just had a chat on the Nightly Channel about this (https://chat.mozilla.org/#/room/#nightly:mozilla.org).

May I suggest to increase priority to P1 to avoid that this slips into the Release branch?

I am happy to provide more details and run tests.

Here are some of my crash report URLs:
https://crash-stats.mozilla.org/report/index/002cd800-58aa-4c93-b0a4-d67af0260614
https://crash-stats.mozilla.org/report/index/632c900b-c033-48b0-a495-b74b70260615
https://crash-stats.mozilla.org/report/index/cdf5fdde-7b08-450b-b1c3-1eb960260615

With appreciation
Henrik

Comment 49

[Henrik Mitsch [:hmitsch]]

Attachment: Crash history

Here is a plot of my about:crashes table. It clearly shows that something is wrong in recent weeks.

Comment 50

[Henrik Mitsch [:hmitsch]]

I was updated to Firefox 154 today and the crashes still exist: 154.0a1 (2026-06-16) (aarch64)

Comment 51

[Henrik Mitsch [:hmitsch]]

Attachment: crash_reports_daily.png

I am on 154.0a1 (2026-07-13) (aarch64) and the crashes are still happening whenever I recycle Firefox to update Nightly.

Please help.

Comment 52

[Mr Beedell, Roke Julian Lockhart [:B.RJL]]

Attachment: `user_id=691781`'s `about:support` JSON

Likewise, every time I close firefox-nightly-154.0a1-20260712113206:

• crash-stats.mozilla.org/report/index/86567252-c8a3-4cac-a2e8-962330260712

• crash-stats.mozilla.org/report/index/a585be03-c7eb-499e-95ec-e15f90260713

Comment 53

[Takanori MATSUURA]

I am also experiencing this issue on Win64 Nightly.

Crash report: bp-4786dbb0-c4c8-4cc9-a09c-ab5ac0260714

MOZ_CRASH Reason:

[Parent 10104, Main Thread] ###!!! ABORT: file checkouts\gecko\dom\serviceworkers\ServiceWorkerShutdownBlocker.cpp:107

Top 10 frames:

0  xul.dll  MOZ_Crash(char const*, int, char const*)  mfbt/Assertions.h:402
0  xul.dll  Abort(char const*)  xpcom/base/nsDebugImpl.cpp:528
1  xul.dll  NS_PrintStackTrace()  xpcom/base/nsDebugImpl.cpp:373
1  xul.dll  NS_DebugBreak(unsigned int, char const*, char const*, char const*, int)  xpcom/base/nsDebugImpl.cpp:511
2  xul.dll  nsDebugImpl::Abort(char const*, int)  xpcom/base/nsDebugImpl.cpp:127
3  xul.dll  XPTC__InvokebyIndex()  /builds/worker/workspace/obj-build/toolkit/library/build/Z:/builds/worker/checkouts/gecko/xpcom/reflect/xptcall/md/win32/xptcinvoke_asm_x86_64.asm:97
4  xul.dll  NS_InvokeByIndex(nsISupports*, unsigned int, unsigned int, nsXPTCVariant*)  xpcom/reflect/xptcall/md/win32/xptcinvoke_x86_64.cpp:56
4  xul.dll  CallMethodHelper::Invoke()  js/xpconnect/src/XPCWrappedNative.cpp:1617
4  xul.dll  CallMethodHelper::Call()  js/xpconnect/src/XPCWrappedNative.cpp:1171
4  xul.dll  XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode)  js/xpconnect/src/XPCWrappedNative.cpp:1117

Comment 54

[Henrik Mitsch [:hmitsch]]

Since updating to Nightly 155, the issue is gone.

Comment 55

[Henrik Mitsch [:hmitsch]]

Attachment: 20260726_about:crashes.png

Screenshot, showing what I mentioned in Comment 54: No crashes since upgrading to Nightly 155 on July 20, 2026 (https://whattrainisitnow.com/release/?version=155).

Comment 56

[Takanori MATSUURA]

I'm still experiencing this issue on Win64 Nightly.
bp-dc4f8fa5-d6e4-4cd5-a600-7b05e0260727

Interestingly, this crash has recently been occurring only in the Nightly build, and has not occurred in the beta, release, or ESR builds.