1524247 - Thunderbird Shutdown crash/hang due to endless wait loop when no password is entered (OAuth2) | WaitForSingleObjectEx | WaitForSingleObject | _PR_MD_WAIT_CV | _PR_WaitCondVar | PR_WaitCondVar | mozilla::CondVar::Wait | nsEventQueue::GetEvent

Status: RESOLVED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Wayne Mery (:wsmwk)]

+++ This bug was initially created as a clone of Bug #1257058 +++

Unclear what impact the patch in Bug 1257058 had. But three signatures still topcrashes. With very similar, and somewhat odd history graphs. Note, the increase in crash volume over time can I think be attributed to users updating to version 60

#1 https://crash-stats.mozilla.com/signature/?release_channel=release&signature=shutdownhang%20%7C%20arena_t%3A%3AGetNonFullBinRun%20%7C%20nsThread%3A%3AShutdown%20%7C%20nsThreadManager%3A%3AShutdown&date=%3E%3D2018-07-31T04%3A33%3A29.000Z&date=%3C2019-01-31T02%3A33%3A29.000Z#graphs

#3 https://crash-stats.mozilla.com/signature/?release_channel=release&signature=shutdownhang%20%7C%20mozilla%3A%3AQueue%3CT%3E%3A%3APush&date=%3E%3D2018-07-31T04%3A32%3A56.000Z&date=%3C2019-01-31T02%3A32%3A56.000Z#graphs

#12 https://crash-stats.mozilla.com/signature/?release_channel=release&signature=shutdownhang%20%7C%20nsCOMPtr_base%3A%3Aassign_with_AddRef%20%7C%20mozilla%3A%3Adetail%3A%3AMutexImpl%3A%3Aunlock%20%7C%20nsThread%3A%3AShutdown%20%7C%20nsThreadManager%3A%3AShutdown&date=%3E%3D2018-07-31T04%3A33%3A55.000Z&date=%3C2019-01-31T02%3A33%3A55.000Z#graphs

and maybe this one
#70 https://crash-stats.mozilla.com/signature/?product=Thunderbird&release_channel=release&signature=shutdownhang%20%7C%20arena_t%3A%3AGetNonFullBinRun%20%7C%20nsThread%3A%3AShutdown%20%7C%20mozilla%3A%3Astorage%3A%3AConnection%3A%3AshutdownAsyncThread&date=%3E%3D2018-07-31T04%3A45%3A10.000Z&date=%3C2019-01-31T02%3A45%3A10.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_sort=-date&page=1#graphs

Comment 2

[Wayne Mery (:wsmwk)]

Perhaps we'll be adding shutdownhang | nsThread::Shutdown | nsThreadManager::Shutdown (bug 1526127) as the version 68 equivalent signatures, because the signatures cited so far for this bug are all v60

Comment 3

[Wayne Mery (:wsmwk)]

https://support.mozilla.org/en-US/questions/1273402 google oauth?

Comment 5

[Wayne Mery (:wsmwk)]

Adding signature from bug 1526127

Comment 6

[Wayne Mery (:wsmwk)]

For version 78, only remaining signature is shutdownhang | nsThread::Shutdown | nsThreadManager::Shutdown, and still a topcrash.

Steady crash rate for last 6 months despite multiple password prompts being gone (bug 1606711), although that's not a specific cause it might have contributed to user confusion.

Comment 7

[Ulf Zibis]

Fortunately version 78 is available from Ubuntu repository since some weeks. The update didn't help. TB still crashes on shutdown, see:
Crash ID: bp-8ebffcf3-cc81-4c47-88e1-793c90210320

Comment 8

[gene smith]

(In reply to Ulf Zibis from comment #7)

Fortunately version 78 is available from Ubuntu repository since some weeks. The update didn't help. TB still crashes on shutdown, see:
Crash ID: bp-8ebffcf3-cc81-4c47-88e1-793c90210320

78.7.1 from ubuntu repo shuts down OK for me with no crash. FWIW, home-built daily for me crashes on shutdown at, I would guess, 80% of the time,
but I just started and stopped 78.7.1 at least 6 times with no crash.
But I rarely run the repo verion so maybe it has to be running for a while to crash on shutdown?

Comment 9

[Ulf Zibis]

See also bug 1524247.

Comment 10

[Ulf Zibis]

(In reply to Ulf Zibis from comment #9)

See also bug 1524247.

Oops, I meant bug 1346060.

Comment 14

[Wayne Mery (:wsmwk)]

Frank, with version 115.5.2, do you still see the issue you mention in bug 1798108 ?

Comment 15

[Frank Gadegast]

Can't tell. I'm still on version 102.15.1, because 115.x.x has to many other bugs, that are'nt fixed yet.

But heh ... it's easy to replicate on any MacOS:

Set a master password, restart TB, don't enter a password, when asked and quit TB from the menu.
Then try to restart it (it won't, because the process is still running in the background and need to be kill manually via terminal).

Comment 16

[Wayne Mery (:wsmwk)]

shutdownhang | nsThread::Shutdown | nsThreadManager::Shutdown crash signature is gone after version 91.13.0

But there are still a few user comments about password for other signatures, primarily:

• Bug 1869297 - Shutdown crash due to "Empty Trash on Exit" for inactive account - shutdownhang | mozilla::SpinEventLoopUntil | nsThread::Shutdown | nsThreadManager::ShutdownNonMainThreads
• Bug 1843744 - Crash in mozilla::dom::workerinternals::RuntimeService::CrashIfHanging shutting down Thunderbird (exit/quit)

Comment 17

[Wayne Mery (:wsmwk)]

This issue might include signature [@ shutdownhang | mozilla::SpinEventLoopUntil | nsThread::WaitForAllAsynchronousShutdowns | nsThreadManager::ShutdownNonMainThreads ] which is current #1 for beta. Increased significantly since ~124.0b4

Comment 19

[Wayne Mery (:wsmwk)]

I had closed the profiler tool in developers, daily build bp-cddec85e-45ee-4029-8400-54cf60250605

Comment 20

[Vlad Lucaci, Thunderbird QA (:vlucaci)]

I just got this crash while having the "Enter credentials for [e-mail] on imap.gmail.com" window opened and left for more than 5 minutes(while I was working on another screen and on another application) on the latest Daily 144.0a1(20250902104421) with macOS 15.6.1

Here is my crash report: https://crash-stats.mozilla.org/report/index/505975e2-4eeb-48e2-9ca6-5e5630250903#tab-details

Comment 21

[Wayne Mery (:wsmwk)]

Bug 1783573 was fixed 2025-10-21 build, 146.0a1. I haven't tested nightly, where crash rate is only a couple per week. We need beta 146 to check for a positive impact where the crash rate is at least 40 per week.

And crashes are still seen on nightly, for example build 2025-10-26 bp-259ba90b-ea75-48af-8fa3-0cf960251101.

Comment 22

[Wayne Mery (:wsmwk)]

I got bp-d061ee60-2205-423b-8847-deddd0251121 with 146.0b2 when attempting to shutdown after failed login because the server name was wrong.

No apparent change in beta crash rate, at least up to 146.0b2. I've commented in Bug 1783573.

Comment 23

[ISHIKAWA, Chiaki]

Can someone who experience the issue under linux (crash at the shutdown) try the patch
in Bug 1956408 ?

I fixed a thread-race issue at shutdown in the patch there, and the problematic scenario fits
the description here and similar bugs during shutdown.
This is related to IMAP, though. So if you are not using IMAP, the problem may not be solved by the patch there.
TSAN build shows no errors in linux XPCSHELL tests for IMAP using TSAN-binary
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=e86c0c989038e24f0c050fb76c2c5b9ccbd6c5dc&selectedTaskRun=dteRxg-nTv-boXG4APHXqA.0

Knowledgeable person can extract the TSAN binary for linux and test there.
Of course, other OS versions can be tested.
But TSAN-binary under linux would print stacktrace when there are thread races, etc. if the crash is related
TSAN-related and I would like to see the TSAN messages.
That is why TSAN-binary under linux is a good choice if the problem persists.

OTOH, if the problem disappears with the binary produced in the job mentioned above, that is a goo d sign.

One of the root cause of strange shutdown issue was that IMAP task was NOT
properly woken up if it was sleeping on a lock and TB was shutdown.
It was sleeping and no cleanup could be performed.
Now it is properly woken up as far as TSAN is concerned, and I no longer see strange
message that reports a task was still running at showdown.

TIA

EDIT: shows no errors in linux XPCSHELL tests FOR IMAP (for imap added). There are errors from other modules. The mochitest show so many issues to my disappointment, but I think it is not IMAP module per se.

Comment 25

[Wayne Mery (:wsmwk)]

A topcrash for this signature for release 149.0/149.0.1, Windows 11 users (who are attempting to update to version 150) is being tracked at Bug 1505660 - Crash in shutdownhang | nsThread::Shutdown | nsThreadManager::ShutdownNonMainThreads. Details at https://bugzilla.mozilla.org/show_bug.cgi?id=1505660#c67

Comment 26

[Wayne Mery (:wsmwk)]

A recent datapoint, from v150, bp-722538d7-4f4c-43a2-a393-fad650260506 "I am unable to download update files; although I downloaded the latest files directly from the official website and performed an overwrite installation, the updates still fail to apply—the download remains stuck at 0 bytes. If I close Thunderbird while it is in this state, a crash report is generated; this cycle simply repeats itself."

(In reply to ISHIKAWA, Chiaki from comment #23)

Can someone who experience the issue under linux (crash at the shutdown) try the patch in Bug 1956408 ?

Not tested by anyone as far as I can tell. But it should be possible to trigger - comment 20 and comment 22 are two possible methods.

Comment 27

[Wayne Mery (:wsmwk)]

Linux Bug 1957882 - Crash in [@ shutdownhang | __syscall_cancel_arch] - may be a duplicate.

(In reply to Wayne Mery (:wsmwk) from comment #25)

A topcrash for this signature for release 149.0/149.0.1, Windows 11 users (who are attempting to update to version 150) is being tracked at Bug 1505660 - Crash in shutdownhang | nsThread::Shutdown | nsThreadManager::ShutdownNonMainThreads. Details at https://bugzilla.mozilla.org/show_bug.cgi?id=1505660#c67

That spike in crashes for Thunderbird users on Windows went away about May 13.

Comment 28

[Max Emig [:maxe]]

Attachment: Bug 1524247 - Prevent shutdown hangs during IMAP OAuth authentication. r=#thunderbird-back-end-reviewers

Comment 29

[Max Emig [:maxe]]

Nowadays, this strongly appears to be just OAuth2, other root causes may have been fixed before.

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/5f5fa03eeb37
Prevent shutdown hangs during IMAP OAuth authentication. r=jtracey

Comment 31

[Wayne Mery (:wsmwk)]

Perhaps consider uplift to 153 after this rides the train?

Comment 32

[Wayne Mery (:wsmwk)]

Beta's historical crash rate is high enough that we should be able to detect whether this v155 patch results in an improvement. So far there are no v155 beta crashes.

Seeing v155 crashes, there does not appear to be a reduction in release crash rate]

Comment 33

[Yury]

For the record: 153 ESR is already fixed since the changes here
https://hg-edge.mozilla.org/comm-central/rev/5f5fa03eeb37#l1.12
got merged into:
https://hg-edge.mozilla.org/releases/comm-esr153/rev/3a65b6886639fece0bb06d001def123229feedeb#l1.61
together with the changes from bug 2064058 (c-c code: https://hg-edge.mozilla.org/comm-central/rev/82e299547233#l1.61).

Comment 34

[Yury]

Plus, there is also a rebase error. This code was explicitly removed:
https://hg-edge.mozilla.org/comm-central/rev/5f5fa03eeb37#l1.43
But still present in 153:
https://hg-edge.mozilla.org/releases/comm-esr153/rev/3a65b6886639fece0bb06d001def123229feedeb#l1.92

And the test got lost.

Comment 35

[Max Emig [:maxe]]

(In reply to Yury from comment #34)

Plus, there is also a rebase error. This code was explicitly removed:
https://hg-edge.mozilla.org/comm-central/rev/5f5fa03eeb37#l1.43
But still present in 153:
https://hg-edge.mozilla.org/releases/comm-esr153/rev/3a65b6886639fece0bb06d001def123229feedeb#l1.92

And the test got lost.

I am not very familiar with how uplifts work and I don't think I can do anything meaningful here.

Comment 36

[Yury]

Well, there is now a remnant hunk on ESR 153:

  nsCOMPtr<nsIRunnable> shutdownNotify = NS_NewRunnableFunction(
      "GetXOAuth2StringShutdownNotifier", [self = RefPtr(this)]() {
        mozilla::RunOnShutdown([self]() {
          // Notify anyone waiting that we're done.
          self->mMonitor.Notify();
        });
      });
  rv = NS_DispatchToMainThread(shutdownNotify.forget());
  if (NS_WARN_IF(NS_FAILED(rv))) {
    return;
  }

that was removed in your patch here.

Comment 37

[Max Emig [:maxe]]

(In reply to Yury from comment #36)

Well, there is now a remnant hunk on ESR 153:

  nsCOMPtr<nsIRunnable> shutdownNotify = NS_NewRunnableFunction(
      "GetXOAuth2StringShutdownNotifier", [self = RefPtr(this)]() {
        mozilla::RunOnShutdown([self]() {
          // Notify anyone waiting that we're done.
          self->mMonitor.Notify();
        });
      });
  rv = NS_DispatchToMainThread(shutdownNotify.forget());
  if (NS_WARN_IF(NS_FAILED(rv))) {
    return;
  }

that was removed in your patch here.

Isn't this the responsibility of release management? I do not think it should stay there, but I am not going to compile it to be certain whether it is a NOOP or actually harmful.

Comment 38

[Max Emig [:maxe]]

Hi Corey, can you take a look at this? It should either be fully uplifted or left-out, I prefer the former.

Comment 39

[Corey Bryant]

We shouldn't have picked this up in esr153 without consulting you.

We're going to back out the fix for bug 2064058 and request esr153 versions for this fix and bug 2064058 fix, assuming changes are needed, and it seems both fixes are needed for uplift.

Comment 40

[Corey Bryant]

Comment on attachment 9610342 [details]
Bug 1524247 - Prevent shutdown hangs during IMAP OAuth authentication. r=#thunderbird-back-end-reviewers

[Triage Comment]
Approved for esr153

Comment 41

[Stephen King (:sking)]

Thunderbird 153.3.0esr:
https://hg.mozilla.org/releases/comm-esr153/rev/3ffcea44ad88

Comment 42

[Wayne Mery (:wsmwk)]

Looking good. The crashes that this patch didn't get, have been caught by Bug 2028760 - For login with a IMAP Gmail account using the Accounts Hub wizard: Crash in [@ shutdownhang | mozilla::SpinEventLoopUntil<T> | nsThread::WaitForAllAsynchronousShutdowns | nsThreadManager::ShutdownNonMainThreads]