1917490 - High CPU and Slow Displays (Hours) with Unified Folders and Saved Searches (250k messages)

Status: RESOLVED INCOMPLETE

(Thunderbird :: Folder and Message Lists, defect, P1)

Description

[Erik]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

I select the Unified Inbox or a Saved Search;

There are three issues with Unified Folders and Saved Searches that are not 100% reproducible .

1. Often the message count is incorrect.
2. When I select the Unified Inbox folder it often takes a very long time to display the messages (sometimes 2-3 hours).
3. When Thunderbird tries to collect and displays the messages, my CPU utilization is very high for up to several hours. During this time, I cannot use Thunderbird and my PC is unresponsive.

Note that I am using IMAP, I have synced the bodies and I am not searching online.

Actual results:

Here is is an example where I previously had selected another folder and then selected the existing (displayed) Unified Inbox. After Thunderbird spent about 40 min collecting the messages, it showed:
https://i.gyazo.com/5264cd8f5224e854f1e3cc1a96a13c66.png
Account 1 has 775 messages and account 2 has 259,242 messages. Total should be 260,017 messages, but TB shows 260,022 messages.

Here is another example after I restarted TB and selected the Unified Inbox:
https://i.gyazo.com/f5d1415bfc452526df682ccb590c1711.png
In this case, the Unified Inbox message count is way off but eventually, after about 3 hrs, showed the correct count. During these 3 hours, the CPU utilization was very high, I was not able to use TB, and I barely could use my PC.

I have the same issues of high CPU when using Saved Searches, and I need to wait a very long time before being able to access the messages. I understand that part of the cause may be the high message count in one of my folders.

Expected results:

Does TB sometimes need to recreate the Unified Inbox by searching all messages?

I would have expected that TB would write the Inbox search results to a file on my hard disk and then pull it up into RAM, within a few seconds, when needed?

My results are not 100% reproducible and I wonder if there may be some bug in writing the Inbox result file to disk and/or recalling it?

NOTE that I do not have these issues with Outlook (2010) where I combine many local POP3 folders in Saved Searches.

Comment 2

[Wayne Mery (:wsmwk)]

There is an improvement in version 128, which unfortunately you will not be able to run because you are using Windows 7. But you still have some choices

1. Check https://bugzilla.mozilla.org/show_bug.cgi?id=1908761#c5
2. Test with Windows started in safe mode https://support.microsoft.com/en-us/windows/windows-startup-settings-1af6ec8c-4d4a-4b23-adb7-e76eef0b847f#start-computer-safe-mode=windows-7
3. Substantially reduce the number of messages in your Inbox by deleting or archiving. For example you can archive by year https://support.mozilla.org/en-US/kb/archived-messages. I would not archive more than 2k messages at a time.

Comment 3

[Erik]

1. I understood that v 128 has the potential to corrupt IMAP folders? Some users have to repair weekly..?

2. So v 115 is the latest version for Windows 7? Any workaround to get later versions working on WIN7? (I may not have time to put together a new PC with WIN 11 until the end of he year...)

3. https://bugzilla.mozilla.org/show_bug.cgi?id=1908761#c5 deals with Quick Filters. My problem is with Saved Searches and subsequently Unified Folders. Does the fixes in v128 also work on Saved Searches/Unified Folders? (What was fixed?). Would it help if I use the workaround of different column sort orders?

4. Why would safe mode help?

5. It is very difficult to archive only 2K at a time when I have 260K messages. Could I use more if I just use manual move or copy/delete?

Comment 4

[Wayne Mery (:wsmwk)]

(In reply to Erik from comment #3)

1. I understood that v 128 has the potential to corrupt IMAP folders? Some users have to repair weekly..?

A moot point for you, but the most recent versions should be beter.

2. So v 115 is the latest version for Windows 7? Any workaround to get later versions working on WIN7? (I may not have time to put together a new PC with WIN 11 until the end of he year...)

Sorry, No

3. https://bugzilla.mozilla.org/show_bug.cgi?id=1908761#c5 deals with Quick Filters. My problem is with Saved Searches and subsequently Unified Folders. Does the fixes in v128 also work on Saved Searches/Unified Folders? (What was fixed?). Would it help if I use the workaround of different column sort orders?

The issue potentially affects folders when not using quick filter.

4. Why would safe mode help?

Because we don't know whether some other software is contributing to your slowness.

5. It is very difficult to archive only 2K at a time when I have 260K messages. Could I use more if I just use manual move or copy/delete?

Filters under Tools can be used to do the move bulk without causing resource issues, because it will move one message at a time.

Comment 5

[Erik]

I tried version 128.2.0esr-bb8-build2 (64 bit) on a WIN 10 PC but I have the same issues;
High CPU, program stops responding, and incorrect number of E-mails in Unified Inbox.

I am currently not able to use a Unified Inbox (and I have 20 E-mail accounts that I would like to combine).

As stated previously, it works fine in Outlook (using POP3).
I hope that it will be fixed in Thunderbird so that it will be possible to use a Unified Inbox with 300K messages....

Comment 6

[betterbird.project+18]

Please find the analysis of this issue in bug 1923520.

Comment 7

[macuc]

This sounds a lot like what I have been experiencing, and reported in bug 1918534.

My theory is that it is some sort of regression in the generation and maintenance of the C:\Users[username]\AppData\Roaming\Thunderbird\Profiles[profile id]\Mail\smart mailboxes\ folder. But it may also be relatedd to these bugs...

Bug 1858695
Bug 1856997

Comment 10

[Erik]

Because of this cache issue, Thunderbird has been very difficult for me to use. The incredibly helpful coders at BetterBird (https://www.betterbird.eu/) has largely fixed the issue. I switched to BetterBird and enjoy much better performance!

Comment 11

[Wayne Mery (:wsmwk)]

Please post after using 128.5.2, available later this week hopefully.

Comment 12

[Erik]

The problem in Thunderbird occurs because of poor implementation of the cache. The issue has been fixed in Betterbird (https://www.betterbird.eu/).

Comment 13

[Erik]

p.s. I strongly suggest that the Thunderbird developers touch base with the Betterbird developers to add the code fixes to Thunderbird.