1844381 - Slow Unified Inbox on startup/initial entry and switching folders. 25,000 messages

Status: RESOLVED DUPLICATE of bug 1875577

(Thunderbird :: Folder and Message Lists, defect, P2)

Description

[John]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36

Steps to reproduce:

Card view, sorted by date, group by sort on unified inbox (~25k messages).

Actual results:

Switching between folders and returning to the unified inbox grouped by sort results in about a 30 second freeze where Thunderbird is unresponsive. Toggling the option on and off works instantly. Problem occurs when toggling between different folders to a grouped by sort folder.

Expected results:

Not freeze.

Comment 1

[Wayne Mery (:wsmwk)]

Toggling the option on and off works instantly.

You mean changing from Grouped to non-grouped?
And this is with both cards and table view?

Comment 2

[John]

Changing from Grouped to non-grouped is OK on the selected folder.
The problem arises when changing holders where the first selected folder is non-grouped changing to a grouped folder.

The attached is a video demonstrating my unified inbox's sort settings/groupings. I switch to a single account folder (non-grouped), no hickups. I then go back to the unified inbox (grouped) and you will see the hang. Included Task manager so you can see the processes working.

https://www.youtube.com/watch?v=VfgReY5BIps

As a side bar, after deleting an email within the Today group for instance, the count breaks. see included screenshot. In the first example, it shows a wrong amount of new being displayed. In the second screenshot, you can see that Today turns into "Older".

Screenshot 1: https://snipboard.io/6X4jWz.jpg
Screenshot 2: https://snipboard.io/L10EoM.jpg

Comment 3

[Wayne Mery (:wsmwk)]

Please use https://support.mozilla.org/en-US/kb/profiling-thunderbird-performance to get a performance profile. Thanks.

Comment 4

[John]

Requested performance profile:
https://share.firefox.dev/3QnQ6r9

Interesting note. When in unified inbox, toggling between group by date and just threaded works instantly. It only happens when changing between folders. (And when you initially start TB and opening up into unified inbox)

Comment 5

[Wayne Mery (:wsmwk)]

Using config editor try setting mail.db.max_open and mail.db.idle_limit to larger values. For example doubled.

Comment 6

[John]

Doubling the values and restarting had no affect.

Comment 7

[Wayne Mery (:wsmwk)]

I think this also affects version 115 - we have quite a few support postings.

(In reply to John from comment #4)

Interesting note. When in unified inbox, toggling between group by date and just threaded works instantly. It only happens when changing between folders. (And when you initially start TB and opening up into unified inbox)

Analysis of performance profile https://share.firefox.dev/3QnQ6r9 via matrix

julienw
PR_Now is called a lot, presumably returns the same thing or near
Otherwise there's a lot of allocations

jesup
Sort algorithm may play into it. Also: 50% is nsMsgDatabase::CreateMsgHdr() - but half of that is ::~nsMsgHeader, and most of the rest of that is in new() or ::nsMsgHeader, so it's creating and destroying a zillion of those

mstange
It's the msgIndex = FindHdr(msgHdr, 0, true); call that's probably not intended to be slow
FindHdr ends up going through a lot of rows and creates headers for each of them
but we're looking for msgHdr which is an object that already exists, so really, we don't want to create any new header objects
so yeah I'd say the problem is "nsMsgSearchDBView::FindHdr does too much unnecessary work"
It causes Hdrs to be created when we're only looking for an existing Hdr
and it also causes repeated calls to nsMsgDBFolder::GetMsgDatabase

Comment 8

[Wayne Mery (:wsmwk)]

Does comment 9 inform a potential solution?

Comment 9

[Magnus Melin [:mkmelin]]

Must be this: https://searchfox.org/comm-central/rev/bb8c94e8c53ed0ba227252e7f9b9c02d4eee92e9/mailnews/db/msgdb/src/nsMsgDatabase.cpp#410

For threaded view, looks like we cache a bit more (MSGHDR_CACHE_MAX_SIZE). https://searchfox.org/comm-central/rev/bb8c94e8c53ed0ba227252e7f9b9c02d4eee92e9/mailnews/base/src/nsMsgThreadedDBView.cpp#60

Comment 10

[Wayne Mery (:wsmwk)]

Geoff/Magnus, which items do you reckon are are most actionable?

Comment 11

[Magnus Melin [:mkmelin]]

Comment 9 could be actionable.

Comment 12

[Hartmut Welpmann [:welpy-cw]]

(In reply to John from comment #6)

Doubling the values and restarting had no affect.

Do you see a difference when the groups are collapsed (by F10 -> View -> Threads -> Collapse All Threads or simply \) before you reopen the folder?

Comment 13

[John]

Sorry I haven't contributed in a while. Because of the unbearable lag, i reorganized my mail boxes and reduced inboxes down to 6k emails and things were tolerable. 124.0b1 made a noticeable improvement and after 124.0b2, I believe the lag is non-existent. I think you guys fixed it.

Thanks!

Comment 14

[Wayne Mery (:wsmwk)]

Thanks for the update, John.

Bug 1875577 was a big part of 124.0b1