1830145 - Hangs switching to unified folder inbox of 260k messages using folder mode settings, threads=All and `Search Online`. Also moving between folders in unified folder mode (example: from Inbox to Sent)

Status: NEW

(Thunderbird :: Folder and Message Lists, defect, P3)

Description

[[:Aureliano Buendía]]

STRs:

1. starts TB and switch to Unified Folders: there is a delay about 10"
2. set message threads on and set to All;
3. switch to All Folder view and toggle Unified Folders to off;
4. close TB
5. starts TB and switch to Unified Folders: there is a delay about 56"

Here profiling as permalink https://share.firefox.dev/40F0Y79

TB 113.0b4

Comment 1

[Thomas D. (:thomas8)]

Aurelio, is it better in Beta 113.0b5 (64-bit) where bug 1817367 has been fixed?

Comment 2

[[:Aureliano Buendía]]

Ciao Thomas, I don't know the answer, because until now I've tried bug #1817367 always with views-threads set to off. The observed result [with views-threads set to off], after the fix, is the one described in my comment https://bugzilla.mozilla.org/show_bug.cgi?id=1817367#c43
Trying (post fix) also with views-threads set to on, I encountered the performance problems described here.

Comment 3

[[:Aureliano Buendía]]

I can confirm this issue also in 114.0a1 (2023-05-04) (64-bit)

Comment 4

[[:Aureliano Buendía]]

The problem still persists in version 114.0beta3. Here the profiling result as permalink https://share.firefox.dev/42N6C9b
Related to nsThread::ProcessNextEvent ?

Comment 6

[Geoff Lankow (:darktrojan)]

How many messages are we talking about here? From the profile's screen shots it looks like your combined inbox has a 5-figure total, but it's hard to tell. I've been testing with a combined 70,000 messages and the busiest functions in your profile are not problematic for me.

Are you also hitting bug 1836393? It looks like you might be, and that could be related. I don't see that problem either.

I don't suppose you know if the same problem existed in 102, do you?

Comment 7

[[:Aureliano Buendía]]

(In reply to Geoff Lankow (:darktrojan) from comment #6)

How many messages are we talking about here?

Hi Geoff.
My 'Inbox on Unified Folders' has around 260k emails. This issue isn't a regression, because it behaves the same way in TB 102 as well.

Upon further investigation, I have verified that by selecting my 'Inbox on Unified Folders', there is high online traffic to my accounts' IMAP servers. Going to the 'Inbox on Unified Folders' virtual folder, I saw that the option 'Search Online (Gives up-to-date results for IMAP etc...)' is selected by default. By setting 'Search Online (Gives up-to-date results for IMAP etc...)' to false, I no longer have the performance problems mentioned here.

This ticket should be considered Invalid, as the performance of a search on an IMAP server depends on the IMAP server itself and not from TB?
For end users, it should be documented that in the 'Inbox on Unified Folders', the flag 'Search Online (Gives up-to-date results for IMAP etc...)'
is it true by default so there might be performance issues with large Inboxes?
Other ideas?

Thank you.

Comment 8

[Wayne Mery (:wsmwk)]

Aureliano, thanks for the update.

Dropping from supernova list based on comment 7

Comment 9

[Wayne Mery (:wsmwk)]

Aureliano, Does this happen with latest beta?

Comment 10

[[:Aureliano Buendía]]

(In reply to Wayne Mery (:wsmwk) from comment #9)

Aureliano, Does this happen with latest beta?

Unfortunately no, the problem is still here and makes the Unified Folders mode unusable in my case.
118.0b4 (64-bit) on Windows 11

Comment 11

[Wayne Mery (:wsmwk)]

(In reply to [:Aureliano Buendía] from comment #10)

(In reply to Wayne Mery (:wsmwk) from comment #9)

Aureliano, Does this happen with latest beta?

Unfortunately no, the problem is still here and makes the Unified Folders mode unusable in my case.
118.0b4 (64-bit) on Windows 11

For every unified folder type?
Tried repairing any of the subfolders?

Comment 12

[Wayne Mery (:wsmwk)]

Using config editor try setting mail.db.max_open and mail.db.idle_limit to larger values. For example doubled.

Comment 13

[Wayne Mery (:wsmwk)]

Basically 100% of the activity is DOM related, specifically the stack is
nsMsgSearchDBView::AddHdrFromFolder
nsMsgSearchDBView::MoveThreadAt
nsMsgDBView::CollapseByIndex
nsMsgSearchDBView::RemoveRows
nsCOMArray_base::RemoveObjectAt
...

So changing mail.db.max_open or mail.db.idle_limit will not help here.

Comment 14

[Wayne Mery (:wsmwk)]

Aureliano, to what degree do you still see this problem?

Comment 15

[[:Aureliano Buendía]]

Hi Wayne, 'Unified Folders' continues to be unusable in production also fro TB 126.0b1 (64-bit) on Windows 11. Here the profiling result as permalink https://share.firefox.dev/3Uc3jFv

Comment 16

[Wayne Mery (:wsmwk)]

Is it still true that this only happen when you have Search Online enabled? (which is the default)

Two open bug reports cite Search online https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1098069%201757315&list_id=17004189

https://profiler.firefox.com/public/p2jpxd2k8wgwq7y6q135reg177eh9ff8zj77td0/calltree/?globalTrackOrder=2w501&hiddenLocalTracksByPid=40080-01~27476-02w7&range=24671m6003&thread=0&v=10 is interesting

Comment 17

[[:Aureliano Buendía]]

(In reply to Wayne Mery (:wsmwk) from comment #16)

Is it still true that this only happen when you have Search Online enabled? (which is the default)

Yes, it is.

Comment 18

[Wayne Mery (:wsmwk)]

Does it help to disable status bar at View > Toolbars ?

Comment 19

[[:Aureliano Buendía]]

No Wayne, I have the same problem even without status-bar: TB in that case is not usable.
TB 126.0b3 (64-bit) on Windows 11

Comment 20

[Wayne Mery (:wsmwk)]

Aureliano, does this still reproduce for you when using a current beta?