724436 - Message Summary View is very slow to construct when many messages selected

Status: RESOLVED WORKSFORME

(Thunderbird :: Message Reader UI, defect)

Description

[Dan Halbert]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20100101 Firefox/10.0
Build ID: 20120129021758

Steps to reproduce:

Select a large number of messages (e.g. Select all in a multi-thousand message folder)


Actual results:

The Summary View shown in the Message View pane is extremely slow to build, especially on an older, slower machine. It can take many minutes.


Expected results:

Suggestions:
1. don't build summary for more than some number N of messages
2. Stop building summary after some amount of time has passed.
3. Allow turning off Summary View

Comment 1

[WADA:World Anti-bad-Duping Agency]

Quick check result with Tb trunk dailly 2012/01/29 build(Tb12.0a1) of "Select All of 10000 mails in local mail folder", "1000 threads, 10 mails per a thread(10 mails has same subject)", No Threaded View, sorted by "Order Received" column.
Difference between mail.operate_on_msgs_in_collapsed_threads = true / false.
false: it took long.
true : it took longer than false, and "Unresponsive script" dialog appeared once,
       but time needed to Select All was not so longer from false in my case.
       problem in true case looks larger peek VM size while executing Select All,
       larger VM size after completion of Select All, larger VM size even after
       only single mail is selected again, and higher CPU usage while executing
       Select All.
Big diffwerence between Tb 9.0.1 and Tb 12.0a1 was not observed.

"long time" in false case may be due to internal threading. It looks that there is no way to prohibit it. I may be relevant to resource for "undelete".
Increased VM size, CPU usage etc. may be cost to show in "Conversation" at message pane.
Because threading is relevant, peek VM size, CPU usage, required time etc. may be higher than O(number of selected mails), and it may depend on number of threads, thread depth etc.

Do you see big difference in "required time to Select All" between operate_on_msgs_in_collapsed_threads = false and operate_on_msgs_in_collapsed_threads = true?

> Expected results:
> Suggestions:
>(snip)
> 3. Allow turning off Summary View

operate_on_msgs_in_collapsed_threads = false didn't work?
Or it's insufficient for your requirement?
Or it's different feature from one you want?

Comment 2

[Dan Halbert]

I see that my scenario is more unusual. I have moved several people from Outlook Express POP3 to Thunderbird IMAP, on XP SP3 on older computers. These people typically have inboxes of thousands of messages. I install Thunderbird and import everything from Outlook Express. I then immediately try to start copying messages and folders from the imported folders into the new IMAP folders. Doing a Select All on the freshly imported huge inbox takes many minutes. (I usually turn off indexing for global searching before doing this.)

Is it possible that the reason is because Thunderbird has not yet built summary information for these imported folders? After I did the POP3->IMAP copies and Thunderbird sat overnight, Select All on large folders takes only a few seconds.

Comment 3

[WADA:World Anti-bad-Duping Agency]

(In reply to Dan Halbert from comment #2)
> I see that my scenario is more unusual.
> I have moved several people from Outlook Express POP3 to Thunderbird IMAP, on XP SP3 on older computers.
> These people typically have inboxes of thousands of messages.
> I install Thunderbird and import everything from Outlook Express.
> I then immediately try to start copying messages and folders from the imported folders into the new IMAP folders.
> Doing a Select All on the freshly imported huge inbox takes many minutes.
> (I usually turn off indexing for global searching before doing this.)
> Is it possible that the reason is because Thunderbird has not yet built
> summary information for these imported folders? After I did the POP3->IMAP
> copies and Thunderbird sat overnight, Select All on large folders takes only
> a few seconds.

In my quick check of "Select All of 10,000 mails", "it took long" and "it took longer" was "it took 3 to 7 seconds". i.e. It took several seconds to Select All 10,000 mails, and it was sufficiently long for simply selecting merely 10,000 mails.

Because your case is "Imported mail folder", following are relevant.
(A) Import correctly creates mail folder file only(file of Inbox instead of Inbox.msf if mail folder named of Inbox. file named Inbox contains mail data). Inbox.msf is not generated by Import of mails. So, internal Rebuild-Index of the imported mail folder is required upon first open of the imported mail folder. As you know well, Rebuild-Index(currently "Repair Folder" of Folder's Properties/General Tab) takes sufficiently long.
(B) Because X-Mozilla-Status: and X-Mozilla-Status2: header is not written upon import, Read/Unread status which is held in X-Mozilla-Status: header is lost if xxx.msf file is deleted after import.
X-Mozilla-Status: and X-Mozilla-Status2: is generated by first/actual execution of "Compact" of the local mail folder.
(C) Because X-Mozilla-Keys: header is not written upon import, tag status of mail is lost if xxx.msf file is deleted after import.
X-Mozilla-Keys: header(which holds tags added to the mail) is generated by second/actual execution of "Compact" of the local mail folder(Compact after X-Mozilla-Status: and X-Mozilla-Status2: is generated by first Compact).

Please separate "Time required for Rebuild-Index of imported local mail folder", "Time required for Select All of mails in a local mail folder", and "Time required to copy many mails from local mail folder to IMAP mail folder".

Comment 4

[Wayne Mery (:wsmwk)]

datapoint, I could have sworn there was a bug about summary view performance, and sanity checking, but I don't find one. Perhaps it was part of the disccussion in the original implementation.

that said, I selected all of a local folder of 35k messages and it only took about 7 seconds on my i7 machine. That seems pretty reasonable to me.

Comment 5

[Dan Halbert]

>(A) Import correctly creates mail folder file only(file of Inbox instead of Inbox.msf if mail folder named of Inbox. file named Inbox contains mail data). Inbox.msf is not generated by Import of mails. So, internal Rebuild-Index of the imported mail folder is required upon first open of the imported mail folder. As you know well, Rebuild-Index(currently "Repair Folder" of Folder's Properties/General Tab) takes sufficiently long.

I tried reproducing this by deleting the .msf file of a Trash folder of about 4k messages. However, selecting it all was quite fast.

So at this point I cannot duplicate this, except by reverting to the original scenario. I don't have any more machines to convert from Outlook Express to Thunderbird, so I may just give up. We could leave the bug for others if it appears in other circumstances.

Comment 6

[Wayne Mery (:wsmwk)]

(In reply to Dan Halbert from comment #5)
> We could leave the bug for
> others if it appears in other circumstances.

we don't normally do that if the reporter is good.