842371 - (tb-mainthreadio) [meta] move IO off the main thread (or next best thing?) to improve Thunderbird responsiveness / reduce jank

Status: NEW

(MailNews Core :: Backend, defect)

Description

[Wayne Mery (:wsmwk)]

short version: This is a meta bug to organize, via other blocking bugs, what is needed to move thunderbird IO off the main thread. This is expected to be very difficult, so there is currently no expectation of when or whether this can be accomplished.  Also unknown is where this might fall in priority compared to spot fixing localized performance issues.  


long version: Thunderbird responsiveness is a frequent support issue, and Thunderbird's dependence on disk IO in key areas is a major cause of poor response. If we can get IO off the main thread it would be a great win, because UI runs on main thread. 

It is not sufficient to simply reduce the amount of IO Thunderbird does (although it can help), nor is it sufficient to make specific functions more efficient (although it can help).  These things are not sufficient because this speed of Thunderbird's IO completion by factors outside Thunderbird's control - by other IO happening on the computer (downloads, AV scans, etc), by a poorly performing disk or by an otherwise slow or overloaded computer. 

So the flipside of this bug, is this bug is NOT about reducing IO, nor about making existing IO more performant on the main thread.

Comment 1

[WADA:World Anti-bad-Duping Agency]

Wayne, meta bug for "File I/O" only? "Network I/O" is excluded?

Please note that, because IMAP access is perhaps currently executed under main task, UI freeze is sometimes reported for IMAP case. And, as for Tb on Win, I saw, in the past, higher dispatching priority of task for SSL than main task of Tb while accesing IMAP server via SSL.
These indicate existence of "not so well task separation" and "not so well task dispathing priority control" in other than "File I/O" area on MS Win.

Because "multiple messenger window" can exist in current Tb, I think separation of tasks like following is needed, without, or with, my own dispatcher.
(i) some special tasks for emergency requests of top priority among all tasks, (ii) one main task with hidden window or without associated window of highest priority among normal tasks, for task-to-task communication, common/never-heavy/high-priority UI-relaed request processing etc., (iii) one UI task per one messager window of second priority among normal tasks, (iv) some tasks for non-main window/panel such as Composition window, Error Console, Activity Manager, of third priority among normal tasks, (v) some tasks for sub-jobs such as file/network I/O of fourth priority among normal tasks, (vi) some tasks for background jobs of lowest priority among normal tasks.

Comment 2

[Wayne Mery (:wsmwk)]

(In reply to WADA from comment #1)
> Wayne, meta bug for "File I/O" only? "Network I/O" is excluded?

Perhaps irving or someone else could comment because I'm not sure how to answer about Network I/O. 
I suspect they should be separate. And the other items you mention may have merit. Perhaps should be discussed in tb-planning to see what is worth pursuing?

FWIW I filed this bug because AFAICT the greatest UI impact (jank) on my laptop is from disk IO.

Comment 3

[:Irving Reid (No longer working on Firefox)]

I'd prefer to have separate meta bugs to track various causes of jank in Thunderbird; keeping this one to track main thread file I/O and opening other tracking bugs for making more of our API layers (such as the message DB) non-blocking.

Wada, IMAP is currently run on separate threads (one for each currently open IMAP connection, which roughly corresponds to an open folder). The problems with UI blocking related to IMAP come from issues such as user-requested actions on a folder needing to wait for background operations such as message download on the same folder, and at a lower level because the IMAP threads must dispatch synchronous operations over to the UI thread to update message databases and mbox files.

Switching the message databases and IMAP over to asynchronous APIs would be a huge task.

Comment 4

[Wayne Mery (:wsmwk)]

Bug 11050 - Use mozStorage instead of Mork for the message summary database - should be a big help, especially if implemented asynchronously.  And issues with msf files will only get worse when maildir is implemented and msf files can get even bigger. wada, can you think of file I/O bugs that should block, or need to be filed and marked blocking?

Also, because I'm not sure how much opportunity there is for purely "main thread" issues, I'm temped to make cast a wider net to include bugs which also *reduce* IO, such as
* Bug 722183 - slow putting a few messages (or single message) into a large local folder
OTOH, bug 1058026 already covers many of these.  

imap related - I haven't found much significan, except bug 516389 - nsImapMailFolder::NotifySearchHit does not chunk its work - which probably has some impact.

async/async issues (some) - https://bugzilla.mozilla.org/buglist.cgi?f1=component&o3=anywordssubstr&list_id=11769935&short_desc=async%20io%20i%2Fo&bug_severity=blocker&bug_severity=critical&bug_severity=major&bug_severity=normal&bug_severity=enhancement&v3=imap%20backend%20database&o1=nowordssubstr&resolution=---&query_format=advanced&f3=component&f2=OP&short_desc_type=anywordssubstr&longdesc=async%20&v1=address%20compos&longdesc_type=allwordssubstr&product=MailNews%20Core&product=Thunderbird

mork perf issues (some) - https://bugzilla.mozilla.org/buglist.cgi?keywords=perf%2C%20&f1=short_desc&keywords_type=allwords&list_id=11769975&o1=nowordssubstr&resolution=---&query_format=advanced&longdesc=mork&v1=crash&product=MailNews%20Core&product=Thunderbird&longdesc_type=allwordssubstr

msf perf issues (some) - https://bugzilla.mozilla.org/buglist.cgi?keywords=perf%2C%20&keywords_type=allwords&f1=short_desc&list_id=11769855&o1=nowordssubstr&resolution=---&classification=Client%20Software&classification=Components&query_format=advanced&f2=OP&longdesc=msf&longdesc_type=allwordssubstr&product=MailNews%20Core&product=Thunderbird

Comment 5

[WADA:World Anti-bad-Duping Agency]

(In reply to :Irving Reid (No longer working on Firefox) from comment #3)
> I'd prefer to have separate meta bugs to track various causes of jank in Thunderbird; (snip)

I see.

> Wada, IMAP is currently run on separate threads (snip)

Thanks for explanation.

Comment 6

[Worcester12345]

(In reply to Wayne Mery (:wsmwk) from comment #2)

(In reply to WADA from comment #1)

Wayne, meta bug for "File I/O" only? "Network I/O" is excluded?

Perhaps irving or someone else could comment because I'm not sure how to
answer about Network I/O.
I suspect they should be separate. And the other items you mention may have
merit. Perhaps should be discussed in tb-planning to see what is worth
pursuing?

FWIW I filed this bug because AFAICT the greatest UI impact (jank) on my
laptop is from disk IO.

How much is this helped by solid state disks? It is still worth working on, but depending on the answer, maybe need to look at big picture (to include network I/O) first.