224381 - GUI freezes during IMAP connect

Status: RESOLVED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Tilman Schmidt]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:1.5) Gecko/20031007
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:1.5) Gecko/20031007

If an IMAP account is configured to check for new mail periodically, the user
interface in all open Mozilla windows (browser, mail, message composition ...)
freezes while Mozilla Mail tries to connect to the IMAP server, ie. dragging a
scrollbar hangs, typed characters don't appear in a text field, etc. As soon as
the actual message download starts, the GUI is operational again, characters
typed during the freeze appear all at once (along with any extraneous ones the
user typed because she didn't see anything appearing), scrollbars move again, etc.

Reproducible: Always

Steps to Reproduce:
1. Configure an IMAP mail account checking for mail on start and every 2 minutes
2. open the mail account and enter password if necessary to enable mail checking
3. open a browser window with a lengthy document or text form
4. work with the document or form for more than 2 minutes, eg. scroll through it
by dragging the scrollbar continuously, or type text in an edit field

Actual Results:  
a) Scrolling: When the time for checking mail arrived, the scrollbar stopped
following the mouse drag. After a few seconds the scrollbar moved again with the
mouse pointer.
b) Typing: When the time for checking mail arrived, typed characters stopped
appearing in the edit field. After a few seconds (depending on network and
server load) all the character typed during the freeze appeared at once.

Expected Results:  
The scrollbar should have continued moving smoothly with the mouse pointer.
Characters should have continued to appear as they were typed.

Classic Theme.
IMAP server is Cyrus IMAP 2.0.16.
Connection is 100 MBit LAN. (WAN connection makes the problem worse.)

Comment 1

[David :Bienvenu]

how big is your inbox? This doesn't lock up the UI for me. Could you attach an
imap protocol log? In theory, we're just sending a NOOP to the server (other
than the first time we update the inbox, which can involve a lot of stuff - are
you only talking about the first connection?)
 http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

thx

Comment 2

[Robert Accettura [:raccettura]]

This couldn't be related to the POP3 issue i reported a little while back bug 217851

It sounds quite similar.

Comment 3

[David :Bienvenu]

no, I think these are unrelated.

Comment 4

[Tilman Schmidt]

Attachment: IMAP log

Zipped IMAP protocol log from about three minutes of Mozilla Mail runtime.
A GUI freeze of about five seconds occurred near the end of the log period.

Comment 5

[Tilman Schmidt]

I have the mail.check_all_imap_folders_for_new option set to true, and all the
folders in my IMAP account together currently hold about 175 MB worth of mail
according to the quota display.

I have attached an IMAP protocol log covering a GUI freeze of about 5 seconds
shortly before its end. Total period covered is about 3 minutes, the raw size an
impressive 2 MB. (zipped to 94 kB)

Am I overtaxing my poor Mozilla? ;-)

Comment 6

[David :Bienvenu]

mail.check_all_imap_folders overtaxes Mozilla - I really need to rewrite that to
use STATUS and check for /RECENT and *not* do a full mailbox sync for every
folder every time it checks. Thx for answering the question I meant to ask but
didn't :-)

However, I would have expected to see us make two passes through all the folders
checking for new messages, and I only see one. Is it the case that we don't
check all folders on startup, but only after the first check for new mail
interval? That's not how I remember the code working but I didn't write it and
it could have changed.

Comment 7

[David :Bienvenu]

Attachment: use STATUS instead of SELECT for updating folder counts

This patch is a stab at using STATUS instead of updating a folder and
downloading headers to check for new messages in non-inbox folders.

Comment 8

[David :Bienvenu]

Ere, I was hoping you could look at this and try it out and see if it does
roughly what you wanted when you did the initial implementation of this.

Comment 9

[Ere Maijala (slow)]

I don't recall doing anything but messing around with other people's code :) I
didn't have time to really dig into it, but the idea looks good. 

One thing comes to mind. Now if we don't resync the folder when checking for
mail, it must happen somewhere after the folder has been opened by the UI,
right? Now if a new message comes in and the user goes to read it, I think it
would be nice if we loaded the message first and updated the folder then. It's
actually a bit of a problem even now if the folder isn't open and a new message
is selected. Maybe also some more verbose status messages?

Comment 10

[David :Bienvenu]

It wasn't so much that you wrote the code but that you needed it badly enough to
mess around with the code :-)

We update the folder when you select it in the UI.  That's when we download the
new headers. We need to download the new headers before we can display something
in the UI for the user to select so they can display the message. I'm not really
understanding your comments...by folder open, do you mean "selected" in the imap
sense? So that you have the folder open in the ui but the folder is not selected
on the server, so that trying to read a message involves selecting the folder on
the server, etc? If so, this patch will make it so the folder is much more
likely to be selected, since the STATUS command doesn't change the selected
state of the connection. What would happen before if you had check_all_folders
for new messages set is that we'd quickly run through the available cached
connections for selected folders and the folder open in the UI could easily end
up without a cached connection.

Comment 11

[Ere Maijala (slow)]

Well, yes, I meant having the folder open in the UI while not in selected state.
Now that I think of it, using status will prevent two Mozillas from fighting for
the selected states (with uw-imap) which should also help. So, this alone will
be a great help for people using multiple clients (like me) too.

Comment 12

[David :Bienvenu]

yes, it should help for folders other than the INBOX. However, even with this
patch, we're still updating the inbox in the old way, by selecting it and
downloading new headers, so that client-side filters can run on the new headers,
etc.  The issue of INBOX contention is somewhat separate.

Comment 13

[David :Bienvenu]

Attachment: turn on the status command

this turns on the status command, but allows you to turn it off with a pref -
"mail.imap.use_status_for_biff"

Comment 14

[David :Bienvenu]

please try tomorrow's build - I've turned on the STATUS behaviour by default.

Comment 15

[Scott MacGregor]

*** Bug 220056 has been marked as a duplicate of this bug. ***

Comment 16

[Alec Flett]

this rocks! thanks SO much for making it use STATUS... just to confirm, this
will also use status when I have "Check this folder for new messages" marked for
a particular folder? I have about 10 folders marked that way, and it always
slows me down to check each one...

Comment 17

[David :Bienvenu]

yes, it uses status on when individual folders are selected to be checked for
new messages, and on all folders if you've set the pref to check all folders. 

Comment 18

[Scott MacGregor]

*** Bug 217324 has been marked as a duplicate of this bug. ***

Comment 19

[Ronny Adsetts]

This fix appears to have some effect on Bug #221792. Setting the pref
"mail.imap.use_status_for_biff" to false causes the bug to go away.

Comment 20

[Aleksey Nogin]

There is a tirival, but ugly regression that was likely caused by this - bug 230888.

Comment 21

[Aleksey Nogin]

When "mail.imap.use_status_for_biff" is enabled, Mozilla fails to notice new
mail in some folders on occasion. And somtimes, the biff just goes away
completely (Mozilla do not notice any new mail at all until I explicitly go to
that folder). I see this both in 1.6 and trunk-2004011522 on Fedora Core 1. Is
this a known issue?