28355 - IMAP CHECK - Synchronize with other svr connections

Status: VERIFIED FIXED

(MailNews Core :: Backend, enhancement, P3)

Description

[Ben Bucksch (:BenB)]

I have a non-Mozilla biff running. If I read a msg with Mozilla (or 4.x),
Mozilla doesn't update the state to READ (not even SEEN?), so my biff keeps
notifying me of "new mail", which isn't new anymore. I have to close (!)
Mailnews (or Messenger) in order to update.

Please give me an option to syncronizise with the server, when Mozilla gets new
mail. At least set SEEN plag, better also READ flag and "expurge" (delete mails
on server, which I deleted in the client).

Comment 1

[Phil Peterson]

cc bienvenu. David, can/should we do this, or should this go into the helpwanted
pile?

Comment 2

[David :Bienvenu]

SEEN is up to the server to set, as is READ. This sounds to me like a problem 
with multiple connections to the same folder not getting synchronized on the 
server. We could optionally do a CHECK command to encourage the server to 
synchronize the connections. I guess this could be a help-wanted bug. There are 
other similar help-wanted bugs about issuing CHECK, I believe, though.

Comment 3

[Ben Bucksch (:BenB)]

May well be, that I'm wrong with the IMAP details. Note however, that the server
is updated (i.e. the biff stops notifying), when I close Messenger (/Mailnews?),
so there must be anything done, which isn't done during normal runtime.

I'm using the UW-IMAP server, if that matters.

Comment 4

[David :Bienvenu]

yes, when we shut down, we logout of the connection, which makes the imap server 
flush the state (basically equivalent to doing a check).

Comment 5

[Phil Peterson]

Couldn't find a helpwanted bug related to this. Sending to the helpwanted list.

Comment 6

[Ben Bucksch (:BenB)]

Not just SEEN/READ, but also delete.

Comment 7

[David :Bienvenu]

yes, these are all part of the mailbox state that the server is not 
synchronizing between open connections.

Comment 8

[John G. Myers]

Servers don't want clients to CHECK every time they read a message.  CHECK is 
expensive, especially for c-client berkeley format mailboxes.

It really should be the server's job to figure out when to checkpoint.  I 
suppose the client could issue a CHECK after, say, 5 minutes of inactivity after 
it knows it has done a state change.

Comment 9

[Ben Bucksch (:BenB)]

John,
I didn't mean to CHECK after the user read a msg, but when we "Get New Mail",
i.e. when we ask the server, if new mail arrived. This is done by default only
every 10(?) mins anyway.

It could also be a pref.

Comment 10

[David :Bienvenu]

yes, I was referring to issuing a check after Get New Mail and associated 
filters run (which I believe you or someone else on the server team suggested to 
me in the 4.5 days) - this would fix the problem where messages get filtered 
twice if you have two clients running. Yes, this should be under the control of 
a preference, as it is today.

Comment 11

[John G. Myers]

It might make sense to do a new mail check using the CHECK command instead of 
NOOP if the client knows it has changed state.  It may make sense to have the new 
mail check done using CHECK all the time.

If client-side new mail filters exist and they have made modifications, then it 
might make sense to then do a CHECK.

If client-side filters are enabled, we may want the new-mail operation be NOOP 
instead of CHECK, with the assumption that a CHECK will be done after the new-
mail check.

I'd like to avoid having a preference.  I'd also like to keep in mind a future 
architecture where the client downloads captions whenever it notices new mail, 
instead of requiring the user to explicitly do a "get new mail" action.

Comment 12

[Ben Bucksch (:BenB)]

Nominating for nsbeta3. Cannot use the product without this bug or bug 42456
(which is [nsbeta3-]) being fixed.

Comment 13

[David :Bienvenu]

Just to be clear, Mozilla and 4.x are the same in this respect, right? Also,
would it be sufficient to make the following pref work in 6.0 the way it works
in 4.x?

("mail.imap.noop_check_count", 1) - if this pref is set to 1, every biff will be
done with a check, not a noop. This will force mozilla to  flush its inbox state
to the server at every biff interval. This preference currently works in 4.x,
but doesn't in 6.0 (though it's a one line change to add it back to 6.0, which
I've already made in my tree - it should have been there all along).

Comment 14

[Ben Bucksch (:BenB)]

> Just to be clear, Mozilla and 4.x are the same in this respect, right?

Yes, it was just my (over-)reaction to the other bug being minused. We need to
fix the otehr bug anyway, so I'll remove the nomination for this bug.

> ("mail.imap.noop_check_count", 1) - if this pref is set to 1, every biff will
> be done with a check, not a noop.

Cool, this would be a workaround. Didn't know this pref, will try out in 4.x.
Personally, I prefer to leave the biff off and do that manually. So, will your
fix work with a manual "Get New Mail" as well? We'd have this bug fixed then,
not?

I can check this in, if that helps.

Comment 15

[David :Bienvenu]

Here's the diff for 6.0 :

Index: nsImapProtocol.cpp
===================================================================
RCS file: /cvsroot/mozilla/mailnews/imap/src/nsImapProtocol.cpp,v
retrieving revision 1.283
diff -c -r1.283 nsImapProtocol.cpp
*** nsImapProtocol.cpp	2000/08/05 14:27:46	1.283
--- nsImapProtocol.cpp	2000/08/09 23:16:17
***************
*** 186,191 ****
--- 186,192 ----
                         &gHideOtherUsersFromList);
      prefs->GetBoolPref("mail.imap.hide_unused_namespaces",
                         &gHideUnusedNamespaces);
+     prefs->GetIntPref("mail.imap.noop_check_count", &gPromoteNoopToCheckCount);
    }
    gInitialized = PR_TRUE;
    return rv;

If you want to check that in, that would be great. (Try it and make sure it
works first :-) )

Yes, that pref affects both biff and manual get new mail.

For 4.x, running on windows or mac, bonking the offline icon will cause us to
logout of all the open imap connections, which effectively flushes their state.
This doesn't work in 6.0 (yet) because we don't get told about the offline state
change because of a bug, whose number I don't know offhand, but I filed it on necko.

Comment 16

[Ben Bucksch (:BenB)]

bienvenu: thanks! you saved me. Will check out the patch, and check in, if it
works :).

Comment 17

[Ben Bucksch (:BenB)]

Tried the patch - works. Seeking approval.

Comment 18

[Ben Bucksch (:BenB)]

a=brenden. checked in.

Bienvenu, thanks again.

Should we enable this by default?

Comment 19

[David :Bienvenu]

cool, thanks for checking it in. No, I don't think it should be the default
since it can increase load on the imap server, and isn't needed in the normal
case of a single client running against a server.

Comment 20

[Ben Bucksch (:BenB)]

OK. Is it worth UI? 4.x had it, and I didn't know of it until you told me here.

Comment 21

[David :Bienvenu]

It's such a power user feature that I don't think it should it have a UI. We do
need to document our hidden prefs better (in 4.x, they were documented for
enterprise customers in the customization documentation, but we need something
similar for 6.0, perhaps online). Seems like an ideal thing for a mozilla
volunteer to write up.

Comment 22

[John G. Myers]

The client should just Do The Right Thing.  A pref makes no sense here.  It would 
be reasonable to act as if the pref were always 1.

Comment 23

[Ben Bucksch (:BenB)]

Just remembered one more case were this matters: If we crash or are otherwise
unable to write out the data later. In that case, I sometimes ended up with msgs
in INBOX, which I had already deleted or moved. I.e. this pref makes sense even
in a single-client setup. And don't tell me, we won't crash ;).

Yes, we certainly need better docs for hidden prefs. I assume, we will have them
:).

Comment 24

[Karen Huang]

QA->me

Comment 25

[Karen Huang]

Verified on Windows 12-05-09-trunk build:

After set that pref 
user_pref("mail.imap.noop_check_count", 1) 
to 1, all NOOP can be replaced by CHECK. 

After set that pref to 2, get new msg 4 times will still get 2 NOOP and 2 CHECK. 
After set that pref to 3, get new msg 6 times will still get 4 NOOP and 2 CHECK.

Will verify on other platforms and check with Scott for documenting the hidden 
prefs for 6.x tomorrow...

Thanks David for providing the info.

Comment 26

[Karen Huang]

Verified on Mac 9.2.1 Build 12-06-04-trunk build
Verified on Linux 6.1 Build 12-06-08-trunk build
After set that pref 
user_pref("mail.imap.noop_check_count", 1) 
to 1, all NOOP can be replaced by CHECK. 
But, there is NSPR bug 113935 logged for IMAP log problem specific for Linux 
platform. Marking as veriifed.