239196 - Folders "borrow" the unread count from other folders, confuses folders.

Status: RESOLVED FIXED

(SeaMonkey :: MailNews: Message Display, defect)

Description

[Aleksey Nogin]

I just upgraded 1.6 (where I had use_status_for_biff set to false) to today's
trunk build (2004033001) and I am now having problems with the unread mail
counts on biff. Very often a folder that is currently empty would be suddenly
shown to have the total number of messages = the number of unread messages = the
number of unread messages in some other folder on the same server. When I click
on the folder in the folders pane, the folder is (correctly) revealed to still
be empty.

P.S. In many ways this is similar to the (fixed) bug 230888.

P.P.S. I am using namespaces to only show a subset of all the folders on the
server. I see some new weirdness (not yet reported as I am not quite sure what's
going on) in that area; namely Mozilla does show out-of-namespace folders in
certain circumstances. I am only mentioning this because of a possibility that
biff gets confused by these "out-of-namespace" folders.

Comment 1

[Aleksey Nogin]

Hm, just noticed - this is not just the empty folders. I have a folder with 3
messages, all read. I just saw this folder suddenly marked with 35 unread
messages and 38 total (where 35 is the unread count of the folder right after it
and 38, I would guess, is just 35 + 3)...

Upping the severity, as this is very annoying (among other things it results in
erroneous "new mail" notifications).

Comment 2

[David :Bienvenu]

Ere sent me a log that had this problem. I'm trying to figure out how it could
be getting the folders confused.

Comment 3

[Aleksey Nogin]

This keeps happening all the time, will random folders getting wrong counts, not
just empty ones. And it looks like it is not just biff either - sometimes when
Mozilla gets confused about a folder during biffing, if I go to that folder (by
clicking on it in the folder pane), then for a brief momdent I see the headers
from the wrong folder, then the header pane becomes empty, then it looks as if
Mozilla is refetching something before it shows the headers again.

In all, this makes Mozilla Mail close to unusable - I can probably tolerate it
for a few days in hope of seing some pattern, but it's bad. Depending on how
common it is and how many users are affected this has potential of being a 1.7
blocker...

Comment 4

[Aleksey Nogin]

Oh, and forgot to say - I see not only false positive biff notifications, but
false negatives too - sometimes a folder would stay unmarked for hours after
receiving several new messages. This makes this bug especially unfriendly.

Comment 5

[Aleksey Nogin]

> ... I see the headers
> from the wrong folder, then the header pane becomes empty, then it looks as if
> Mozilla is refetching something before it shows the headers again.

Just saw it again - and this time the folder was big enough to give me a chance
to look in the title bar - Mozilla was indeed refetching _all_ headers for the
folder!

Comment 6

[David :Bienvenu]

Definitely a blocker, imo. Aleksey, can you attach or e-mail me a protocol log
showing this happening, with annotations as to which folder had the wrong counts
in the UI, etc? Thx.

Comment 7

[David :Bienvenu]

Attachment: proposed fix

I didn't reproduce this, but I believe I know what's going on, and I think this
should fix it. Basically, for IMAP Idle support, we were hanging onto the
nsImapMailFolderSink, which meant we could end up using the wrong front end
imap mail folder when a protocol connection switched folders it was operating
on. Aleksey, if you have a source tree, it would be great if you could try this
patch.

Comment 8

[David :Bienvenu]

Ere, if you have a source tree, could you try this as well? thx.

Comment 9

[David :Bienvenu]

Comment on attachment 145177 [details] [diff] [review]
proposed fix

hmm, this fix will only help if your server supports IDLE - I'll need to fix it
for the general case.

Comment 10

[Ere Maijala (slow)]

I'll try it.

Comment 11

[David :Bienvenu]

Attachment: fix for non-idle imap servers

Comment 12

[Scott MacGregor]

Comment on attachment 145181 [details] [diff] [review]
fix for non-idle imap servers

Hopefully this will fix the issue I was seeing too!

Comment 13

[David :Bienvenu]

Comment on attachment 145181 [details] [diff] [review]
fix for non-idle imap servers

yes, I hope it should fix your issue too - using the wrong folder would
definitely make it look like uid validity had changed.

Comment 14

[Aleksey Nogin]

Attachment: A well-formed version of the above patch.

Tha patch above is ill-formed, I am attaching the corrected one. Will try
finding some time to compile and test it today.

Comment 15

[Ere Maijala (slow)]

Running a build with the second patch. In the beginning everything seemed ok. I
left Mozilla running through the night and in the morning problems started. Junk
wasn't moved from inbox (don't know if that's related) and when I selected
another folder with three new messages, two other folders started increasing
their unread count by three every second. I switched to one of the folders with
the increasing number and it was corrected, but now two other folders started
increasing their numbers by one every second. I clicked on one unread message
and a completely wrong message was loaded from a different folder. Something
still seems broken.

Comment 16

[Ere Maijala (slow)]

Also, repeatedly pressing Get Msgs causes one folder to show 32 new for a
fraction of a second and then change it to 1 (32 was the correct count). There
is another folder with 1 unread message. Also happens with the current folder
that has no new messages. First it shows it normally for a fraction of a second,
then makes it bold and says there are 2 new messages. There is another folder
that really has 2 new messages.

Comment 17

[David :Bienvenu]

Attachment: fix one more case...

I missed one case where we should have been clearing the imap mail folder sink,
and that's when there's already a url ready to run (and if you check a lot of
folders for new mail, that will be the case sometimes). I hope this is
well-formed - I have to edit out a lot of other diffs.

Comment 18

[Ere Maijala (slow)]

So far no problems. I've been using the latest patch for about 14 hours.

Comment 19

[David :Bienvenu]

Comment on attachment 145261 [details] [diff] [review]
fix one more case...

carrying forward mscott's sr, requesting 1.7 approval - this bug will be very
noticeable to people who check folders other than the inbox for new messages
(and there seem to be a lot of those folks :-) )

Comment 20

[Mike Kaply [:mkaply]]

Comment on attachment 145261 [details] [diff] [review]
fix one more case...

a=mkaply

Comment 21

[David :Bienvenu]

fix checked in.

Comment 22

[David :Bienvenu]

*** Bug 238353 has been marked as a duplicate of this bug. ***

Comment 23

[Aleksey Nogin]

The original problem indeed appears to be fixed, but not all folders marked to
be checked are being properly checked for incoming messages. Any ideas whether
this is something related, or somethign new? How would I go about debugging it?

Comment 24

[Aleksey Nogin]

I just upgraded from BuildID 2004050518 (1.7 branch) to BuildID 2004053123
(trunk) and now this bug appears to be back:

- for forders that have a bunch of unread messages (but no new ones), it shows 0
unread most of the time and true # of _read_ messages in the "total" column. 
- somtimes it borrows the number of unread from another folder, but the number
of read is still correct (e.g. it shows a bogus unread number and a "bogus
unread + actual read" in the "total" column).
- sometimes it does show things correctly,

Any ideas what this is?

Comment 25

[David :Bienvenu]

is it possible that these are folders you've opened in the UI and thus have a
connection to? I checked in a fix that made it so we won't issue STATUS on
folders that we have a cached connection to; instead we just issue a NOOP,
because it's illegal to issue a STATUS on a folder you have in the selected
state. Can you attach a log where you have this problem?

Comment 26

[Ashley Bischoff (blog at handcoding.com)]

Related to bug 245044?

Comment 27

[David :Bienvenu]

re-closing. this is a different bug, with similar symptoms (but not identical).
There is a potential fix in  bug 245044