162342 - Mail inbox shows wrong message count

Status: RESOLVED FIXED

(SeaMonkey :: MailNews: Message Display, defect)

Description

[Sheela Ravindran]

This could be related or eventually a dup of bug 121660. Searched bugzilla and 
was not able to find anything.   
Many people have reported that their inbox was showing a wrong message count. 
Saw this on Steve Elmer's machine as well.  There were only 4 new 
messages and the message in the brackets showed 6 new messages.  Logging a new 
bug to track this problem that has been reported on and off. 

There are existing bugs for wrong message count in newsgroups.
Bugs: 34406 and meta bug -71728 which track the mssg count problem.  But did not 
see anything for mail.

From Esther's comments:

Removing the inbox.msf corrected the problem for people who were seeing this.  
However,  replacing the old msf file did not reproduce the problem either. So 
this probably is not due to msf file corruption. Bug 121660 has comments that 
this could be a result of panacea.dat not getting updated when we close the 
application.

Comment 1

[scottputterman]

reassigning to bienvenu.  David, any ideas on what a number of people are seeing
wrong mail counts?

Comment 2

[David :Bienvenu]

I really don't know - I suspect that the db count isn't getting put into
panacea.dat in some situations - perhaps due to a crash or panacea.dat not
getting written out on shutdown.  I'll try to recreate this.

Comment 3

[Ian Neal]

I get this or something very similar happening to me alot, especially after
opening an attachment I get an extra unread email which consists of a no-subject
message dated 1970 or 1980 or something like that. Usually I can get rid of it
by switching to another folder and back again but sometimes I have to exit
mozilla, go into my profile and delete the inbox.msf file.
I didn't see this in 2002080508 but I think I have in 2002082008 and definitely
in 2002082814+

Comment 4

[Antony Mawer]

I've noticed this on recent builds (20020819 atm) manifesting itself in the
little popup on lower-right of the screen not showing the correct message count
from time to time. I'll see if I can get something more informative/concrete
than that to contribute sometime soon.

This is connecting to an Courier IMAP server on my local network, with various
subfolders. I'm not sure how Mozilla handles new message counts with multiple
folders - does it recurse through the folder list and sum together each
individual folders unread message count to give the total?

Comment 5

[Antony Mawer]

Just as an example then, the little popup appeared telling me I had 17 unread
mail messages. That compared with the total of 3 unread messages I _actually_
had -- 2 in one folder, one in another.

Comment 6

[David :Bienvenu]

that popup has nothing to do with this bug - that popup tells you the cumulative
new messages in all your folders since you started the app (where new msg is
defined as one that was put in the folder after the last time you opened the
folder).

Comment 7

[David :Bienvenu]

Attachment: proposed fix

this fix makes it so we ignore hdrs with an invalid uid, and so that we
initialize the hdr cache with the invalid uid. This fixes the problem that you
get the msgs with date in 1969 when you open imap attachments. The root problem
has to do with the imap parser thinking we're downloading hdrs when we're
actually downloading the attachment. I'll try to fix that as well, but that's
much more scary, and this fixes the problem the user sees.

Comment 8

[David :Bienvenu]

Cavin, can you review this? thx.

Comment 9

[(not reading, please use [e-mail] instead)]

Comment on attachment 97883 [details] [diff] [review]
proposed fix

sr=sspitzer

Comment 10

[Cavin Song]

Comment on attachment 97883 [details] [diff] [review]
proposed fix

r=cavin.

Comment 11

[Asa Dotzler [:asa]]

Comment on attachment 97883 [details] [diff] [review]
proposed fix

a=asa (on behalf of drivers) for checkin to 1.2a

Comment 12

[Fused]

*** Bug 170678 has been marked as a duplicate of this bug. ***

Comment 13

[David Hagood]

I've been seeing this for the past month or so - not only is the count wrong,
but if you do an "Empty Trash" the count on the screen is not updated.

If you then do a "Compact folders", or if you select the trash folder, then the
count is updated.

Comment 14

[Fused]

Still showing wrong counts in 1.2b.

Is there some reason this hasn't been fixed yet? I see a patch with r, sr, and
approval above ...

Comment 15

[David :Bienvenu]

there seem to be several problems that cause this. There are several bugs still
open for problems of this nature - the patch in this bug fixed one case that was
easily reproducible by downloading an attachment, and is fixed. There still seem
to be other problems with the unread count that are not reliably reproducible -
if someone has simple steps to reproduce the problem, that would be very
helpful. If you ran a build that had this problem that this patch fixes, you
would need to delete your .msf file to get the problem resolved.

Comment 16

[David Shochat]

I opened 169870 on 11/2 and I would say that it is closely related to this. I 
have been wondering why no one has responded to it in any way. The strange thing 
is that I have not found a problem with the Inbox count, only the counts for 
other folders when adding new messages to them. But I'm glad to see from this 
bug that I'm not the only one who has been seeing message count anomalies.

Comment 17

[Rob Sherwood]

I still see this bug in Mozilla 1.4b.  My inbox still shows a higher message
count than the actual # of messages.  Here is one (awkward) way to fix it.  Exit
Mozilla.  Delete the INBOX.MSF file.  Restart Mozilla.  It will rebuild the
INBOX.MSF file and show a correct message count.  Do a file search to find
INBOX.MSF.

Comment 18

[Gábor Lipták]

Yes this still shows in

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5a) Gecko/20030603

Started coming up about a month ago (with current builds).

Very annoying together with the 

"Unable to delete messages in folder Inbox because it is in use by some other
operation."

Comment 19

[Gábor Lipták]

deleting inbox.msf and/or compressing the mailbox only temporary helped in my
case. the message count starts growing soon (with every POP mail download?)

(I also noticed that the mailbox grew to an unusual size and "phantom" messages
started showing up with dates 1969?)

Comment 20

[Gábor Lipták]

also I'm running

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4b) Gecko/20030507

on a different machine which does not seem to have this problem

Comment 21

[David :Bienvenu]

Gabor, are you using IMAP or POP3? Do you have mail filters set up?

Comment 22

[Gábor Lipták]

pop

I had one filter set up (I just removed it), but the filter shows no activity.

I do run the junk mail filters (those are pretty active)

Comment 23

[Daniel Wang]

Comment on attachment 97883 [details] [diff] [review]
proposed fix

looks like the patch was already checked in.

Gábor, do you have messages imported from elsewhere? that may cause the problem

Comment 24

[Gábor Lipták]

importing as from outlook? no

but I compressed the folders (which one would think recreates a brand new mail
file), removed msf file, to no good effect

the one "strange" thing I'm doing is to run the same profile/mail folders with
multiple versions of Mozilla ... 

Comment 25

[David :Bienvenu]

Not at the same time, I hope? That shouldn't be possible anyway, but if you're
doing it somehow, it will definitely break things.

Comment 26

[Gábor Lipták]

nope :), it is smart enough to tell me when the profile is in-use

Comment 27

[Gábor Lipták]

ok, it seems that this bug is not going away so I rolled back 1.5a to

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4b) Gecko/20030507

which does not seem to experience this problem.

someone with CVS access being setup may want to check if this patch was lost
when transitioning to 1.5a ...

Comment 28

[Gábor Lipták]

and I'm also seeing this with

ThunderBird 0.1a (20030708)

(probably the same codebase?)

Comment 29

[David :Bienvenu]

yes, same backend codebase. I'll have another crack at trying to reproduce this
at some point.

Comment 30

[Matthew Darwin]

For what it's worth, I've been seeing this bug for months now.  My inbox was
showing 4 more messages than actually existed.  I checked the IMAP server 
back-end and the messages were not there.

Removing the .msf file had the expected result... the count is now correct.

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624

Comment 31

[Nick Hansen]

I'm seeing this problem as well. I'm connecting to an IMAP server, with many
filters enabled. Removing the msf file works to fix the problem for a while, but
is such a PITA as all my prefs get blown away. I'm running this version:

 Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030922

Has there been any progress made on this issue?

Comment 32

[David :Bienvenu]

*** Bug 227284 has been marked as a duplicate of this bug. ***

Comment 33

[David :Bienvenu]

Can you try 1.6b? there's a fix in there that might prevent the counts from
going wrong. 

Comment 34

[Henrik Gemal]

I often also see negative numbers. Perhaps a IMAP log statement could be added
if the number is negative?

Comment 35

[Henrik Gemal]

Attachment: screenshot with negative and ??? numbers

Comment 36

[David :Bienvenu]

wow, I've not seen that. That's the unread column, right? The IMAP log doesn't
know about the msg counts in the folder. However, there's code that should
prevent the unread count from getting negative, at least in the db. Does the
count stay negative when you select the folder?

Comment 37

[David :Bienvenu]

the negative counts Henrik saw were due to the new use of STATUS to check imap
folders for new messages, and has been fixed. I have a patch upcoming that
attempts to correct the counts when they've gone wrong...

Comment 38

[David :Bienvenu]

Attachment: patch to detect inconsistent counts and correct them

this patch has a bunch of changes:

1. Remove a couple unused nsIMsgDatabase methods (hasThreads and
AllMsgKeysImapDeleted)
2. fix nsIDBFolderInfo use of "numNew" to be "numUnread", which is what the
counts really are.
3. Add a SyncCounts method which iterates over the db counting total messages
and unread messages, and if this differs from our global counts, make the
global counts match the calculated counts.
4. Try to detect potential mismatched counts and call SyncCounts if we think
the counts might be wrong. We do this when we get told the unread count from
the imap server, when we create a flat view (which allows us to easily iterate
through the flags counting the unread messages), and when we open a db, at
which time we can easily see if the total number of headers in the db matches
the total count in the nsIDBFolderInfo.

Comment 39

[David :Bienvenu]

Ignore the nsMessenger diffs - they're unrelated.

Comment 40

[David :Bienvenu]

fix checked in, but the imap part is worrisome - some servers are not reporting
the correct unseen count...

Comment 41

[Bill Gianopoulos [:WG9s]]

Just a thought.  On the IMAP issue, should imap.use_status_for_biff be a per
account preference and externalized to the advanced server settings in the GUI
so it can be turned off for ill behaved servers?  Would this fix the IMAP issue
with servers returning incorrect unseen counts?

Comment 42

[David :Bienvenu]

fixed on m4 branch