422729 - IMAP unread message count frequently incorrect if using server-side mail filters

Status: RESOLVED DUPLICATE of bug 1396495

(Thunderbird :: Mail Window Front End, defect)

Description

[Bob Lord]

I have many (45) IMAP folders on my company's mail server. I also switch between several computers running Linux and Windows. When I switch between machines, the number of unread messages displayed next to each folder is very likely to be wrong since I read mail on another machine.

Also, I use server-side mail filters so I don't have to deal with copying complex filters to multiple machines. 

Can the client to ask the server for unread messages for all folders?  If not, I would propose that the client ask for the status for each folder. While I'm reading mail in my Inbox, the other folders can update in the background.

Related to this problem is that the per-folder pref called "Check this folder for new messages".  The default is to be UNchecked, which seems silly.  Why would I want to miss mail? :-)  The default should be to check for mail on all folders, regardless of where they were created (which is a different pref).

Comment 1

[David :Bienvenu]

silly for you, not as silly for people w/o server side filters, or who don't tend to access the same accounts from multiple machines.

Comment 2

[Bob Lord]

Maybe we can combine a number of these items into a combined pref? 

Is the concern the load on the IMAP servers?  

Comment 3

[David :Bienvenu]

server load is always our primary concern, when talking to server folks :-)

there's also network load, and local machine cpu load (updating an imap folder involves loading the client db and comparing the information in it against the server).

Comment 4

[Bob Lord]

(In reply to comment #3)
> server load is always our primary concern, when talking to server folks :-)

Is the check against the server expensive?
 
> there's also network load, and local machine cpu load (updating an imap folder
> involves loading the client db and comparing the information in it against the
> server).

With modern machines, are network and CPU loads very big problems?  Maybe the checking could be done somewhat slowly? 

Comment 5

[David :Bienvenu]

> Is the check against the server expensive?

It depends on the server, of course, and how many messages there are in the mail folder. The bigger the folder, the more server load, and the more network traffic and cpu load to check the folder.

Doing the checks on idle would be an improvement, instead of chaining them like I think we do now (initially they were done simultaneously, which was very painful)

Comment 6

[Bob Lord]

There may be other ways to prioritize the load.  For example, update the message count for folders that I mark as a Favorite first (or the first 10 Favorites to prevent people like me from making them all Favorites ;-) ).  Then lazily update the others. It doesn't have to be immediate, but could take place over several minutes.

Are there any new IMAP commands to make these kinds of requests more efficient?

Comment 7

[Magnus Melin [:mkmelin]]

Doesn't mail.check_all_imap_folders_for_new true do exactly what you want?

Comment 8

[David :Bienvenu]

It does what he wants, more or less, but we were discussing why it wasn't the default, and what we could do to make the general process of updating folders smoother.

I think there's a proposed imap extension that allows the client to tell if the mailbox has changed at all since the last time we checked - if it hasn't changed, then we don't need to bother with the sync at all. But I don't have the details in front of me.

Comment 9

[David M. St. Pierre]

Running Thunderbird 2.0.0.14 (20080421) on Mac OS 10.4. Retrieving mail from IMAP server. It appears the Inbox unread mail counter has an off-by-one error, i.e., still showing 1 unread email after all mail has been read. Current workaround has been to manually show folder as read.

Comment 11

[Wayne Mery (:wsmwk)]

(In reply to comment #8)
> It does what he wants, more or less, but we were discussing why it wasn't the
> default, and what we could do to make the general process of updating folders
> smoother.
loosely tied to bug 289208

(In reply to comment #8)
> I think there's a proposed imap extension that allows the client to tell if the
> mailbox has changed at all since the last time we checked - if it hasn't
> changed, then we don't need to bother with the sync at all. But I don't have
> the details in front of me.

bienvenu, Bob, is that still the case?

Comment 12

[Bob Lord]

(In reply to comment #11)
> bienvenu, Bob, is that still the case?

I haven't seen this lately, but given my change in usage patterns I don't know if that really means anything. Sorry!

Comment 13

[Peter Davis]

It's been at least 6 years since this was reported, and it's still not fixed. What is the problem? As far as I'm concerned, this is the single most obnoxious bug in Thunderbird, and it shows no sign of ever being resolved.

I'm still seeing this with Thunderbird 24.4.0 on OS X 10.9.2 ("Mavericks")

Comment 14

[Peter Davis]

Sorry, my vote and comment are in response to the title of this issue. The unread count for IMAP folders (Fastmail.fm in my case) is wrong. I do have mail.check_all_imap_folders_for_new set to "true", but the counts are still frequently incorrect.

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to Peter Davis from comment #14)
> The unread count for IMAP folders (Fastmail.fm in my case) is wrong.
> I do have mail.check_all_imap_folders_for_new set to "true", but the counts are still frequently incorrect.

Read/Unread of a mail in imap == mail with \Seen flag in  imap Mbox at server / mail without \Seen flag in imap Mbox at server.
If imap flag is changed by other client, it can be know only by:
   (a) flag change notification from imap server via IDLE
   (b) explicit "uid fetch N Flags" without (CHANGEDSINCE modseq_no) or with  (CHANGEDSINCE modseq_no) .

In detection of "flag change by other client", there are already known issues.
If CONDSTORE is used, Bug 1123617 and Bug 1123094 can occur. (recently flound)
Even when CONDSTORE is not used, Bug 693204 can occur.

Are you looking phenomenon of these bugs?

"Number of Unread at Folder pane of Tb" == "Number of mails shown as Unread at Thread Pane of Tb"?
Which mail's Read/Unread status is different(or incorrect) in Tb from actual status of the mil at imap  Mbox?
(Order Received column value in Tb == UID of the mail) if imap folder)

From perspective of "flag state", there is no difference between \Seen flag and Tag in Tb(user defined flag).
If "different Read/Unread status" occurs on a mail of UID=NN, is "add Tag/remove Taa by other client" reflected to Tb as expected?

Is there any possibility of "count shown by Tb is correct" and "count you are expecting is invalid, wrong, improper"?
For example, 
if new mail check is not executed yet by Tb, "mail count in Tb != mail count at server" is pretty normal status.
if "Tb-A's IMAP delete model setting = Just mark it s deleted" but "Tb-B's IMAP delete model setting = Move to trash",  "mail count in Tb-A != mail count in Tb-B" is also pretty normal.

> I do have mail.check_all_imap_folders_for_new set to "true", (snip)

Is "new mail alert" shown by Tb as expected?

Comment 17

[Wayne Mery (:wsmwk)]

significant, newer activity in bug 1396495