1284471 - Not showing unread mail count for mail in folders other than Inbox for office365 server side filters

Status: RESOLVED DUPLICATE of bug 1396495

(MailNews Core :: Networking: IMAP, defect)

Description

[Lauri Loebel Carpenter]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

Steps to reproduce:

My mail server is an office365 server.  I have it autofilter mail to many different folders.  Until recently Thunderbird would show unread mail counts in other folders (after I did much configuration of things like mail.check_all_imap_folders_for_new=true and mail.server.default.check_all_folders_for_new=true and mail.imap.use_status_for_biff=true as suggested by http://kb.mozillazine.org/Checking_for_new_messages_in_other_folders_-_Thunderbird).  Recently I do not see any unread count in other folders until I manually click on the folder name to move into that folder. 


Actual results:

Not seeing new (unread) mail count in other folders until I visit that folder.


Expected results:

Should see an unread count for all folders to which I am subscribed (if they contain new unread mail) without having to visit the folder.

This is an absolute show-stopper for me -- if I am missing emails because Thunderbird isn't showing me that they exist, I will be forced to migrate to a different email client.  I sure hope this can be fixed ASAP!

Comment 1

[Lauri Loebel Carpenter]

Apparently this has been going on since at least May 13 2016.  (I just walked through each and every single email folder and found old mail in lots and lots of places; the oldest unseen unread message is from May 13).  

Complicated by the fact that during the time since 13 May 2016 I *have* received new mail notification for *some* folders ("frequently visited folders", perhaps), but not *all* folders.  All of these folders have the "subscribe" feature checked.

Comment 2

[Lauri Loebel Carpenter]

The problem is even worse.  I walked through all of my folders, and for each I clicked on "Properties" and then checked the box that says "When getting new messages for this account, always check this folder".  That was a long and tedious process, but it had the side effect of updating the new message count for each folder as I visited it.

Then I exited Thunderbird, and from a different account I sent myself a message that I knew would be auto-filtered to one of the non-inbox folders.  Then I opened Thunderbird again.

And here is where it gets worse: when Thunderbird opened, it still had the cached unread count for each of the folders I had walked through.  But as the spinning little icon spun around and Thunderbird tried to look for new mail, the unread mail count went back to being completely blank for every single folder.  Checking the box had no effect whatsoever, and updating the folders got "the wrong answer".  (But if I manually move to the folder in question, it immediately updates with the correct unread mail count -- which is not acceptable!).

In case it matters, my mail server is some office365 thing "in the cloud".  

-- lauri

Comment 3

[Lauri Loebel Carpenter]

Does anybody in the Thunderbird developer community really ever look at these bug reports?  I'm getting very very depressed.  I guess I will have to drop thunderbird as my email client, after many long years.

What a pity.

Comment 4

[richard rigby]

this same problem has causing me trouble for the last few weeks.

i *think* it's actually a problem with the microsoft imap service, rather than any issue with thunderbird ...

if i switch on imap logging in thunderbird, i can see this information being logged at the exact moment the unread count disappears from one of the folders:

  STATUS pathto/folder (UIDNEXT 174696 MESSAGES 7 UNSEEN 0 RECENT 0)

the unread count in this particular case should be 4.

if i open the folder, the status of the messages is checked, and the unread counts updates / displays correctly.

so, it seems that the microsoft imap servers may be sending incorrect information to the client ...

thanks,

richard

Comment 5

[Lauri Loebel Carpenter]

Given that other IMAP clients around here (specifically, the mac email client) are NOT seeing this problem, I can't necessarily agree that it is a microsoft imap service problem.  

I've been trying lots and lots of different things, and yesterday I stumbled upon something that, so far, appears to have "fixed" the problem: see http://www.techiecorner.com/1737/how-to-clear-thunderbird-imap-cache/.

By removing all of the local folders, and then letting thunderbird rebuild everything, I am now seeing the new mail count of non-Inbox folders.

This makes me suspect some buffer-overflow (too many messages, too many folders, too much stuff, ???) problem in my local Thunderbird rather than problems with the server.

Still haven't heard anything from the Mozilla/Thunderbird support community, which is kinda depressing, but at least I seem to have staved off the day when I am forced to migrate to using Outlook.  I'd rather poke forks into my eyes.

-- lauri

Comment 6

[richard rigby]

i had a bit of a closer look ...

using a bit of python, firstly to use the imap status command to check for unseen messages, and then to select the folder, and search for unseen messages.

roughly:

  import imaplib
  import getpass
 
  m = imaplib.IMAP4_SSL('imapserver','993')
  m.login('username', getpass.getpass())
  status0 = m.status('INBOX','(UNSEEN)')
  m.select('INBOX')
  status1 = m.search(None,'UNSEEN')
  m.logout()
 
  print status0
  print status1

the microsoft / office 365 server returns:

  ('OK', ['INBOX (UNSEEN 0) '])
  ('OK', ['1 4 5 7 9 10 11 12 13 14 15 16 19 20 21 22 23 24'])

the unseen count returned from the status command is always 0 ... querying other folders produced the same results.

output from other imap accounts / servers, where things behave correctly:

  ('OK', ['INBOX (UNSEEN 9)'])
  ('OK', ['4 5 6 7 8 9 10 12 14'])

  ('OK', ['INBOX (UNSEEN 1)'])
  ('OK', ['2'])

fortunately, it appears there is already a thunderbird configuration option to deal with this:

  mail.imap.use_status_for_biff

which can be set to false. see:

  http://kb.mozillazine.org/Offline_folders#Configure_the_folder_for_offline_use

particularly:

  "If Thunderbird doesn't recognize that a folder contains unread messages try setting mail.imap.use_status_for_biff false. It causes Thunderbird to explicitly select each folder to update the message summaries, rather than use the STATUS command. It has more overhead, but some IMAP servers don't return STATUS correctly."

have only given this a quick test, but it seems to do the job.

thanks,

richard

Comment 7

[Wayne Mery (:wsmwk)]

richard, thinks for so many details.  So setting mail.imap.use_status_for_biff  to false solve the problem?

Comment 8

[Wayne Mery (:wsmwk)]

> Still haven't heard anything from the Mozilla/Thunderbird support community,
do you mean in support.mozilla.org ?

> Apparently this has been going on since at least May 13 2016.
Have a look at tool | options | advanced | update | show update history.
What dates and versions are shown there?

Comment 9

[Javi Rueda :javirid]

I am not an Office 365 user, so unable to test or give any thought on it.

Comment 10

[richard rigby]

hi wayne,

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #7)

> richard, thinks for so many details.  So setting
> mail.imap.use_status_for_biff  to false solve the problem?

that seemed to get everything working fine - no problems since setting this option.

the issue does seem to be with microsoft mail server behaviour, rather than any problems with thunderbird.

thanks,

richard

Comment 11

[Lauri Loebel Carpenter]

This was so long ago that I barely remember what I did to fix it.  Setting mail.imap.use_status_for_biff=false was NOT enough to fix it in my case.  I think I ended up deleting the local folders from my Windows (at work) and Mac (at home) file systems, which caused thunderbird to then rebuild the local file cache completely.  I forget where I read about this, something I googled pointed me at this solution.  Sorry I don't remember more about it, this was several months ago.  

-- lauri

Comment 12

[Wayne Mery (:wsmwk)]

duping to bug 1396495 which has greater detail