1916455 - 'Unread count' on Taskbar icon incorrect caused by including hidden POP Inbox which was deferred to "Global inbox"

Status: NEW

(Thunderbird :: Folder and Message Lists, defect)

Description

[Anje]

This bug report has been recreated at the request of Francesco
Originally - bug 1916107 Another person reported a similar issue but count for new mail and it was decided I should recreate the original bug report ...so here it is - it's not a duplicate.

Windows 10 OS
Beta 130.0b3

No chat.
Just two imap accounts and one pop account.
Note: The Pop account was originally set up as a normal mail account using it's own folders, but settings have been since altered and now Pop account uses Global Inbox (Local Folders).

Using setting : View > folders > All
Not unified etc.

• In Settings > General
• 'Incoming Mails' section
• click on 'App icon Options'
• select 'Count of unread messages'
• Force display to function by restarting Thunderbird

Actual Results
Thunderbird beta Icon badge on Taskbar says 78 unread messages.
Actual total number of unread messages across two imap accounts and one pop account using Local Folders as Global Inbox is 608
IF you only count the unread messages in Only the Inboxes of all accounts (2+9+46) the total is 57
But as most emails get auto filtered and moved to different folders when incoming - the unread mail is mainly not in an 'Inbox'.

• preference: mail.notification.count.inbox_only is set to true, so only inboxes are counted,

Expected Results
As total of unread in all the Inboxes of all accounts (2+9+46) the total is 57
Thunderbird beta Icon badge on Taskbar to say 57 unread messages.

To Debug:

• Settings > General
• Scroll to bottom and click on 'Config Editor'
• set preference: mail.notification.loglevel to All
• open and clear 'Error Console'
• Exit Thunderbird and restart
• Check 'Error Console'

Actual Results in Error console
Info says:

mail.notification: mailbox://nobody@Local%20Folders/Inbox has 46 unread MailNotificationService.sys.mjs:97:19
mail.notification: imap://[e-mail]/INBOX has 2 unread MailNotificationService.sys.mjs:97:19
mail.notification: imap://[e-mail]/INBOX has 9 unread MailNotificationService.sys.mjs:97:19
mail.notification: mailbox://[e-mail]/Inbox has 21 unread MailNotificationService.sys.mjs:97:19
mail.notification: Unread mail count changed to 78 MailNotificationManager.sys.mjs:169:18

Discovered:
The pop account says 21 unread, but the pop account is not set up as a separate account - it uses 'Local Folders' Inbox which says 46

So it seems the count of pop hidden Inbox is added thus count is 78. (57 + 21)
That explains the count anomaly, but still seems odd.

It looks like the unread count is including the pop account Inbox prior to swapping account to a Deferred account. So including data from an unseen/hidden Inbox - not in Folder Pane because account is a deferred account.

Verified
Access profile - the 'Mail' folder shows the pop account and the Inbox mbox file has the 21 unread emails.

Process

• Emptied error console.
• Exit Thunderbird
• Access profile pop Inbox mbox file and copied emails into a new mbox called 'Old Pop' which I put into Local Folders.
• Empty the pop account Inbox file - save it and delete the Inbox.msf file

It now says Inbox size is 0 and it really is empty,
So that account Inbox should now not be included as having unread mail when I restart Thunderbird.

• Restart Thunderbird

Actual Results

• I've got an 'Old pop' folder in Local Folders - so I can see those 21 emails.
• checked error console
• unread count in icon on taskbar remained the same 78 and the pop account data still said it had 21 mails !
• mail.notification: mailbox://[e-mail]/Inbox has 21 unread MailNotificationService.sys.mjs:97:19

Expected REsults

• 'Old pop' folder in Local Folders
• unread count in icon on taskbar to say 57 matching total of Inbox unread.

Check folderCache.json file
Access profile and 'folderCache.json' file has this:

"C:\Users\User name\AppData\Roaming\thunderbird\Profiles\mni4eecl.default-beta\Mail\mail.livemail.co.uk\Inbox.msf" :
{
"expungedBytes" : 0,
"flags" : 4100,
"folderName" : "Inbox",
"folderSize" : 646121,
"pendingMsgs" : 0,
"pendingUnreadMsgs" : 0,
"totalMsgs" : 21,
"totalUnreadMsgs" : 21,
"useServerRetention" : "1"
},

It seems the folderCache.json does not get updated if manually altering profile mbox and .msf files.
It seems the count does not take into account when a pop account is swapped over to use a Global Inbox, so unseen folders are being included in the count.

Process:

• Empty Error Console

• Exit Thunderbird

• Access profile

• renamed 'folderCache.json' to 'folderCacheold.json'

• start Thunderbird

• new 'folderCache.json' is ceated.

Actual Results
Check folderCache.json file
It looks correct.
"C:\Users\User name\AppData\Roaming\thunderbird\Profiles\mni4eecl.default-beta\Mail\mail.livemail.co.uk\Inbox.msf" :
{
"expungedBytes" : 0,
"flags" : 4100,
"folderName" : "Inbox",
"folderSize" : -1,
"pendingMsgs" : 0,
"pendingUnreadMsgs" : 0,
"totalMsgs" : 0,
"totalUnreadMsgs" : 0
},

Check Error Console
error console info has updated:

• mail.notification: mailbox://[e-mail]/Inbox has 0 unread MailNotificationService.sys.mjs:97:19

This is now correct

check Taskbar icon unread count
Total unread count on Taskbar icon says 57 and this matches the total combined unread count in just the Inboxes for all accounts.

BUG ISSUE:
One reason for the Taskbar icon unread count to be incorrect is because the 'folderCache.json' file contains the data of a hidden pop account Inbox which has unread mail.
When pop account was converted from a normal mail account using own folders to a deferred account using 'Global Inbox (Local Folders)', that set of data was not removed from the folderCache.json file.
Even after I had manually removed the emails from the hidden pop account Inbox and restarted Thunderbird, the folderCache.json file was not auto updated. I had to access again and either delete or rename the folderCache.json file to force a reload of data.
The point is this - the original pop account data is still in the 'folderCache.json' file, but no longer causes a problem because all Inbox data has been removed. If the count preference included all folders then this issue would extend to any hidden pop folder containing unread mail. But why is this pop account data still in folderCache.json file when pop account is a deferred account using a different account Inbox.

I have provided information on how to find out what the count is including.
I have provided information on how to fix the issue for this instance should people want to check and fix themselves. But, it is not intuitive nor user friendly to do.

Hopefully the information will be useful in sorting out this bug.

Comment 1

[Francesco]

Attachment: Deferred inbox, original inbox not hidden - TB 102.png

As I said in bug 1916107 comment 18, the main issue introduced with Supernova is that the (old) account's inbox is hidden. In 102 it wasn't hidden and the user could clearly see, access, read, move, delete, etc. the messages.

UPDATE: That statement is mostly wrong. Deferred inboxes are hidden, also in 102, only that they didn't get hidden immediately.

Comment 2

[Francesco]

I'm changing the subject slightly. Yes, the unread count is coming from the folder cache, but that's not a bug.

Comment 3

[Anje]

(In reply to Francesco from comment #1)

Created attachment 9422298 [details]
Deferred inbox, original inbox not hidden - TB 102.png

As I said in bug 1916107 comment 18, the main issue introduced with Supernova is that the (old) account's inbox is hidden. In 102 it wasn't hidden and the user could clearly see, access, read, move, delete, etc. the messages.

You seem to be talking about about something completely different. You seem to be not understanding this issue.

You can Never see any folders for any deferred pop account and never have been able to see them. In fact it you can see them you have messed up the process.
OR IF you really needed to see them - you would have to swap all the settings again and make sure everything points to the old original files and restart Thunderbird just to get the pop account visible again. Then remove unread mail and then go through the entire process again to reset as deferred.

1. When you create a normal pop mail account you can see that pop mail account in the Folder Pane and it has it's own set of folders.

The settings I'm talking about are here:

• In Account SEttings
• Select 'Server SEttings' for the POP account
• Click on 'Advanced' button
• For a standard normal pop mail account select 'Inbox for this account' and click on 'OK'

If you remove that pop account from Thunderbird, you cannot see it in Thunderbird, but in profile you will find the pop mail account folder containing all mbox files and emails.

SO - even if the pop account is 'removed' and not visible anywhere in Thunderbird, that pop account folder is still in profile.
Does that mean the old pop account is still displaying in the folderCache.json file ?
If yes, and it contains old unread mail then it's likely to mean the Total unread count will be wrong in the Taskbar icon.
This may be one of the causes of an incorrect total unread count.

2. When you initially create a pop account and set it up at that point as a deferred account to use Global Inbox (Local Folders)' you do not see that pop mail account name in the Folder Pane and it does not have it's own set of folders - that pop account is completely using 'Local Folders'.

The settings I'm talking about are here:

• In Account SEttings
• Select 'Server SEttings' for the POP account
• Click on 'Advanced' button
• For a deferred pop mail account select: 'Inbox for a different account' and choose 'Global Inbox(Local folders)
• Then also select 'Include this server when getting new mail' and click on 'OK'

If you remove that pop account from Thunderbird, you cannot see in Account Settings, however all the pop account emails are still visible because they are in 'Local Folders' mail account. But in profile you will find the pop mail account folder- it contains no emails because nothing was ever stored in that location.

So regardless of whether the folderCache.json still contains data for that pop account, it never had any emails stored in it, so nothing will be unread and therefore will not flag up or effect the total unread count in Taskbar icon.

3. When you initially create a normal standard pop mail account, you can see it in the Folder Pane with it's own set of folders ...BUT if after receiving some emails you choose to alter that account and now change the settings to become a deferred account and use Global Inbox (Local Folders)' , then you will lose the pop mail account from the Folder Pane, you will NOT see any of that pop account folders and from then onwards all incoming mail for that pop account will go to the Local Fodlers Inbox - which is visible.

I did number 3 above.

Now that means there is a problem because the folderCache.json file still contains the pop mail account data and it still displays whatever was left in that pop account own folders/files.

So if unread mail was still in the old - now hidden - files then they end up being included in the 'toal unread count' and it cause a problem.

This issue may have been around for ages.

Comment 4

[Anje]

(In reply to Francesco from comment #1)

Created attachment 9422298 [details]
Deferred inbox, original inbox not hidden - TB 102.png

As I said in bug 1916107 comment 18, the main issue introduced with Supernova is that the (old) account's inbox is hidden. In 102 it wasn't hidden and the user could clearly see, access, read, move, delete, etc. the messages.

After altering a normal pop account to be a deferred account, you have to restart Thunderbird and then the original pop account set of folders disappears.
If you still see the original pop mail account and it's own set of folders in Folder PAne then either you did not restart Thunderbird OR you have set it up incorrectly.

Correctly set up deferred account is Never visible in Folder Pane and never was visible - this is not the issue.

It has everything to do with the folderCache.json still containing data for deferred and possibly removed accounts AND where there are unread mails left in the Inbox or any other folder. In which case it will effect the total unread count.

Deferred and removed pop accounts -
They have pop account folders in profile which you would not want to auto remove because they may contain emails and may need to be reconnected to use.

In both instances, do they still have accounts and folders etc mentions in folderCache.json ?
Definitely yes if account is deferred.
I've Not tested if pop account is removed.
But if yes, then this causes a problem if unread messages are still present in files.
Why - because the means of counting unread uses the data in the folderCache.json file.

The fact that I had to perform some workaround to remove those unread messages from an old hidden file and had to remove or rename the folderCache.json file just to force a rebuild and update the data, is just the current means to work around the problem.

But the issue and bug is this:
Basically how to force the folderCache.json file to do one of two things:
Either not include data for accounts that are deferred or removed (the latter option still not tested).
OR if it must contain those mail accounts then set some sort of flag and alter code to say if account has X flag then do not include in any count.

Comment 5

[Francesco]

You can Never see any folders for any deferred pop account and never have been able to see them.

I didn't produce the screenshot in comment #1 with PhotoShop, but I've just realised: When you defer the inbox, it's not hidden immediately, that's when I took the screenshot. It's hidden after a restart of TB.

You are right, since the hidden inbox is only hidden but not removed, its count remains cached and contributes to the unread count.