Closed Bug 1103333 Opened 10 years ago Closed 8 years ago

The number of messages has reduced to zero on Mac. Gone with disabling of Spotlight

Categories

(Thunderbird :: Folder and Message Lists, defect)

Product:
Thunderbird
Component:
Folder and Message Lists
Version:
31 Branch
Platform:
All
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: lyon, Assigned: aceman)

References

Details

(Keywords: ux-userfeedback, ux-visual-hierarchy)

Attachments

(1 file, 1 obsolete file)

modified Extra folder columns extension
19.05 KB, application/octet-stream
Details
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36 Steps to reproduce: This happens on its own (like a background process). Actual results: The display showed 167k messages, now shows 100k messages. The messages are there, but reduced to zero Expected results: The number of messages stored in the folders should not reduce to zero. If I visit the folders, the messages are still there, and the count goes back up, again.
What is your server type for this folder? POP3 or IMAP4? Where do you check the number of messages?
This is a pop3 folder. All files are stored on the local file system.
the number of messages is shown in the total on the folder. The number goes down right before my eyes (i.e., counting down to zero as though there is a background process zeroing out the message number). But the messages are not being deleted. They are still there! Visiting the folder restores the count.
So you mean the counter in the window status bar? Saying "Unread:" and "Total:" ? May it be some addon doing this? Try Help->Restart with addons disabled. Also try rightclicking the folder, choose Properties and Retention policy. See if you do not have some automatic removal of messages enabled (even though you say it is not really removing anything).
I have no idea where the counter is in the window status bar. Nor do I know where to find the window status bar. I am looking at the number of messages in the folders. It is now up to over 128k, prior to this, it dropped to 90k It used to be 164k. Not sure what is going on with "the folder"...which folder? How can I determine how many folders there are? The root folder has a radio button on retention policy that says "Don't delete any messages" - Thanks! - Doug
There is no "total on the folder" displayed by default. Are you using the "Extra folder columns" extension? That is why I asked about the message count in the window status bar. But maybe it is called differenly in OS X. You can also check by rightclicking on a folder -> Properties->Number of messages . Also, the retention policy can be set independently on each folder.
When I right click on the root folder, I am told the number of messages is zero and the size on the disk is 0 kb. But when I look at the number contained in the folder it says: "128446" This is a little closer to accurate, but still not correct. - Doug
Here is a video documenting how to show the total number of messages, by default. Thanks! - DL http://screencast.com/t/p9Nm2VvQbVyV
Thanks, now I understand what you mean. Because you have not mentioned many facts in the bug description :) So: 1. You seem not to use any extension to show the message count. 2. The count in brackets is the number of Unread messages, not total. 3. The count displayed changes to 0 only when you expand the folder. That is a feature of TB that when a folder is collapsed (the subfolders not shown), it accumulates the count of messages from subfolders and shows them at the parent (a-z in your case). But that is only a display feature, no messages are moved or anything. The a-z folder itself contains no messages and is 0KB in size as shown in its Properties. So fortunately there is no serious problem in TB here, no dataloss. You are just confused that the number changes. Yes, we need an indication when the number shown is the count from the folder itself and when it is accumulated total from all its children (subfolders). I plan to change this display a bit in bug 464973. So after that one is fixed, we'll see if your problem still happens.
Assignee: nobody → acelists
Status: UNCONFIRMED → NEW
Depends on: 464973
Ever confirmed: true
Keywords: ux-userfeedback, ux-visual-hierarchy
OS: Mac OS X → All
Hardware: x86 → All
OK, I think what I am looking for is a what to display the number of messages (read or unread) in the aggregate folder hierarchy. Sorry for my confusion.
Yes, if you want to see the total number of messages (read+unread), then you can see it in the bottom right corner of the TB window at the "Total:" label (it is seen in the video). By default it can't be displayed in the hierarchy near the folder names. You can install the "Extra folder columns" extension for now. Or wait until bug 464973 is finished, which moves that extension into the default TB base.
OK, I think that if I could mark all messages as unread (for the present and all subfolders) that would solve my problem (showing the total number of messages). Extra folder does not appear to address the problem at all. I made a video documenting what I am seeing: http://screencast.com/t/diTH07Ls6NfT Thanks for your help! Regards, - Doug
Yes, I see. "Extra folder columns" adds a "Total" column, but it only shows the count per folder. It does not do the accumulation of subfolders you want. I'll upload a modified version of it here, that I am using, that has this feature.
Oh wow, that is so nice! Please let me know when the upload is complete, and I will give it a try. Thanks! - Doug
The unread message number keeps getting reduced,in the display. However, the messages are, in fact, to be left unread. You can see what is going on in the following video: http://screencast.com/t/ONaAypdset A visit to the fold reinstates the unread message number, but then later, it gets reduced to zero. I am not sure, but it seem like there is something going on in the background reducing the unread message count. Any idea what might be wrong? Thanks! - Doug
Try to install this extension. It will accumulate message count across subfolders. It will indicate with a star (*) whether the shown number is accumulated or if it is a count of message only in the folder itself.
Comment on attachment 8527814 [details] modified Extra folder columns extension Thanks very much for your efforts! Subfolders are not giving a tally to the parent folders. Also, they do not count the message number until or unless they are visited; http://screencast.com/t/WCrLvc9vSQ Shows what I am seeing. Is that how it is supposed to work? Thanks! - Doug
Go into Options->Advanced->General->Config editor... Create a new BOOLEAN preference with the name "extensions.extra-cols@jminta_gmail.com.sumSubfolders" (without the quotes) and give it a value of true.
Thank you so much for your efforts! Sorry to say, the same process that runs around zeroing out messages impacts the extra column add-on, as it did prior to the addition of the add-on. http://screencast.com/t/ourZYWzb25G Is a video documenting my battle with this background thread. My visit to reinstate the numbers is temporary, as the numbers get zero's afterwards. Am I doing something dumb? Thanks! - Doug
I do not understand now. I see the video. But which folder should I watch? I see you click various folders and the numbers do appear. I do not fully understand why the numbers aren't there immediatelly, but as far as I known once initialized, they stay there forever, also after TB restart. So when do they disappear? On which folder name can I see the effect?
I saw the total number reduced after running around and visiting a bunch of folders. It does not record when I do a JING recording. I am not sure why. Is there some why to count all the messages in all the subfolders recursively without having to visit them all? After watching the number fall back to 65k, I was able to visit folders and get it to over 87k. But it is still over 100k off. Thanks! - Doug
(In reply to :aceman from comment #17) > modified Extra folder columns extension > > Try to install this extension. It will accumulate message count across subfolders. > It will indicate with a star (*) whether the shown number is accumulated or if it is a count of message only in the folder itself. Oh, Extra folder columns has come back... If "Extra folder columns" is disabled, "accumulated Unread mail count" was always shown by Tb when folder is collapsed. If "Extra folder columns" is enabled, "correct/normal/expected-by-ordinal-user count of the folder" was always shown by Addon when folder is collapsed. :aceman, how can I see state of "indicated with a star (*) when the shown number is accumulated" with the "Extra folder columns" addon? It looks for me that "Extra folder columns" is powerful sword for us to kill pretty confusing "accumulated Unread count" :-) For Unified View. Unified Folder(Inbox, Drafts, Sent etc.) is a Virtual Folder(Search Folder). So, shown count is always "never accumulated number when collapsed". However, Unified Folder has actual folders under it, and they can be collapsed and expanded. It may produce further confusion by users. I think "count of Unified Folder" is better shown with a star (*), if "Extra folder columns" addon is used, to avoid user's confusion. :aceman, what do you think?
(1) For message count of a folder. Phenomenon of "mail count continuously decreases at a folder" or phenomenon of "mail count continuously increases at a folder" can be observed by "Run Filters on Folder". IIUC, "filter move" in this case is "for each mail{ if move is needed, move to a folder; } So, MsgDBHdr_Added event occurs at move target folder on each mail upon each move, and MsgDBHdr_Deleted event occurs at move source folder on each mail upon each move. Phenomenon of "mail count continuously decreases at a folder" can occur upon MsgPurge or JunkPure too. IIUC, "Msg/Junk Purge" is "for each mail{ if purge is needed, delete it without copy to Trash; } So, MsgDBHdr_Deleted event occurs at the folder upon each purge of mail. (2) Unread mail count can suddenly increase in current Tb. (2-1) Tb has problem of Bug 840418. "Read flag" is not always physically written to X-Mozilla-Status: header. (2-2) If local mail folder, "outdated msf condition" can occur on any folder. - Append mail data to file named XXX, before physical update of XXX.msf file, system crash. - phenomenon like Bug 624806 - symptom of Bug 261419/Bug 495760, Bug 232047 etc. (2-3）If "outdated msf condition" exists in locl mail folder, Rebuild-Index(Repair Folder) is invoked upon explicit folder open. So, if Bug 840418 happens and Rebuild-Index happens, some mails are marked Unread again, then Unread mail count suddenly increases. (3) If (2) happens on a local mail folder, and if MsgPurge or JunkPurge occurs, following can occur. (3-1) Mail is deleted by MsgPurge/JunkPurge. (3-2) For each deleted mail by MsgPurge/JunkPurge, MsgDBHdr_Deleted event occurs. (3-3) If deleted mail has Unread status, Unred count at Folder Pane is updated(decreased) upon deletion of each mail. Status of "deleted mail by MsgPurge/JunkPurge" is usually "Read". However, due to (1) & (2), status of many mails is changed back to Unread. If phenomenon like above, "Total mail count in a folder" should be decreased by MsgPurge after such phenomenon was observed. But nothing about "actual total mail count before/after problem" is provided by bug opener. I can say nothing about phenomenon what bug opener saw. But I can say (3) can occur if MsgPurge is enabled and if (1) & (2) occurs in user's environment. Even if phenomenon like above, I can't imagine reason why such phenomenon can be observed so frequently. Even if Tb has problems, "Rebuld-Index due to Outdted MSF Condition" is mandatory in above phenomenon. I can't imagine reason why "Rebuld-Index due to Outdted MSF Condition" so frequently happens in an user's environment.
(In reply to WADA from comment #23) > :aceman, how can I see state of "indicated with a star (*) when the shown > number is accumulated" with the "Extra folder columns" addon? > It looks for me that "Extra folder columns" is powerful sword for us to kill > pretty confusing "accumulated Unread count" :-) If you want to see the numbers marked with a star (accumulated), install the extension from the attachment here and set the pref "extensions.extra-cols@jminta_gmail.com.sumSubfolders" to true. In bug 464973 I try to put this same code into core TB. The accumulation (summarization) of subfolders is then selectable via the pref.
(In reply to :aceman from comment #25) > and set the pref "extensions.extra-cols@jminta_gmail.com.sumSubfolders" to true. It worked as expected in All Folder View. If Unified View, it looks to work as extensions.extra-cols@jminta_gmail.com.sumSubfolders=false. I believe it's proper decision. I believe there is no need of "accumulated mail count in Unified View". It merely produces user's confusions. Thanks for improvement. Useful "accumulated mail count" now can be used for "finding new messages which were moved to various folders by filter" without any confusion.
(In reply to lyon from comment #22) > I saw the total number reduced after running around and visiting a bunch of folders. Oh sorry. I missed this commentt. Do you enable MsgPurge by account's DiskSpace setting or "Folder Properties/Retension Policy", or JunkPurge by Junk Setting?
I do not delete any messages, yet the number of messages keeps decreasing as a function of time. A video documenting the magic background removal of messages is shown here: http://screencast.com/t/YemKCFepMbAb Right before your eyes, you can see the ever decreasing message count. What could be doing this?
(In reply to lyon from comment #28 > A video documenting the magic background removal of messages is shown here "accumulated Unread mail count" of a-z/b/bill is being decreased when a-z/b/bill is collapsed. However, when a-z/b/bill is expanded, Unread count of any folder under a-z/b/bill looks unchanged... a-z, b, bill is ordinal pop3 mail folder which you manually created via Tb's UI? Or folder generated by addon for mail folder housekeeping? Because Unread count only is shown by standard Tb, it's impossible to know total mail count by Folder Pane data only. Because we are processing "problem in mail count display", please use attached "Extra folder columns" with extensions.extra-cols@jminta_gmail.com.sumSubfolders=true for ease of problem analysis.
FYI. If there is "hidden folder under a-z/b/bill by add on"(mail folder surely exists in Tb, but it's not shown at folder pane by addon), and if Unread mail count of the not-shown-at-folder-pane folder is decreased by someone, phenomenon of the video can be explianed.
Hi Wada, Please see the ls -al output, below. As you can see, there is no hidden folder until bill in the file system. Thanks! - Doug In: /Users/lyon/current/tbird/lyon/tbird.sbd/a-z.sbd/b.sbd/bill.sbd ls -al | more total 22712 drwxr-xr-x 92 lyon staff 3128 Nov 26 05:14 ./ drwxr-xr-x 56 lyon staff 1904 Nov 26 05:14 ../ -rw-r--r--@ 1 lyon staff 1107 Oct 18 09:26 Bill_li drwxr-xr-x@ 3 lyon staff 102 Nov 12 08:55 Bill_li.mozmsgs/ -rw-r--r--@ 1 lyon staff 2923 Nov 26 05:14 Bill_li.msf -rw-r--r--@ 1 lyon staff 7731841 Nov 20 10:17 bill_Taylor drwxr-xr-x@ 1693 lyon staff 57562 Nov 20 10:17 bill_Taylor.mozmsgs/ -rw-r--r--@ 1 lyon staff 943006 Nov 20 10:17 bill_Taylor.msf -rw-r--r--@ 1 lyon staff 2765 Oct 18 09:26 bill_abbot drwxr-xr-x@ 3 lyon staff 102 Nov 12 08:40 bill_abbot.mozmsgs/ -rw-r--r--@ 1 lyon staff 2877 Nov 26 05:14 bill_abbot.msf -rw-r--r--@ 1 lyon staff 21650 Oct 18 09:26 bill_alpert drwxr-xr-x@ 8 lyon staff 272 Nov 12 08:40 bill_alpert.mozmsgs/ -rw-r--r--@ 1 lyon staff 5172 Nov 26 05:14 bill_alpert.msf -rw-r--r--@ 1 lyon staff 1446 Oct 18 09:26 bill_anderson drwxr-xr-x@ 3 lyon staff 102 Nov 12 08:40 bill_anderson.mozmsgs/ -rw-r--r--@ 1 lyon staff 2951 Nov 26 05:14 bill_anderson.msf -rw-r--r--@ 1 lyon staff 5652 Oct 18 09:26 bill_baxter drwxr-xr-x@ 3 lyon staff 102 Nov 12 08:40 bill_baxter.mozmsgs/ -rw-r--r--@ 1 lyon staff 2966 Nov 26 05:14 bill_baxter.msf -rw-r--r--@ 1 lyon staff 5731 Oct 18 09:26 bill_buckles drwxr-xr-x@ 4 lyon staff 136 Nov 12 08:40 bill_buckles.mozmsgs/ -rw-r--r--@ 1 lyon staff 3358 Nov 26 05:14 bill_buckles.msf -rw-r--r--@ 1 lyon staff 17074 Oct 18 09:26 bill_coote drwxr-xr-x@ 7 lyon staff 238 Nov 12 08:41 bill_coote.mozmsgs/ -rw-r--r--@ 1 lyon staff 4757 Nov 26 05:14 bill_coote.msf -rw-r--r--@ 1 lyon staff 86695 Oct 18 09:26 bill_dieckmann drwxr-xr-x@ 6 lyon staff 204 Nov 12 08:42 bill_dieckmann.mozmsgs/ -rw-r--r--@ 1 lyon staff 4656 Nov 26 05:14 bill_dieckmann.msf -rw-r--r--@ 1 lyon staff 25300 Oct 18 09:26 bill_dornfeld drwxr-xr-x@ 7 lyon staff 238 Nov 12 08:43 bill_dornfeld.mozmsgs/ -rw-r--r--@ 1 lyon staff 6521 Nov 26 05:14 bill_dornfeld.msf -rw-r--r--@ 1 lyon staff 25117 Oct 18 09:26 bill_dryden drwxr-xr-x@ 7 lyon staff 238 Nov 12 08:44 bill_dryden.mozmsgs/ -rw-r--r--@ 1 lyon staff 5139 Nov 26 05:14 bill_dryden.msf -rw-r--r--@ 1 lyon staff 19670 Oct 18 09:26 bill_erickson drwxr-xr-x@ 6 lyon staff 204 Nov 12 08:45 bill_erickson.mozmsgs/ -rw-r--r--@ 1 lyon staff 4729 Nov 26 05:14 bill_erickson.msf -rw-r--r--@ 1 lyon staff 4731 Oct 18 09:26 bill_gorden drwxr-xr-x@ 3 lyon staff 102 Nov 12 08:45 bill_gorden.mozmsgs/ -rw-r--r--@ 1 lyon staff 2969 Nov 26 05:14 bill_gorden.msf -rw-r--r--@ 1 lyon staff 23817 Oct 18 09:26 bill_graham drwxr-xr-x@ 4 lyon staff 136 Nov 12 08:45 bill_graham.mozmsgs/ -rw-r--r--@ 1 lyon staff 3670 Nov 26 05:14 bill_graham.msf -rw-r--r--@ 1 lyon staff 14862 Oct 18 09:26 bill_greenspan drwxr-xr-x@ 5 lyon staff 170 Nov 12 08:45 bill_greenspan.mozmsgs/ -rw-r--r--@ 1 lyon staff 4052 Nov 26 05:14 bill_greenspan.msf -rw-r--r--@ 1 lyon staff 451281 Oct 18 09:26 bill_guelakis drwxr-xr-x@ 122 lyon staff 4148 Nov 12 08:55 bill_guelakis.mozmsgs/ -rw-r--r--@ 1 lyon staff 60360 Nov 26 05:14 bill_guelakis.msf -rw-r--r--@ 1 lyon staff 5237 Oct 18 09:26 bill_herrity drwxr-xr-x@ 4 lyon staff 136 Nov 12 08:55 bill_herrity.mozmsgs/ -rw-r--r--@ 1 lyon staff 3305 Nov 26 05:14 bill_herrity.msf -rw-r--r--@ 1 lyon staff 3293 Oct 18 09:26 bill_matthews drwxr-xr-x@ 3 lyon staff 102 Nov 12 08:55 bill_matthews.mozmsgs/ -rw-r--r--@ 1 lyon staff 2991 Nov 26 05:14 bill_matthews.msf -rw-r--r--@ 1 lyon staff 8896 Oct 18 09:26 bill_mulligan drwxr-xr-x@ 4 lyon staff 136 Nov 12 08:55 bill_mulligan.mozmsgs/ -rw-r--r--@ 1 lyon staff 3521 Nov 26 05:14 bill_mulligan.msf -rw-r--r--@ 1 lyon staff 8837 Oct 18 09:26 bill_perlman drwxr-xr-x@ 5 lyon staff 170 Nov 12 08:55 bill_perlman.mozmsgs/ -rw-r--r--@ 1 lyon staff 3811 Nov 26 05:14 bill_perlman.msf -rw-r--r--@ 1 lyon staff 1829 Oct 18 09:26 bill_pugh drwxr-xr-x@ 3 lyon staff 102 Nov 12 08:55 bill_pugh.mozmsgs/ -rw-r--r--@ 1 lyon staff 2827 Nov 26 05:14 bill_pugh.msf -rw-r--r--@ 1 lyon staff 2844 Oct 18 09:26 bill_romatzick drwxr-xr-x@ 3 lyon staff 102 Nov 12 08:55 bill_romatzick.mozmsgs/ -rw-r--r--@ 1 lyon staff 2862 Nov 26 05:14 bill_romatzick.msf -rw-------@ 1 lyon staff 1617930 Nov 25 06:49 bill_ryan drwxr-xr-x@ 358 lyon staff 12172 Nov 25 06:49 bill_ryan.mozmsgs/ -rw-r--r--@ 1 lyon staff 166724 Nov 26 05:14 bill_ryan.msf -rw-r--r--@ 1 lyon staff 1734 Oct 18 09:26 bill_sapone drwxr-xr-x@ 3 lyon staff 102 Nov 12 09:05 bill_sapone.mozmsgs/ -rw-r--r--@ 1 lyon staff 3381 Nov 26 05:14 bill_sapone.msf -rw-r--r--@ 1 lyon staff 6122 Oct 18 09:26 bill_schonberg drwxr-xr-x@ 3 lyon staff 102 Nov 12 09:05 bill_schonberg.mozmsgs/ -rw-r--r--@ 1 lyon staff 3549 Nov 12 09:05 bill_schonberg.msf -rw-r--r--@ 1 lyon staff 40848 Oct 18 09:26 bill_stewart drwxr-xr-x@ 12 lyon staff 408 Nov 12 09:06 bill_stewart.mozmsgs/ -rw-r--r--@ 1 lyon staff 7676 Nov 12 09:06 bill_stewart.msf -rw-r--r--@ 1 lyon staff 25251 Oct 18 09:27 bill_tripodi drwxr-xr-x@ 10 lyon staff 340 Nov 12 09:45 bill_tripodi.mozmsgs/ -rw-r--r--@ 1 lyon staff 7458 Nov 12 09:46 bill_tripodi.msf -rw-r--r--@ 1 lyon staff 34086 Oct 18 09:27 bill_wessman drwxr-xr-x@ 5 lyon staff 170 Nov 12 09:45 bill_wessman.mozmsgs/ -rw-r--r--@ 1 lyon staff 4506 Nov 12 09:45 bill_wessman.msf -rw-r--r--@ 1 lyon staff 3066 Oct 18 09:27 bill_yario drwxr-xr-x@ 3 lyon staff 102 Nov 12 09:45 bill_yario.mozmsgs/ -rw-r--r--@ 1 lyon staff 3262 Nov 12 09:45 bill_yario.msf -rw-r--r--@ 1 lyon staff 30234 Oct 18 09:27 billy_weitzer drwxr-xr-x@ 7 lyon staff 238 Nov 12 09:45 billy_weitzer.mozmsgs/ -rw-r--r--@ 1 lyon staff 5083 Nov 12 09:45 billy_weitzer.msf
It appears that the *.mozmsgs folders are created to integrate Thunderbird 3 with Search. These folders and files are created in a manner that allows Windows/spotlite to index them.
(In reply to lyon from comment #32) > It appears that the *.mozmsgs folders are created to integrate Thunderbird 3 > with Search. These folders and files are created in a manner that allows > Windows/spotlite to index them. I don't know what those special folders are, but they are suspicious (not part of default TB operations). I asked you to disable all addons temporarily in comment 5. Please do it and report the results.
Hi All, A video documenting the total message count in the extra column appears here: http://screencast.com/t/IU68GFsu Is there a way to rebuild the index for the entire account? Xpunge (the extension) did not work. Thanks! - Doug
All extensions aside from the extra column add-on are disabled. The message count total is still over by over 100k. - Doug
(In reply to WADA from comment #26) > (In reply to :aceman from comment #25) > > and set the pref "extensions.extra-cols@jminta_gmail.com.sumSubfolders" to true. > > It worked as expected in All Folder View. > If Unified View, it looks to work as > extensions.extra-cols@jminta_gmail.com.sumSubfolders=false. I believe it's > proper decision. I believe there is no need of "accumulated mail count in > Unified View". It merely produces user's confusions. > Thanks for improvement. Useful "accumulated mail count" now can be used for > "finding new messages which were moved to various folders by filter" without > any confusion. Yes, there the accumulation is intentionally done only in All folders view, because the used methods on nsIMsgFolder operate on real subfolders of the folder, disregarding the active folder view. I could skip those native methods and iterate over the subfolders using the information from the folder view. But I have not yet chosen to do that. You could propose that in bug 464973, but it seems you also like it as it currently is, without changes needed.
(In reply to lyon from comment #28) > I do not delete any messages, yet the number of messages keeps decreasing as > a function of time. > A video documenting the magic background removal of messages is shown here: > http://screencast.com/t/YemKCFepMbAb > Right before your eyes, you can see the ever decreasing message count. > > What could be doing this? Now this is a video may move us forward. In this single video I see the number decreasing "right before my eyes". Remember, the number in brackets is the number of UNREAD messages. Not the total number of messages. Please re-record the video with the addon enabled and the Total and Size columns enabled so we can see if the total changes. I see 2 possibilities of the cause here: 1. the initial number of unread messages (shown in the video) is correct but something inside TB (maybe addon or retention policy) is "reading" some of the messages (marking them read). So that the total number is decreasing. 2. the initial number of unread messages (in the video) is wrong. The msf files of the folders are outdated so TB has wrong data (as WADA says in comment 24). It is automatically recreating the files and statistics in the folders about the real number of unread messages and is updating the display "before your eyes". This would also be supported by the fact you need to click on the folders for the number in brackets to show up. After the click TB immediately updates that folder with priority. Also try to see if your panacea.dat file is properly writable by TB and has correctly-looking time of modification after TB exits.
OS: All → Mac OS X
You wrote: "Also try to see if your panacea.dat file is properly writable by TB and has correctly-looking time of modification after TB exits." Here is panacea.dat: ls -al panacea.dat -rw-rw-r--@ 1 lyon staff 2309251 Nov 26 08:42 panacea.dat mini{lyon}70: whoami lyon Does that look OK to you? Thanks! - Doug
You write: "the initial number of unread messages (in the video) is wrong. The msf files of the folders are outdated so TB has wrong data (as WADA says in comment 24). It is automatically recreating the files and statistics in the folders about the real number of unread messages and is updating the display "before your eyes". This would also be supported by the fact you need to click on the folders for the number in brackets to show up. After the click TB immediately updates that folder with priority." I have deleted a .msf file, and restarted TB. However, the number of messages did not update. Does this relate to the theory that the msf files are outdated? If not, how can we force their rebuilding? Thanks! - Doug
You write: the initial number of unread messages (shown in the video) is correct but something inside TB (maybe addon or retention policy) is "reading" some of the messages (marking them read). So that the total number is decreasing. ------- That is almost right. Something inside of TB is marking them read. The totals are also decreasing. What could be doing this? - Doug
In the video I see strange element on the bottom of the TB window, reading:"(G)o (S)ave (C)opy ...". What is that? Doesn't look to me as standard TB feature. It looks like an addon. Can you attach a screenshot from the account manager showing that all extensions are really disabled?
Here is a video showing not only the unread message number being incorrect, but also the total number of messages being incorrect. http://screencast.com/t/rGo0XJYKPMnU A visit to the folder enables restoration of the number of messages. What does visiting the folder do in order to update the message number in TB? Thanks! - Doug
Here is a video showing extensions disabled and the number of messages as incorrect. http://screencast.com/t/oZnCLwpkN3Jg Thanks! - Doug
Spotlight; On a mac: preferences:advanced; System Integration "Allow Spotlight to search messages" is checked. Could this enable the spotlight file location program to interact with TB in a non-deterministic manner? I have turned it off, to see...
(In reply to lyon from comment #44) > Spotlight; > On a mac: preferences:advanced; System Integration > "Allow Spotlight to search messages" is checked. > > Could this enable the spotlight file location program to interact with > TB in a non-deterministic manner? I would hope Spotlight does not alter TB's data in this way. But it could interfere with the msf files somehow and force them to be rebuilt, causing the effect you see. Yes, you can try disabling it so we are sure.
(In reply to lyon from comment #42) > Here is a video showing not only the unread message number being incorrect, > but also the total number of messages being incorrect. > http://screencast.com/t/rGo0XJYKPMnU > > A visit to the folder enables restoration of the number of messages. > What does visiting the folder do in order to update the message number in TB? It may rebuild the msf index file and determine new number of unread messages. However, it should tell that on the status bar too (bottom of the TB window). But in this video we do not see the decreasing of the number of messages. I'd like to see that when number of unread msgs changes, if total number changes too. That can't be decided in this video.
(In reply to lyon from comment #43) > Here is a video showing extensions disabled and > the number of messages as incorrect. Yes, so where does the "(G)o (S)ave (C)opy ..." message come from? Josiah, have you seen anything like that on OS X? Please see the TB status bar in video http://screencast.com/t/YemKCFepMbAb .
Flags: needinfo?(josiah)
The nostalgy extension was turned on...but I turned it off for making the video...and restarted TB. - D
Hi All, It would appear, for now, that the spotlight interaction with Thunderbird was the culprit. To repeat on a mac: preferences:advanced; System Integration "Allow Spotlight to search messages" is checked. Spotlight wakes up and files are slowly marked as "read" Thus, spotlight appears to be interacting with TB in a non-deterministic manner. My test shows that the number of messages is now back up to over 120k..and appears stable, for now. Shall we call this a bug?
(In reply to lyon from comment #49) > Shall we call this a bug? I think problem is : "accumulated Unread mail count" feature increments/decrements accumulated Unread mail count of folder XX or parent of XX when onReadChanged/onHdrFlagsChanged event is notified for MsgDBHdr of a mail under XX.mozmsgs directory. "Extra folder columns" addon also increments/decrements accumulated Total/Size upon onHdrAdded/Deleted, OnItemAdded/Deleted etc. at "psuedo/faked mail folder under XX.mozmsgs directory". Because "msgFolder for the mail under XX.mozmsgs directory" is not available via Enumerator named msgFolder.subFolders, "actual Unread/Total/Size of each actual mail folder" is not affected. I think "update of accumulated Unread mail count" should be executed only when event target folder is actual mail folder which is actually available via. msgFolder_of_parent_folder.subFolders(==message folder which is shown at folder selection list in Account Manager, Message Filter etc.).
Phenomenon was; When a-z/b/bill is collapsed, accumulated Unread count is updated. When a-z/b/bill is expanded, no "Unread count iupdate" is seen. "Extra folder column" simply calls "let unread = this._folder.getNumUnread(true);" when folder is collapsed && extensions....sumSubfolders==true. > http://mxr.mozilla.org/comm-central/source/mailnews/base/util/nsMsgDBFolder.cpp#4209 > NS_IMETHODIMP nsMsgDBFolder::GetNumUnread(bool deep, int32_t *numUnread) nsMsgDBFolder::GetNumUnread(true) simply accumulates numUnread of all subfolders reccursively. "MsgDBHdrAdded/Deleted event at a-z/b/bill/bill_###.mozmsgs/ directory" is counted as "Unread count of a-z/b/bill/bill_###"? If so, large unread count of a-z/b/bill/bill_### may be explained. Or URL format of "pseudo/faked mail folder for .mozmsgs" is "a-z/b/bill/bill_###/.mozmsgs/..." like one, and it's returned by msgFolder.subFolders but is hidden at folder pane/folder selection list by Thunderbird? If so, phenomenon is as follows? nsMsgDBFolder::GetNumUnread(true) returns accumulated count of "a-z/b/bill/bill_###" as designed. Because folder of "a-z/b/bill/bill_###" is not collapsed(no subfolder at folder pane), "Extra folder columns" can't show "*", and can't show sub folders of a-z/b/bill/bill_###.
URL format of "pseudo/faked mail folder for .mozmsgs" is "a-z/b/bill/bill_###.mozmsgs" like one, and it's returned by msgFolder.subFolders but is hidden at folder pane/folder selection list by Thunderbird? If so, phenomenon is perhaps as follows. If a-z/b/bill is collapsed, unread count of hidden "a-z/b/bill/bill_###.mozmsgs" is counted because of "accumulated count". If a-z/b/bill is expanded, "a-z/b/bill/bill_###.mozmsgs" is never shown at folder pane because it's hidden by Thunderbird. There is no need of "File" in MsgStore(bill_### or bill_###.mozmsgs) itself. Tb shows AAA/AAA.msf as folder named AAA, even when "AAA" is directory. In maildirstore, MsgStore corresponds to folder named AAA is directory named AAA. And because there is no need to access a-z/b/bill/bill_###.mozmsgs directory as message folder in Tb, no need of .msf file for it. In-memory-only-MsgDB is sufficient for a-z/b/bill/bill_###.mozmsgs. I think this can explain phenomenon.
(In reply to :aceman from comment #47) > (In reply to lyon from comment #43) > > Here is a video showing extensions disabled and > > the number of messages as incorrect. > Yes, so where does the "(G)o (S)ave (C)opy ..." message come from? > > Josiah, have you seen anything like that on OS X? Please see the TB status > bar in video http://screencast.com/t/YemKCFepMbAb . I've never experienced anything like this unfortunately, and that "(G)o (S)ave (C)opy ..." message is unknown to me. Although, I don't use POP, so perhaps that's related.
Flags: needinfo?(josiah)
In winserch/spotlight support, MsgDB like one doesn't look used. Upon folder re-discovery, ignore .../xxx.mozmsgs, .../xxx.sbd, .../xxx.msf, because these are reserved. If .mozmsgs directory for mail folder is not found, create it from "file path of relevant mail folder" + ".mozmsgs". When mail is added to a mail folder, write mail data to a file under the "file path of relevant mail folder" + ".mozmsgs" directory. When mail is removed from a mail folder, remove file under the "file path of relevant mail folder" + ".mozmsgs" directory. If .mozmsgs directory is used, when onHdrDeleted/onHdrAdded/onReadChanged/onHdrPropertyChanged event occurs at a mail folder, update of count is executed twice by different event listners in different components? Or such event is invoked again when indexing by winserch/spotlight support is invoked?
"winserch/spotlight support, MsgDB like one doesn't look used." I think that is correct. Each file is a partial copy (up to 20kb) of a message in the mail folder, stored in a .mozmsgs subdirectory. Looking at the source for the Spotlight component, nsSpotlightIntegration.js https://code.google.com/p/reply-manager/source/browse/mail/components/search/nsSpotlightIntegration.js?spec=svnb5ff8a99506dc25406c9e46adbee71b18e50cebf&r=b5ff8a99506dc25406c9e46adbee71b18e50cebf It is used by the optional Spotlight Integration component under OSX and thus only an issue on Mac builds.
File for spotlight is treated as msgFolder? ItemAdded event is fired when file for spotlight is added? Added file : .../a-z.sbd/b.sbd/bill.sbd/bill_###.mozmsg/abc...xyz If this is treated as subfolder of a-z/b/bill, and if ItemAdded event is invoked, and if item passed to event listner returns true to "if(Item instanceof Components.interfaces.nsIMsgFolder)", it's "subfolder is added to a-z/b/bill" for all OnIemAdded evet listner. Because "accumilated number" feature merely counts total of Unread mail count of all subfolders recursivelly via. msgFolder.subFolders enumertor, msgFolder_for_a-z_b_bill.subFolders, msgFolder_for_a-z_b_bill.subFolders[i],subFolders, ..., should return msgFolder objects, and each msgFolder for spotlight file should have Unread count=1 initially... lyon@docjava.com(bug opener), can you see your problem consistently? (I think phenomenon can be observed by "copy many mails to empty folder") If so, "checking rerurned object by msgFolder.subFolders while problem is occuring" may help problem analysis. I have tool for it.
Just a note to let you know, it has been 3 days since I disabled the ability of spotlight to search messages in TB. During that time, the number of messages has been a monotonically increasing function of time (as it should be). Now up over 120k Not sure where the 160k+ messages went to (hope they are still around, somewhere!). Thus, the question: "can you see your problem consistently?" No, the problem appears to be corrected with spotlight disabled. This may be one of the cases when correlation is linked to causation. Aceman writes: "I would hope Spotlight does not alter TB's data in this way. But it could interfere with the msf files somehow and force them to be rebuilt, causing the effect you see. Yes, you can try disabling it so we are sure." Aceman: Good call! Thanks! - Doug
Can you shed any light on this bug, or should I say"spotlight" :)
Flags: needinfo?(Nomis101)
(In reply to Joe Sabash [:JoeS1] from comment #58) > Can you shed any light on this bug, or should I say"spotlight" :) I also use POP and I have enabled spotlight. But I have never seen something like this. I don't have that many unread messages in my inbox, maybe thats the reason. I will keep an eye on this in the next time...
Flags: needinfo?(Nomis101)
Accumulted Unread number feature uses GetNumUnread(true). "Extra folder columns" uses GetNumUnread(true) and GetTotalMessages(true), and uses same logic for msgFolder.sizeOnDisk for Size column. > NS_IMETHODIMP nsMsgDBFolder::GetNumUnread(bool deep, int32_t *numUnread) > http://mxr.mozilla.org/comm-central/source/mailnews/base/util/nsMsgDBFolder.cpp#4209 > NS_IMETHODIMP nsMsgDBFolder::GetTotalMessages(bool deep, int32_t *totalMessages) > http://mxr.mozilla.org/comm-central/source/mailnews/base/util/nsMsgDBFolder.cpp#4236 In both routine, counted subfolder is "msgFolder object which is not Virtual Folder". if (!(folderFlags & nsMsgFolderFlags::Virtual)) { folder->GetNumUnread(deep, &num); total += num; } > nsMsgFolderFlags > http://mxr.mozilla.org/comm-central/source/mailnews/base/public/nsMsgFolderFlags.idl#15 Condition maybe should be one like next. (Directory should be excluded) if ( ( ( (folderFlags & nsMsgFolderFlags::Mail) && !(folderFlags & nsMsgFolderFlags::Virtual) ) || ( folderFlags & nsMsgFolderFlags::Newsgroup ) ) && ( !(folderFlags & nsMsgFolderFlags::Directory) ) )
"msgFolder with Directory flag" is excluded from TotalSize counting by addon. while (subFolders.hasMoreElements()) { let subFolder = subFolders.getNext() .QueryInterface(Components.interfaces.nsIMsgFolder); // For Bug 1103333 if( subFolder.flags & nsMsgFolderFlags["Directory"] ) continue; if ( subFolder.flags & nsMsgFolderFlags["Mail"] ) { } else if( subFolder.flags & nsMsgFolderFlags["Newsgroup"] ) { } else continue; if( subFolder.flags & nsMsgFolderFlags["Virtual"] ) continue; // End of For Bug 1103333 let subSize = getFolderSize(subFolder); if (subSize == kUnknownSize) return subSize; else size += subSize; } }
(In reply to lyon from comment #57) > Thus, the question: "can you see your problem consistently?" > No, the problem appears to be corrected with spotlight disabled. Can you see your problem consistently with spotlight ENABLED? Do you see "Size column value change" with modified addon(attached to comment $61) when incrementing/decrimenting is being observed at Unread/Total column with spotlight enabled?
(In reply to lyon from comment #42) > Here is a video showing not only the unread message number being incorrect, > but also the total number of messages being incorrect. > http://screencast.com/t/rGo0XJYKPMnU I viewed the video again. 1. When parent is expanded, child folders has no Unread count and no Total count. 2. When a child folder is clicked, Unread count and Total count is shown. 3. At status bar, "Done" is shown. I could see this phenomenon by; 1. At directory named Parent.sbd for folder named Parent, when Folder1 to FokderN doesn't exist under Parent.sbd, Copy file named FolderA(some are Unread mail) under Parent.sbd to Folder1, Folder2, ..., FolderZ. Never copy FolderA.msf, never create FolderN.msf. 2. Restart Tb. Parent is expanded status. If not expanded, expand. 3. Folder1 to FolderZ : Unread=space, Total=space, Size=file size of file named FolderN. 4. Click FolderN => Unread and Total is shown, Size is unchanged. 5. "Done" is shown at status bar === Rebuild-Index was invoked, so mail count is shown. "Automatic change of Unread/Total of collapsed folder" can be easily observed by : 1. Create Search Folder under a pop3 account(calll VF-01). Search Target = FolderX and FolderY and FolderZ of Local Folders in which many mails are held and many mails are Unread. Condition is "All messages". 2. Select other than VF-1, FolderX, FolderY, FolderZ at folder pane, and terminate Tb. Delete FolderX.msf, FolderY.msf, FolderZ.msf. Delete panacea.dat, folderTree.json. 3. Restart Tb. 4. Collapse "Local Folders" account. 5. Click VF-1 of pop3 account. Rebuild-Index is invoked for FolderX/Y/Z by open of VF-1 because "Search Folder open" explicitly opens target folders. , So, Unread count and Total count of collapsed "Local Folders" is automatically updaated. How did you create message folders under a-z folder? Created via Tb's UI? Or copied file for mail folder in Tb from somewhere?
FYI. Because nsShouldIgnoreFile returns true if file name ends with ".mozmsgs" as done on ".sbd" and ".msf", FolderName.mozmsgs is always excluded from directory scan list. So, as I thought before, FolderName.mozmsgs won't be returned to msFolder.subFolders and GetSubFolders(). > http://mxr.mozilla.org/comm-central/source/mailnews/local/src/nsMsgLocalStoreUtils.cpp#27 > nsMsgLocalStoreUtils::nsShouldIgnoreFile
Attachment #8530577 - Attachment is obsolete: true
Hi All, As I mentioned, the problem is fixed, now that spotlight indexing is turned off. Thanks! - DL
I should also mention that the problem has not recurred since spotlight has been disabled. I would like to avoid reenabling spotlight until there is a potential fix in place. Thanks! - DL
(In reply to lyon from comment #66) > I should also mention that the problem has not recurred since spotlight has been disabled. > I would like to avoid reenabling spotlight until there is a potential fix in place. What is evidence of "spotlight is cause"? At least http://screencast.com/t/rGo0XJYKPMnU is evidence that Rebuild-Index happened when you clicked folder. Phenomenon is "because xxx.msf of many folders was broken by you, Rebuild-Index happened when you clicked folder", isn't it?
aceman, feel free to reopen if you think there is something that needs fixing.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
Summary: The number of messages has reduced to zero! → The number of messages has reduced to zero on Mac. Gone with disabling of Spotlight
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: