Bugzilla

Please see bug 881829
This is a bit of a "me too" post but it could be a different bug altogether, or shed light on the one I linked. In a nutshell, the author said three years ago the problem had resolved itself. For me, it is re-emerging, and encryption does not seem to be the culprit.

I use Thunderbird 60.6.1 on a W8.1 laptop and a W10 desktop, but it occurred with other versions of TB and with W7 too.

Incidentally, I am also a consultant who has to be extremely careful about client data. TB is installed on both machines, but the profile and mail files are stored on an encrypted file on a USB key. This means I do not have to synchronise TB between two machines; I do not have to use IMAP (some of my e-mail hosts---notably TalkTalk---have been hacked before, so now I exclusively use POP and delete all server e-mails); and if I lose my USB key the encryption protects its content. For encryption, I use a file container initially with TrueCrypt, now with VeraCrypt. I have about 8GB of mail. 

A few years (~6 years) and versions of TB and Windows ago, I noticed that some filters failed to work. It turned out the MSF file of the folder where the filter was going to move the message was missing. Clicking on the offending folder was not always the solution: I would get an hourglass and the message list would not show at all (I believe this exonerates indexing and gloda, even though gloda was wrong as a result). 

TB was very temperamental as to which folder I had to open before opening the faulty ones. I never discovered what the rule was, but proximity seemed to play a part. Usually, the faulty folders were at the bottom of the tree, but not always; past activity, particularly filters, seems to play a part too. Once I found the right sequence of folders to open and TB accepted to rebuild the MSF file, the filter could be run. But not for long. At the next session, or sometimes even within the same session, some MSF files would disappear again. Some of those files were the same as before, some where different; randomness seemed to be inescapable. There were also several error messages about missing address books.

About two? years ago, I solved the problem by deleting all MSF files, gloda and possibly other files (panacea.dat possibly). This also fixed the address-books problems. Unfortunately, I did not write down exactly what I did... No comment.

This week, the problem resurfaced. But this time, there was no encryption. I had made a copy of the entire TB profile and mail files on my C: drive and pointed TB to use that. Promptly, TB went haywire and could not run searches properly. Deleting all MSF files led to only some of them being rebuilt. Deleting gloda led some folders to be indexed but not all of them (at least that is what the Activity Manager told me). The MSF files could forcibly be rebuilt by clicking on the folder, but the fix was ephemeral: as soon as I clicked away from a problem folder (on a saved search), I could see the corresponding MSF file disappear from Windows Explorer, and the mail count disappear from the folder pane.

Is the saved search part of the problem? It looks at the entire folder tree (with a few exceptions) for messages within a certain year; it rarely gets the same results from one session to the next; and it always counts fewer than there really are. If I break the search down into two searches, one on each half of the folder tree, both searches work without problem, the counts are correct, and the NSF files remain in place. So volume is also a factor. However, this does not explain why some MSF files would disappear from one session to the next, or within one session when I do not use the search facility. Is there a process similar to a search taking place when TB starts or receives mail, which accidentally vaporises MSF files?

By contrast, the profile and mail files on the encrypted USB key still seem fine. All the more puzzling that since making the backup on my C: drive, I have increased the number of folders (from the high 500s to around 800), though not the number of messages, by archiving some mail for the first time (hence the temporary backup on my hard drive and the need for searches to make sure I could reconcile the message counts).

All this is very woolly, random and thus difficult to fix. I wish I had some advice and guidance to bring a bit of science and usefulness to my observations so that the disappearing MSF problem can be addressed.