msf files disappear/get emptied out when stored in an Windows EFS directory, also disappearing MSF files on Mac.
Categories
(MailNews Core :: Database, defect)
Tracking
(Not tracked)
People
(Reporter: davofanmail, Unassigned)
References
Details
(Keywords: dataloss)
Updated•8 years ago
|
Comment 1•8 years ago
|
||
Reporter | ||
Comment 2•8 years ago
|
||
Comment 3•8 years ago
|
||
Comment 4•8 years ago
|
||
Comment 5•8 years ago
|
||
Comment 6•8 years ago
|
||
Comment 7•8 years ago
|
||
Comment 8•8 years ago
|
||
Comment 9•8 years ago
|
||
Comment 10•8 years ago
|
||
Comment 11•8 years ago
|
||
Comment 12•8 years ago
|
||
Comment 13•8 years ago
|
||
Reporter | ||
Comment 14•8 years ago
|
||
Comment 16•8 years ago
|
||
Reporter | ||
Comment 17•8 years ago
|
||
Comment 18•8 years ago
|
||
Comment 19•8 years ago
|
||
Comment 20•8 years ago
|
||
Comment 21•8 years ago
|
||
Comment 22•8 years ago
|
||
Reporter | ||
Comment 23•8 years ago
|
||
Reporter | ||
Comment 25•5 years ago
|
||
Yes. Same symptoms, same conditions.
I have some new (to me...) info that may provide some clues. First off...behavior is the same for Win7 and Win10 (on different hardware) and has been around since before I first reported it (originally noticed it on WinXP!)
Secondly, EFS does some housekeeping occasionally, touching all the files under its control for a few moments. It's a "lazy background" task that seems to cycle through the whole system every couple of weeks. When EFS is doing its thing, it works one file at a time: (1) it EFS creates a temporary copy of the file its working on, (2) changes some metadata, (3) saves the temp file, (4) deletes the original file, and (5) immediately renames the updated temporary copy to the original file name. Why does it do this? I have no clue.
I suspect this is somehow causing Gloda or some other TB module to get confused and declare the MSF files invalid. The effect is that all MSF files will eventually be zeroed-out.
It's possible that TB has a file handle that it doesn't release, and that contention between the app-level handle and the kernel-level privilege causes the collision. Just speculation on my part...
Reporter | ||
Comment 26•4 years ago
|
||
Well, since there was an update on this from Klaus, I decided to look at all my MSF files (since this isn't something I do every day...) and guess what: this symptom has disappeared sometime over the last two years!
So AFAIC this is a closed one...but I'll leave that up to you folks.
Comment 27•4 years ago
|
||
My problem of MSF files vanishing silently may have gone away.
I haven't seen it happen since I did a major cleanup of my Thunderbird config.
My config was pretty old and had experienced lots of transitions.
1996: I started using Netscape Messenger on Windows
2005: Moved to Thunderbird, importing from Netscape.
Also, edited profiles.ini to move my Thunderbird profile into the
tree of folders that I back up regularly, rather than the default location
2009: Moved to Mac, copying my Thunderbird profile folder from Windows
2010: Copied to a 2nd Mac, using Mac Import Wizard to copy all files
2014: Moved to a 3rd and 4th Mac, using Mac Import Wizard to copy all files
I'd also added and removed various add-ons along the way.
And added and removed various email accounts and RSS feeds.
And changed lots of settings.
So perhaps things were getting a bit crufty.
2021: Did a clean install and config on a 5th Mac. Did NOT copy any files
from my old Thunderbird profile, except for:
- The mail files themselves, but not the *.MSF files which I allowed to be re-created
- abook.sqlite
- msgFilterRules.dat
- persdict.dat
- userChrome.css
I manually recreated the 6 email accounts that I currently use.
And manually changed all the settings I care about. Did NOT copy prefs.js.
Things work pretty well now. I'm running my automated script to watch
for MSF files vanishing, and it hasn't reported any yet. See:
I've been using Netscape Messenger and then Thunderbird for 25 years now,
with mostly very good results. I've kept copies of every single mail message
that I've sent or received (except spam and list blasts). My 600+ mail folders
now hold almost 500,000 messages and use about 6 GB of storage. Thank
goodness for the ability to detach attachments, or it would be MUCH bigger!
Thunderbird is still VERY fast for interactions and even for searches. It's a
REALLY great tool!
FYI,
--Fred
Comment 28•4 years ago
|
||
Thanks for your comment, let's talk in 2022. I have the problem of vanishing .msf files once or twice a year, so if you started again in 2021 you can't exclude the possibility for it to happen again. I've a local build so I'll look into the code that's triggered when deleting panacea.dat, bug 1093217, comment #5. Maybe that will give us a clue which code path triggers the mass exodus.
Reporter | ||
Comment 29•4 years ago
|
||
(In reply to Fred Stluka from comment #27)
My 600+ mail folders
now hold almost 500,000 messages and use about 6 GB of storage. Thank
goodness for the ability to detach attachments, or it would be MUCH bigger!
Thunderbird is still VERY fast for interactions and even for searches. It's a
REALLY great tool!FYI,
--Fred
I've been keeping my email continuously since 2001, starting originally from Eudora.
I don't have as many messages as you, but my storage is over 50 GB because of all the attachments.
You mention "the ability to detach attachments" -- is there any way to do that automatically (as Eudora did), with the linkage to the document maintained within the message??
Obviously, this is way off topic for this bug thread, but I'm curious about this issue.
Comment 30•4 years ago
•
|
||
(In reply to Fred Stluka from comment #27)
My problem of MSF files vanishing silently may have gone away.
Here are the details of the symptoms I was seeing:
--Fred
Comment 31•4 years ago
|
||
(In reply to David Taber from comment #29)
I've been keeping my email continuously since 2001, starting originally from Eudora.
I don't have as many messages as you, but my storage is over 50 GB because of all the attachments.
You mention "the ability to detach attachments" -- is there any way to do that automatically (as Eudora did), with the linkage to the document maintained within the message??Obviously, this is way off topic for this bug thread, but I'm curious about this issue.
David,
I don't know how to cause all attachments to be detached automatically,
if that's what you're asking. But yes, when I detach one manually, the
linkage to the attachment is maintained within the message. I can still
click the link from within Thunderbird to open the attachment, just as if
it were not detached. The only difference I see is that when I forward a
message, attachments are forwarded also, but NOT if they were detached.
I strongly encourage people who send me email to NOT send attachments.
Embed the textual content in the email message body instead. Email is
formatted as HTML, so it supports text, colors, fonts, lists, tables, etc.
Even embedded images if necessary. No need to attach Word docs and
other such inefficient stuff. See:
To detach an attachment, right-click it in Thunderbird and choose
"Detach...". Then answer the prompt about where to save it.
Here's what I've told others about this:
I really love the fact that this means I can put it in a file in my
regular directory tree, not necessarily within my Thunderbird
profile directory.That saves a lot of space, especially with large binary
attachments like Word docs, PDF files, etc., because saving such
a binary file as a base64-encoded e-mail attachment takes
about twice as much disk space as saving it as a simple binary
file. So it saves me half the size overall, and none of that size
is inside the Thunderbird mail files/folders.An additional advantage to this approach is that all such
attachment files can reside in my regular directory tree, where
I can open/edit them as usual. I'm not stuck using my mail
tree as a filing cabinet for non-mail items. However, I can
still access them through the mail interface, by clicking on the
attachment icon of a mail message as usual. It's the best of
both worlds.
Hope this helps! If you feel this is too off topic to continue here,
feel free to email me directly at fred@bristle.com.
--Fred
Updated•3 years ago
|
Comment 33•3 years ago
|
||
also encryption related (possibly) Bug 1600280 - Thunderbird not working when profile is on a luks encrypted partition (linux). "Thunderbird is already running, but is not responding."
Comment 34•3 years ago
|
||
Comment from Gene in the other bug, I think he meant to post it here...
"This sounds almost exactly like what was described here: bug 1726319 but I don't think it is really EFS (Encrypted Files System) specific. Someone also made a script to monitor the number of files and saw them change."
Description
•