Closed Bug 1322476 Opened 8 years ago Closed 3 years ago

some .msf files get lost after Thunderbird restart

Categories

(Thunderbird :: Folder and Message Lists, defect)

Product:
Thunderbird
Component:
Folder and Message Lists
Version:
45 Branch
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1551173

People

(Reporter: mschark, Unassigned)

Details

(Keywords: dataloss) 

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36

Steps to reproduce:

restarting Thunderbird


Actual results:

After restarting on the next e-mail download Thunderbird starts to complain about some random folder it can't move e-mails (filter rules) to.
When forcing the the re-filtering manual, the e-mails are moved, but the target mail folder does not show a number of unread or read mails (just size).


Expected results:

currently this must be fixed manually by selecting the folder in the folder list.
the missing .msf file is rebuild in the moment of selection (you will shortly see the account info in the folder content pain). sorting and column setting are reset.

the need this for this manual fix is a pain (having 1000+ folder) and should be avoided.
When exactly did this start happening?
Is it also 100% reproducible with Windows and Thunderbird started in safe mode at the same time?
Flags: needinfo?(mschark)
Yes, there is some anecdotal evidence that MSF files get lost at times. See bug 1279344. That bug is mainly about storing the TB profile on some - say - less that usual file system, but in bug 1279344 comment #7 I describe an incident where I lost 1000 MSF files. I don't know how it happened and it hasn't happened since. So if you have a reproducible case, please let us know.
Status: UNCONFIRMED → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE
I run Thunderbird on Win 7 64-bit with NTFS and this issue can be reproduced any time.

!!!! This is neither a duplicate of the above bug 1279344 nor resolved !!!!

It happend today right after the actual release update installation (incl. restart)
again. This issue exist since many releases, I don't even remember since when.
The only visual indicator is a folder list in which some or many folder have a 
size (even after compression) but no e-mail counter.
Status: RESOLVED → UNCONFIRMED
Flags: needinfo?(mschark)
Resolution: DUPLICATE → ---
(In reply to mschark from comment #3)
> I run Thunderbird on Win 7 64-bit with NTFS and this issue can be reproduced
> any time.
I run Thunderbird on Win 7 64-bit with NTFS and this issue *CANNOT* be reproduced any time.
If fact, in many years of using Thunderbird I've seen it once in June 2016 as documented in bug 1279344 comment #7. It hasn't happened since.

If you can give me some steps to reproduce this problem, I'm interested.

> The only visual indicator is a folder list in which some or many folder have
> a size (even after compression) but no e-mail counter.
So you're saying that if you use View > Layout and switch on "Folder Pane Columns" you can spot the problem? I've switched this on now and I will observe it.
(In reply to mschark from comment #0)
> Actual results:
> After restarting on the next e-mail download Thunderbird starts to
> complain about some random folder it can't move e-mails (filter rules) to.

What do you mean by "complain about some random folder it can't move e-mails (filter rules) to"?
Are you talking about phenomenon of bug 931303?

> When forcing the the re-filtering manual, the e-mails are moved,
> but the target mail folder does not show a number of unread or read mails (just size).

This is normal phenomenon, if you do "Run filters on folder and action=move to folder is executed", "manual copy/move mail to not-opened folder" and so on, even though state of "outdated msf condition", "no msf file", "msf fileSize=ZERO" exists. 
You need to do "Folder open by folder click at folder pane", "Repair Folder", and so on, in order to invoke Rebuild-Index in order to remove such unexpected/unwanted state, before you do "Run filters on ..." etc.

> the need this for this manual fix is a pain (having 1000+ folder) and should be avoided.

Was these 1000+ folders healthy before you saw your problem?
 Healthy" in this context == 
   When you recently opened the folder by click at folder pane before you saw your problem, you could view mail normally.

(In reply to mschark from comment #3)
> It happend today right after the actual release update installation (incl. restart) again.
> This issue exist since many releases, I don't even remember since when.
> The only visual indicator is a folder list in which some or many folder have a size (even after compression)
> but no e-mail counter.

What do you mean by "compression"? "Compact" or "Compact Folders" in Thunderbird?
If so, because "Compact" != "Repair Folder", phenomenon you saw is normal when something bad such as "outdated msf condition" still exists in the folder.

What do you mean by "actual release update installation (incl. restart) again"?
Did you upgrade from Thunderbird 45.a.b to Thunderbird 45.x.y where a<x or a==x && b<y?

> This issue exist since many releases, I don't even remember since when.

Do you mean that your problem occurred on same 1000+ folders upon your any Thunderbird version up/Thunderbird release up?
Did you cleared "error status" of all relevant folders when you saw your problem upon a "release update installation"?
If not, it's pretty normal, because the "error status" won't be cleared automatically unless action such as "explicit folder open by folder click at folder pane", "Repair Folder", is actually executed by you.

Please note that (a) "issue of msf file is broken/deleted/size=0" and (b) "problems due to you kept such error status for long time" are different.
Here is B.M.O for developers to resolve issue of (a). Here is not support forum for (b).
"
(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #1)
> When exactly did this start happening?
> Is it also 100% reproducible with Windows and Thunderbird started in safe
> mode at the same time?

It is enough to restart Thunderbird.
There is no relationship to any kind of Windows reboot !
(In reply to WADA from comment #5)
> (In reply to mschark from comment #0)
> > Actual results:
> > After restarting on the next e-mail download Thunderbird starts to
> > complain about some random folder it can't move e-mails (filter rules) to.
> 
> What do you mean by "complain about some random folder it can't move e-mails
> (filter rules) to"?
> Are you talking about phenomenon of bug 931303?
> 

No, since there is no error message I just have no idea why and which mail filter
will fire next. But if the msf file is gone, the filter execution moves the mail to
the inbox instead. The code seems to care about the situation, but not solving it.


> > When forcing the the re-filtering manual, the e-mails are moved,
> > but the target mail folder does not show a number of unread or read mails (just size).
> 
> This is normal phenomenon, if you do "Run filters on folder and action=move
> to folder is executed", "manual copy/move mail to not-opened folder" and so
> on, even though state of "outdated msf condition", "no msf file", "msf
> fileSize=ZERO" exists. 
> You need to do "Folder open by folder click at folder pane", "Repair
> Folder", and so on, in order to invoke Rebuild-Index in order to remove such
> unexpected/unwanted state, before you do "Run filters on ..." etc.
> 

How can you claim this to be normal ? Filtering into a folder with no msf
means you can't see or find these mail any more with no message telling you at all
that this issue exists. As a matter of fact you a kind of loose your mail, since
they are hidden. I have a monitoring solution (RoboTask) running to notify me, if
this problem shows up again.

There is no index repair necessary to fix the issue, you just have to click on the 
folder and the msf file will be recreated.

Why isn't Thunderbird doing this automatically, since it knows the issue (all folders in the
folder list, where a size is shown but the mail count is missing are having this issue) ?

Doing it manually is no solution (just a bad workaround) since you can't search the folders
in the folder list and due th the folder hierachy you can't see all folders at once.


> > the need this for this manual fix is a pain (having 1000+ folder) and should be avoided.
> 
> Was these 1000+ folders healthy before you saw your problem?
>  Healthy" in this context == 
>    When you recently opened the folder by click at folder pane before you
> saw your problem, you could view mail normally.
> 

Yes, it seems to be no issue of the folder or mail data file in the folder.
Just the msf file is lost.

> (In reply to mschark from comment #3)
> > It happend today right after the actual release update installation (incl. restart) again.
> > This issue exist since many releases, I don't even remember since when.
> > The only visual indicator is a folder list in which some or many folder have a size (even after compression)
> > but no e-mail counter.
> 
> What do you mean by "compression"? "Compact" or "Compact Folders" in
> Thunderbird?
> If so, because "Compact" != "Repair Folder", phenomenon you saw is normal
> when something bad such as "outdated msf condition" still exists in the
> folder.
> 
Yes, compact might be the term in Englisch (I am using the standard funktion in
the folder popup menu and Thunderbird is automatically running this function once in a while)

> What do you mean by "actual release update installation (incl. restart)
> again"?
> Did you upgrade from Thunderbird 45.a.b to Thunderbird 45.x.y where a<x or
> a==x && b<y?
> 
I updated to the latest version 45.5.1 in the release update-channel (see Thunderbird -> About)

> > This issue exist since many releases, I don't even remember since when.
> 
> Do you mean that your problem occurred on same 1000+ folders upon your any
> Thunderbird version up/Thunderbird release up?

Yes

> Did you cleared "error status" of all relevant folders when you saw your
> problem upon a "release update installation"?
> If not, it's pretty normal, because the "error status" won't be cleared
> automatically unless action such as "explicit folder open by folder click at
> folder pane", "Repair Folder", is actually executed by you.

Updates do not seem to touch the mail data at all.
I never had to run "Repair Folder" since a simple click on the folder is 
enough to recreate the msf. What do you want to fix, if the file is gone ?
It just gets newly created.

Doing this manual is a bad workaround as I described above.

> 
> Please note that (a) "issue of msf file is broken/deleted/size=0" and (b)
> "problems due to you kept such error status for long time" are different.
> Here is B.M.O for developers to resolve issue of (a). Here is not support
> forum for (b).
> "

Tare you sure, you understood the issue WADA ? This comment does not apply !
(In reply to Jorg K (GMT+1) from comment #4)
> (In reply to mschark from comment #3)
> > I run Thunderbird on Win 7 64-bit with NTFS and this issue can be reproduced
> > any time.
> I run Thunderbird on Win 7 64-bit with NTFS and this issue *CANNOT* be
> reproduced any time.
> If fact, in many years of using Thunderbird I've seen it once in June 2016
> as documented in bug 1279344 comment #7. It hasn't happened since.
> 
> If you can give me some steps to reproduce this problem, I'm interested.

Just a Thunderbird restart for any reason is enough.
It never occurred while runing Thunderbird, only after a restart (or crash).

> 
> > The only visual indicator is a folder list in which some or many folder have
> > a size (even after compression) but no e-mail counter.
> So you're saying that if you use View > Layout and switch on "Folder Pane
> Columns" you can spot the problem? I've switched this on now and I will
> observe it.

Yes, my folder list usually shows the number of open mails, total mail and
the folder size. In case of the lost msf files the mail counts disappear but
the folder size is still shown and new mails moved to the folder will not 
highlight the foldername any more.
Could it be your virus tool is accessing the TB files? Could you try to exclude the profile directory (C:\Users\YOUR USERNAME\AppData\Roaming\Thunderbird) from checking through your virus tool?
(In reply to mschark from comment #7)
> > Please note that (a) "issue of msf file is broken/deleted/size=0" and (b)
> > "problems due to you kept such error status for long time" are different.
> > Here is B.M.O for developers to resolve issue of (a). Here is not support
> > forum for (b).
> 
> Tare you sure, you understood the issue WADA ? This comment does not apply !

Does it mean following?
 You did click of 1000+ folders at folder pane of Thunderbird and forced recreation of .msf of 1000+ folders,
 upon each "deletion of 1000+ .msf files which always happened when you did update via. release update-channel".
 i.e. You repeated "click of 1000+ folders at folder pane of Thunderbird" N times
      where N >= number of your execution of "update via. release update-channel" in the past.

Or following?
 You didn't do so, and you clicked some folders only when you saw your "deletion of 1000+ .msf files" problem,
 and you are requesting "automatic recovery of the deleted 1000+ .msf files" by this bug report at B.M.O,
 in addition to fixing problem of (a)?

> > Are you talking about phenomenon of bug 931303?
> No, since there is no error message (snip)
> But if the .msf file is gone, the filter execution moves the mail to the inbox instead.

(i) "outdated msf condfition", (ii) "fileSize=0", (iii) "file doesn't exist", are different conditions.
It seems that error message of bug 931303 is shown only when (i).

> The code seems to care about the situation, but not solving it.

You are right.
As written in bug 931303, meaning of error message of bug 931303 is "outdated msf condition was detected, but I can do nothing on it, so please do folder click at folder pane or do Repair Folder for recovery", although "statement written in the error message" is far different from it :-)
Flags: needinfo?(mschark)
(In reply to Richard Marti (:Paenglab) from comment #9)
> Could it be your virus tool is accessing the TB files? Could you try to
> exclude the profile directory (C:\Users\YOUR
> USERNAME\AppData\Roaming\Thunderbird) from checking through your virus tool?

I am using Microsoft Security Essentials,  I guess we would know, if that's an issue ...
Even if some tool blocks you from writing to a file (which MSE only does in case of a virus)
you get a message and Thunderbird would know and should react correctly.
Severity: normal → critical
Keywords: dataloss

Last year MS replied "No" to "Do you use any tools like ccleaner?"

So what could this be?

Let's make this a forward duplicate of bug 1551173.

Status: UNCONFIRMED → RESOLVED
Closed: 8 years ago → 3 years ago
Flags: needinfo?(mschark)
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.