Random massive data loss in local mbox files

RESOLVED WORKSFORME

Status

defect
--
critical
RESOLVED WORKSFORME
13 years ago
11 years ago

People

(Reporter: nospam, Unassigned)

Tracking

({dataloss})

Firefox Tracking Flags

(Not tracked)

Details

Reporter

Description

13 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1
Build Identifier: Thunderbird 1.5.0.9

Today I found out that most of my mbox mail files are totally messed up. Messages are empty, others contain parts of mbox organisation structures, corrupted "From" parts and multiple e-mail fragments in one message. Other messages from another folder are numerously repeated under the taglines of lots of other mails. Compressing the folder seems to make the problem persistent if it wasn't before. Deleting the folder summary file from the disk makes some messages disappear. Then I noticed a total mess in the inbox folder of that account. Usually I have rules to distribute mailing list messages in their folder depending on the List-Id header, there have been mails from all around in my inbox. The problem seems to be limited on that one account. "Local folders" is still intact as far as I can see. The remaining accounts are IMAP.

Reproducible: Didn't try




If you want me to do tests, ask me. I have no idea where to start. I'm going to freeze the entire account for later investigation.

I have the impression that using mbox files is highly dangerous. This format is very error prone. Additionally, my hard disk is totally fragmented all the time which makes using Thunderbird (and creating backup archives) extremely slow. Can't you use maildir format like any decent MTA or a database like SQLite?
Reporter

Updated

13 years ago
Version: unspecified → 1.5
maildir is planned for tb3.
Do you see this in the thunderbird ui, or where?
Reporter

Comment 2

13 years ago
Yes, clicking on the messages in the message list in the Thunderbird window or using arrow keys to navigate shows this problem.

Maybe some more info on the data: The entire account is uncompressed 174 MB on the disk. I haven't tried compacting more than the one folder. As far as I can remember there have been around 10,000 to 15,000 messages in all folders. I've checked the filesystem (NTFS) and it's clean. I haven't been using a filesystem defragmenting tool for a long time (it just had no effect although it should). The other account, Local Folders, is currently 315 MB on the disk (also not compacted).

Thunderbird 3 will still take some time I guess.

Comment 3

13 years ago
I had this exact same problem today (on Mac OS X) when I decided to give Thunderbird 2 Beta 2 a try. Fortunately I had made a full backup of my profile before installing the new version.
Hmm, why the rash of recent cc: 
shishz?

Yves, have you seen this in recent Thunderbird 2?
Assignee: mscott → nobody
Keywords: dataloss
Reporter

Comment 5

11 years ago
I don't trust Thunderbird enough so I avoid large folders or touching a large number of messages at a time.

Waiting for maildir or a database backend in Thunderbird and continuing to dream of my own MUA with an intuitive UI, true tagging organisation and integrated Jabber support...
Yves, this was an imap account? A the problem folders all the target of a filter? (i.e. you move messages to them?)

João writes "I no longer use Thunderbird so I can't really say if the bug is still present. I switched over to Apple's Mail.app about 7 or 8 months ago."
Reporter

Comment 7

11 years ago
As I already said in my initial report, I do use filters there. I don't know whether all of the affected folders were filter targets. Long ago, cleaned away, never seen again.
Thanks. If you no longer see the problem - corrupted "From" parts and multiple e-mail fragments in one message. Other messages from another folder are numerously repeated under the taglines of lots of other mails - let's close this WFM.  You can reopen if you see the problem again in the future.
Status: UNCONFIRMED → RESOLVED
Last Resolved: 11 years ago
Resolution: --- → WORKSFORME
Reporter

Comment 9

11 years ago
Uh wow, this is the second more or less serious bug I opened here that was almost ignored for over a year and then closed as WFM because the error didn't happen again. Interesting strategy to make a programme more stable... Why do I report bugs at all?
(In reply to comment #9)
> Uh wow, this is the second more or less serious bug I opened here that was
> almost ignored for over a year 

I didn't ignore it - I just came on the scene. And, as I've also seen some examples of what you describe I'm semi qualified to say I haven't seen the myself for a long time.  Besides, maildir is not necessarily the answer to the problem, even if it still exists.

Do you have a proposal of how to resolve a reported bug that is x years old and 1/1+ releases ago?
Reporter

Comment 11

11 years ago
I don't have a proposal how to resolve a reported bug x years old. I would say it should have been done x years ago... I don't mean to offend anyone or you in particular. This is just my impression of this facility. I know that I cannot demand anything for what I haven't paid. Indeed, it is hard to do anything in these cases now. Best to leave them sleeping and wait for somebody else to report it again in a newer version because it probably wasn't analysed or fixed in the past. There should be another bug status: MISSED or FORGOTTEN to close this kind of reports. I guess there are plenty of them.
(In reply to comment #11)
> Best to leave them sleeping and wait for somebody else to
> report it again in a newer version because it probably wasn't analysed or fixed
> in the past. There should be another bug status: MISSED or FORGOTTEN to close
> this kind of reports. I guess there are plenty of them.

In Thunderbird, by my estimate, there are 10,916 open bugs at this very moment, 3,296 of which are UNCO and probably another 2,000-3,000 which should be UNCO because they were confirmed under much less stringent conditions. Extrapolating from some of my experiences, the number of identifiable, distinct, and valid bugs hovers between 5,000 and 7,000, while the number of bugs explained enough as they stand to be fixed is around 1,000 or less.

Keeping open a bug because it may or may not have been fixed in a later version, the OP doesn't exist for bugzilla purposes, and that precious few people will ever come across anyways is pointless.

Besides, there's a standard procedure for these bugs:
RESO WFM (or INCO in cases where the OP leaves). If someone has the same problem, he/she can open a new bug or reopen the old one (the former is more likely).
You need to log in before you can comment on or make changes to this bug.