278411 - Thunderbird can nuke all your sent mail

Status: RESOLVED INVALID

(MailNews Core :: Networking: IMAP, defect)

Description

[Gervase Markham [:gerv]]

Twice now, Thunderbird has deleted my entire sent-mail folder when I attempted
to delete another folder. The first time, I lost five months of mail, the second
time, three weeks. Despite having hit the bug twice, I don't know exactly why it
occurs.

My configuration is to use IMAP; I use several copies of Thunderbird to access
the same account(s) from different machines. I have the "mark as deleted" delete
model, which means that folder deletion is immediate and irreversible (protected
by a warning message). My usual sent-mail folder is called "sent-mail", and it's
on the IMAP server as a subfolder of my INBOX, like all the others.

What happened is that I deleted a folder, called "Sent" in both cases, which was
_not_ configured as my Sent Mail folder in that copy of Thunderbird - at least,
not according to the Thunderbird UI, given that it doesn't offer the "Delete"
right-click menu option for the configured Sent Mail folder. That folder
disappeared, but the entire contents of my "sent-mail" folder went with it. 

More specifically, on the server disk, the cur, new and tmp directories inside
"sent-mail" vanished but the folder itself remained. This meant that Thunderbird
refused to create a new folder, but could also not properly delete the old one.
This had to be fixed by a server admin. This is definitely not right.

In the second case (which I can remember rather better) it appeared that one of
my copies of Thunderbird (1.0RC1) had been saving mails to this Sent folder (or
rather, to a folder several levels deep inside it), even though it was not
configured to. I believe I may have just subscribed to this folder from the
other Thunderbird and so noticed it. I found a week or two's mail in there, so
perhaps my last Thunderbird upgrade had caused this behaviour.

Anyway, I moved the mails to my usual "sent-mail" folder and deleted the Sent
folder, and that's when the bug occurred.

The Thunderbird versions used were 1.0RC1 and the one on my laptop - I'll look
it up and add a comment. And if anyone tells me to reproduce it using a more
recent version, I will... not be happy.

I've attempted to reproduce it using a test account, but have so far not been
able to. This may not be much to go on, but please look into it - it's a really
nasty dataloss problem. If you need more info, please let me know.

Gerv

P.S. Irritating thing number 2 is that Thunderbird does not download mail for
offline use until you actually click "Work Offline"! After the first problem, I
marked all my folders for offline use so that I'd have a backup. But, three
weeks later, when it happened, the files were all still size 0! :-((

Comment 1

[David :Bienvenu]

>More specifically, on the server disk, the cur, new and tmp directories inside
>"sent-mail" vanished but the folder itself remained. This meant that Thunderbird
>refused to create a new folder, but could also not properly delete the old one.
>This had to be fixed by a server admin. This is definitely not right.

Did this happen both times? To me, it sounds like a problem on the server, and
possibly related to your loss of mail. We should not be able to corrupt the
server mail store like this through legal (or illegal) IMAP protocol.

When you deleted the Sent folder, and the sent-mail folder got corrupted, did
the Sent folder itself get deleted correctly?

Are you in this situation again where you have a Sent folder that you want to
delete? (how did this Sent folder get created? Did you run thunderbird/seamonkey
against that account and send mail w/o changing the fcc folder to "sent-mail"?)

Two things would be helpful - your prefs.js, and an imap protocol log of a
session where you connect to that imap account, so I can see what the folder
discovery looks like (i.e., what folders are on the server...). A protocol log
of where you delete the Sent folder would also be interesting...

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

It's highly unlikely that the client could cause folder B to get corrupted on
the server when folder A is deleted - the protocol that we send is very specific
to the folder that its operating on. 

Comment 2

[Gervase Markham [:gerv]]

> Did this happen both times?

Yes.

> When you deleted the Sent folder, and the sent-mail folder got corrupted, did
> the Sent folder itself get deleted correctly?

Yes.

> Are you in this situation again where you have a Sent folder that you want to
> delete?

Not right now. If it occurs again, I will take and verify three independent
backups, turn on the IMAP protocol log, and log the session.

> Two things would be helpful - your prefs.js, and an imap protocol log of a
> session where you connect to that imap account, so I can see what the folder
> discovery looks like (i.e., what folders are on the server...).

I will attach both.

> It's highly unlikely that the client could cause folder B to get corrupted on
> the server when folder A is deleted - the protocol that we send is very 
> specific to the folder that its operating on. 

Indeed - but it could be that somewhere in Thunderbird, the folder name "Sent"
is hardcoded, such that when such a folder is referred to, it assumes it means
the configured sent-mail folder. Or something like that.

Does the IMAP server or datastore have any notion of a special sent mail folder,
or is that purely a client concept?

Gerv

Comment 3

[Gervase Markham [:gerv]]

Attachment: My prefs.js

Comment 4

[Gervase Markham [:gerv]]

Attachment: IMAP protocol log of startup and "Get All Messages"

This is an IMAP protocol log. I started Thunderbird, waited a bit, pressed
Ctrl-Shift-T, waited a bit more, then pulled down the Get Mail menubutton and
picked "Get All New Messages", then closed Thunderbird.

Comment 5

[David :Bienvenu]

From the log, it looks like you do have a sub-folder of the Inbox called Sent
for your markham-4 account. Is that a test account? Which is your main account,
the one you had the problem in?

The IMAP server doesn't have an idea of a Sent folder, but the client code
doesn't look for a folder called "Sent" - it uses the fcc pref. My guess is that
the Sent folder got created because the default value of that pref is "Sent",
and you changed the fcc pref after the "Sent" folder was created. But more
importantly, the code that deletes a folder doesn't have any special case
handling for special folders like Sent, except that it disables the delete
operation in the front end. I also see no way the client imap code could delete
two folders when you delete one folder in the UI, and as I said before, I don't
know how the client could corrupt the server store...My suspicion is that
accessing the same account from multiple clients might have got the server
confused - I say this only because I know nothing about the courier server code.
Perhaps if the shoe was on the other foot, I'd suspect the client code...

Comment 6

[Gervase Markham [:gerv]]

OK, I've figured this one out. Some unknown thing (perhaps the webmail) was
creating symlinks within my folder hierarchy, so that when I deleted this odd
folder, it was a symlink to my sent-mail, and so it emptied that out.

While incredibly irritating, I'm afraid I can't blame Thunderbird...

However, the similar Local Folders bug I filed, bug 278612, is still a valid
dataloss bug.

Gerv