213636 - Persistence of deleted emails in local folder subfolders

Status: RESOLVED DUPLICATE of bug 185740

(MailNews Core :: Database, defect)

Description

[Raphael LAVANCHY]

User-Agent:       Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.3) Gecko/20030312
Build Identifier: Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.3) Gecko/20030312

I'm using Mozilla 1.3
I created subfolders in the local folder directory to sort and make order in my
emails. I remarked since yesterday that after deleting mails previously located
in subfolders and after emptying the trash folder, the global weight of the
local folder directory remained the same !!
I checked the weight through a Windows's window right-clicking on the directory
then selecting Property.
I remarked that deleted mails were EFFECTIVELY deleted only if i deleted the
subfolder....
So i guess there is a bug regarding an efficient deletion of emails. Normally,
deleting an email it would be fully deleted and of course the weight of the
subfolder should be lower...


Reproducible: Always

Steps to Reproduce:
1/ create a subfolder in the local folder directory
2/ place some mails within (100  for exemple to make sure the weight of the
subfolder be large enough)
3/ check the weight of the local folder directory
4 / delete 30 mails from your subfolder. Empty the trash of the local folder
directory
5/ check back the weight of the local folder directory, normally its weight
remains the same, that's NOT normal..given you deleted mails

Actual Results:  
it happens that the weight of the local folder directory remains the same while
mails have been deleted which is NOT normal.


Expected Results:  
Normally, when a email is being deleted, it is placed in the trash folder. Once
you empty the trash folder so emails which were inside should be fully and
completely erased. Currently the only way to erase them completely is to erase
the subfolder. SO it seems that erased emails STILL remain even if they are no
longer displayed.

Comment 1

[Fernando Garcia Bermudez]

Affected Builds:
Mozilla 1.3 - 1.4 (including nightly builds in the period in between)
Mozilla Thunderbird 0.2 (20030901)

Operating systems tested:
Windows 2000 Service Pack 4
Red Hat Linux 9

I am actually using Thunderbird 0.2
I have my "Local Folders" in a directory located in a vfat (Fat 32) Hard Disk.
As it goes, I delete a lot of messages that I archive in my "Local Folders" as
these get older.
When, for any reason, I reinstall any version Mozilla or Thunderbird and I
create a new profile which I later link to these same "Local Folders", all the
messages that I have deleted reappear, and sometimes multiply out of nowhere. I
have a hard time redeleting most of them once again because, the worst thing, is
that they also appear as unread. So, no matter how many times I delete them, or
how many times I empty the trash, they still reappear every time that I happen
to reinstall a new version of mozilla with a new profile.

I have reproduced this same bug when installing mozilla 1.4, after completely
deleting 1.3 (any build) on Windows. I have also recently installed Thunderbird
0.2 for Linux (Red Hat 9) and linked it to these same folders to find the
problem reappear.

I really need this bug to be fixed so as to greatly manage my e-mails, even in
this cross-platform fashion. If not, there appears to be no other way of keeping
my (5000+) e-mails backuped in case of a reinstallation of the mail client (just
because this is the only way to "export" the e-mails).



Reproducible: Always

Steps to reproduce:
1- Specify the "Local Folders" to point to another directory other than the
profile folder.
2- Create a subfolder inside "Local Folders"
3- Move a significant amount of e-mails.
4- Restart mail client.
5- Erase the e-mails from this folder and empty the trash.
6- Restart mail client to check if they were deleted.
7- Uninstall ThunderBird (or Mozilla) completely and delete the profile folder
(I think it still works if you only remove the profile folder).
8- Install Thunderbird (or Mozilla) again and create a new profile when it restarts.
4- Specify the "Local Folders" directory to be the previous one.
5- Restart the mail client and go to the folder where mails were erased to see
that they are still there and sometimes, even unread and multiplied.

Actual Results: What must be happening is that the email folder file is never
updated and only the index is. After reinstalling the mail client, this one must
be rebuilding the e-mail database out of each of the folders (which still
contain the erased e-mails). And because these are not in the previous index,
this is why it must be marking them as unread.

Expected Results: E-mails erasing efficiently so as to reduce not only the size
of the mail folder but the hassle of dealing with redeleting them.

Note: Because every time this happens, some of the messages multiply, my e-mail
folder is reaching almost 90 Megs of space without even having as many e-mails
with attachments. I sometimes get almost 6 copies of the same e-mail reappearing.

Thanks for taking the time to read the bugs and to produce such incredible
browser/mailnews clients.

Comment 2

[David :Bienvenu]

Raphael, you need to compact folders to have their size be reduced. You can
either manually compact the folders (File | Compact Folders), or set up Mozilla
to automatically compact folders when it will save a certain amount of space.

Re Fernando's comment, are you using pop3 and leave on server? Are these
messages that are multiplied getting downloaded from the pop3 server? And how
did these messages get into Mozilla in the first place? Were they downloaded, or
imported? We store the read state of the messages in the actual local folder, as
well as the .msf file, so that if we have to regenerate the .msf file, we can,
from the local folder. But this is only possible if the mail was put in the
local folder by Mozilla.

Comment 3

[Fernando Garcia Bermudez]

I am using POP3 but I have the option to delete the messages as they move from
my inbox or are deleted to trash. However, the bigger problem comes in the Local
Folders, not the Inbox of the account (where all my messages are downloaded).
I've been using Mozilla for the last 8 months or so, so I had to import all my
previous e-mail messages. What I did was the following:

1- I archived and downloaded the messages of my Yahoo! Mail account.
2- I dragged and dropped the messages into Outlook (which was the only program I
had which allowed me to do this for .eml messages - Mozilla wouldn't work)
3- When I installed Mozilla 1.3, I imported all the e-mails from outlook and
then I organized them into the "Local Folders".
4- Later on, I continued installing updates of Mozilla up to the 1.4 milestone.
Always using the same "Local Folders" and sometimes deleting the profile,
because in the middle I reinstalled Windows and everything. Obviously, I kept
the "Local Folders".
5- Two weeks ago, I installed Thunderbird, configured the same local folders and
started having this same problem. Yesterday I even installed Thunderbird 0.2 in
Red Hat 9 and experienced the same while accesing my "Local Folders". Needless
is it to say that when I installed it on Linux, and the problem arised again,
because I was using the same files, when I went back to Windows, the problem
appeared there as well (for obvious reasons).

Note: I had some problems in the first place when I imported the messages,
because I mistakingly imported a great amount of messages three times, just
because some of the e-mails didn't appear to have imported. At that time I
didn't know that Mozilla was not going to overwrite the previous e-mails, but,
on the contrary, would allow exact copies of the messages. However, and using my
reasoning and experience, I should have because of this problem only 3 copies of
some e-mails, but not 7 copies.

Thanks for helping better this excellent program...

Comment 4

[David :Bienvenu]

OK, the import tool is not correctly adding an x-mozilla-status header to each
message. That's why the read state is getting lost when we reparse the folder.
in Moz 1.5, and Thunderbird 0.2, I made it so compacting a folder would add an
x-mozilla-status line to messages that didn't have them. The hope was that
folders would get compacted in the normal order of things, and the problem would
just go away...

So, can you look at one of your local folders that has this problem, verify that
the messages don't have an x-mozilla-status line, and then try compacting the
folder, and verify that the x-mozilla-status line is added? If so, that will
make it so you don't lose the read-state of messages. I don't know what's
causing the doubling up of messages..

Comment 5

[Fernando Garcia Bermudez]

I chose a folder with those renewing unread messages, checked that none of the
messages had the "x-mozilla-status" line, compacted the folder, restarted
ThunderBird 0.2 and checked the file again to see that no line had been added. I
made a search for "x-mozilla-status" and find none whatsoever in the whole
folder. Also, I tested it with a specific message that had the problem and read
the header completely before and after compacting. It doesn't seem to be working.

Also, how can I setup thunderbird to compact the folders on a periodic basis. It
seems that if that is the answer to my problems, then there should be a better
way than just going folder by folder and compacting them every time I delete a
significant amount of mails from each. At least a "compact all folders" options
should be nice.

Lastly, the problem with mail reappearing after being deleted, which is the one
that is troubling me the most(apart from the equally annoying multiplication).
Would that be solved by simply compacting the folder in question?

(I know once I had a problem with my junk folder in one of my accounts. This one
showed 8 messages unread and there were none inside. Compacting it solved the
problem. What does compacting really do?)

Thanks very much for helping me out...mozilla rocks...

Comment 6

[David :Bienvenu]

Did you delete a message from the folder before compacting? I forgot to mention
that you need to do that :-) When you delete a message, we simply mark the
message as deleted, but don't remove the message from the mail folder, because
that would involve rewriting the folder all the way to the end, and prevent
undoing the delete. what compact does is remove all the deleted messages by
copying the non-deleted messages to a temp folder and then copying the temp
folder over the original folder.

Edit | Preferences | Mail & Newsgroups | Offline & Disk Space

then check the box that says "Compact Folders when it will save over XX Kb"

Also, File | Compact Folders should compact all folders that have deleted messages.

Comment 7

[Fernando Garcia Bermudez]

I have thunderbird 0.2, and the Offline and Disk Space tab appears to have
removed. It would be nice to have it back in future builds. Still, th
File/compact Folders is working, but it would be better to have it done
automatically.

So far, I think this information will solve my problems. Thanks very much for
all and if it continues bothering, I just let you know.

I'll look forward to sseing the Offline and Disk Space feature back in the
future builds. Drop me a line if it is done before the next milestone...thanks...

Comment 8

[OstGote!]

duping to some of the many reports about the original reported issue

*** This bug has been marked as a duplicate of 185740 ***