Closed
Bug 260226
Opened 20 years ago
Closed 20 years ago
Email marked as spam disappears - not in trash folder or junk folder
Categories
(Thunderbird :: General, defect)
Tracking
(Not tracked)
RESOLVED
DUPLICATE
of bug 260981
People
(Reporter: mskorman, Assigned: Bienvenu)
References
Details
(Keywords: dataloss, regression)
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10 Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10 Email marked as spam disappears. It does not appear in trash folder or junk folder. Reproducible: Sometimes Steps to Reproduce: 1.Can't figure it out. There is an optition in the buttons extension that allows junk mail to be completely deleted but I never checked that. To be sure, I uninstalled buttons. 2. 3. Actual Results: The frequency has increased. Expected Results: Placed email marked as spam in the trash folder and shown the messages as unread.
Comment 1•20 years ago
|
||
Which version of TB? If you're using 0.8, are you also using the Global Inbox feature? See bug 257421.
Confirming on version 7.0 (20040909) - CVS build I've seen this a few times over the last few weeks. I wasn't sure at first as I have several mailboxes and it's not easy to waych them all as it's d/ling. I've now turned off auto fetching and d/l one mailbox at a time. I've just watch it get 72 mails for an account, the junk filter ran, and I ended up with 2 in the Inbox and two in Junk - 68 missing!! This is serious, even though my junk filter is very accurate (months since I had a false positive). I am NOT using Global Inbox but I've noticed something odd. I don't have a folder named Junk - I have the junk filter send junk straight to Trash - but a Junk.msf file exists in every mail directory (but no Junk file) with a current timestamp.
Comment 3•20 years ago
|
||
I'm going to confirm this because I think bug 260216 and bug 260370 might be the same problem. I hope this gets some attention soon from people who can figure it out better than I can. (In reply to comment #2) > I've just watch it get 72 mails for an account, the junk filter ran, and I > ended up with 2 in the Inbox and two in Junk - 68 missing!! You mean "two in Trash," right? You might try turning off the setting to automatically move junk mail. You'll get a lot more pointless notifications, but it would be good to know whether this deletion is taking place as a function of the "move junk mail" or the "detect junk mail" portion of the problem.
> You mean "two in Trash," right?
Duh! yes, I do
Does the Junk Filter move/delete messages in the same way as doing it manually,
i.e. it doesn't physically delete them from Inbox, but flags them deleted, to be
removed when the Inbox is Compacted?
I've just opened the Inbox in question in a text editor and the 68 "missing"
mails are there, all with the status flags set to
X-Mozilla-Status: 0008
X-Mozilla-Status2: 00010000
OK, I've just seen this happen on 3 mailboxes: Go to Inbox Click the Get New Messages button Watched several mails arrive in the Inbox, automatically marked as Junk Open the "junk" folder (Trash in my case) - messages not there Switch back to Inbox In Exporer, go to the mail folder and delete Trash.msf Switch to Trash - and there are the messages (checked timestamp to confirm) So, it seems that the Junk filter is DTRT, but the .msf file of the "junk" folder isn't being updated. Also, I have seen a variation on this where the mails disappear from Inbox but "Trash (xx)" in the folder tree doesn't chamge to bold and the unread count doesn't change, but when I switch to Trash it does, and the deleted messages are there.
Assignee | ||
Comment 6•20 years ago
|
||
X-Mozilla-Status: 0008 that means the message is deleted. I believe the junk mail plugin moves messages with the same code as filters and ui move/copies. I wonder if what's happening is that the target folder db gets out of date, so the summary file is invalid, and we copy the message to the dest folder w/o updating the db, but then some code comes along and erroneously marks the db as valid.
Assignee: mscott → bienvenu
I can't be 100% sure of this yet, but it appears that it may only happne when the "junk" folder is empty, but maybe that's because I'm using Trash as my "junk" folder and it is a special folder - the Empty Trash function?
Comment 8•20 years ago
|
||
If it happens when the destination folder is empty, I wonder if it is more widespread than junk mail handling. For example, I created a bunch of empty folders to hold mail from each of my friends...
Re comment 8 It seems to only affect Junk. I have a lot of filters set up and they seem to be working OK.
Comment 10•20 years ago
|
||
In bug 260370 comment 2 (a probable dupe), reporter says that the setting for "Move detected junk to..." is turned off. She also is getting notifications when all the incoming mail has disappeared.
Reporter | ||
Comment 11•20 years ago
|
||
As far as I could tell this only happened with mail marked as junk on my computer. It didn't happen every time but often enough. I never had this problem at all until I upgraded to TB .8. I have been using Mozilla 1.x mail for years and switched to Thunderbird/Firefox 4 months ago. Two nights ago, I uninstalled TB .8 and FF pr1.0 then saved mail, abook, and bookmarks only, then deleted all Mozilla/Thunderbird profiles, then cleaned up Mozilla/Firefox/Thunderbird in the registry, then reinstalled Firefox .9.3 and Thunderbird .7.3 . So far everything is fine.
Comment 12•20 years ago
|
||
My concern is that I will lose some mail that is real mail, but that Thunderbird might think is junk, such as newsletters, plus whatever bug this is may affect other parts of mail handling, if not now then in the future. I haven't run Mozilla mail or Thunderbird previous to installing Thunderbird a few days ago. I did have a previous version of Firefox installed. I assume, perhaps incorrectly, that there is a way to tell Thunderbird don't check for spam, let everything thru to the inbox, but I can't deduce this from the Junk Mail Controls settings. I was thinking turning off the adaptive filter might do this, but I caught thunderbird losing mail even with that done (a spam email was on the server but nowhere on my system.)
Comment 13•20 years ago
|
||
(In reply to comment #12) In reference to to the above comment, I have turned off Junk Mail Controls and all Junk mail now hits my Inbox, and I delete each one manually. Unfortunately, I have just had the experience (in the last few minutes) of deleting an unread spam message from my Inbox (not marked as Junk, btw), then going to the Trash folder to check the contents of that message (was it really spam?), and finding it gone. WTF? I am using a nightly build from 2/3 days ago.
Assignee | ||
Comment 14•20 years ago
|
||
Is the message really in the Trash folder, but just not showing up? Is the Trash folder threaded or sorted flat (e.g., by date)? If you delete the .msf file, and then look in the Trash, does the message appear? Or is it in the trash but marked as deleted with an x-mozilla-status line of 0008 or 9? I can't reproduce this problem myself?
Comment 15•20 years ago
|
||
(In reply to comment #14) > Is the message really in the Trash folder, but just not showing up? Is the Trash > folder threaded or sorted flat (e.g., by date)? If you delete the .msf file, and > then look in the Trash, does the message appear? Or is it in the trash but > marked as deleted with an x-mozilla-status line of 0008 or 9? I can't reproduce > this problem myself? I did a WinHex (Hex Editor) check on files Trash (10Kb) and Trash.msf (2Kb). the .msf file does not reference the missing email, but I found the missing email in the Trash file. Inbox and Trash Folders are both sorted by date, but one is ascending, one descending.
Assignee | ||
Comment 16•20 years ago
|
||
what does the mozilla status line of the missing e-mail in the trash look like? If you shutdown tbird, delete the .msf file, launch tbird, and click on the trash, does the message reappear?
Comment 17•20 years ago
|
||
(In reply to comment #16) > what does the mozilla status line of the missing e-mail in the trash look like? > If you shutdown tbird, delete the .msf file, launch tbird, and click on the > trash, does the message reappear? I shut down Mozilla completely (not using TB, using Moz mailnews) and deleted the msf file, restarted Moz, and voila: the missing email (status unread) is back in the Trash! [It's important to tell people to get right out of Moz when doing this, including the quick launch memory resident module, or else the msf file gets instantly recreated, and you can even get into a loop in M$ Explorer Search function doing this, as happened to me last time I tried this.] Do you still need the status line from the Trash file? I got his from the Trash file: X-Mozilla-Status: 0200 X-Mozilla-Status2: 00000000
Comment 18•20 years ago
|
||
David, I have stopped doing Empty Trash after checking the messages are actually spam, just marking the Trash folder as Read and I have not had any disappearing mail since (although I have had the count not updated, as per the last paragraph of comment 5). It seems as though the problem only manifests itself if the "junk" folder is empty - at least for me using Trash as the "junk" folder. If you can point me to the general area of the code where this is handled I can spin a debug build and see if I can work out what is happening.
Assignee | ||
Comment 19•20 years ago
|
||
Parish, my guess is that somehow your Trash .msf file is getting its timestamp out of sync with the Trash folder on disk. If we hit this line of code, that means we had a db with a bad timestamp: http://lxr.mozilla.org/mozilla/source/mailnews/db/msgdb/src/nsMsgDatabase.cpp#1116 I've tried forcing this (by Touch'ing the Trash folder) but haven't seen any problems. I've also tried emptying trash w/o a problem. Is you local mail directory in the standard profile dir?
Comment 20•20 years ago
|
||
(In reply to comment #19) I was wondering about timestamps, but couldn't see how it could be assuming that Trash is updated (on disk) and then the summary file is updated. I'll build a debug version and see what is happening around that part of the code since I can reproduce the problem. > Is you local mail directory in the standard profile dir? Yes.
Assignee | ||
Comment 21•20 years ago
|
||
Here's where we add the hdr to the db: http://lxr.mozilla.org/mozilla/source/mailnews/db/msgdb/src/nsMsgDatabase.cpp#2935 all evidence is that we're not getting here. When running a debug build, you should look at the console window and see if anything interesting gets dumped there. Do you have it set so that when you delete or move a message locally from the inbox, it's removed from the pop3 server? Does this bug happen when you delete messages from folders other than the inbox?
Comment 22•20 years ago
|
||
I did Empty Trash, then Get New Messages; it got one message which was marked as Junk (automatically) and moved to Trash - it didn't appear when I looked in Trash. This is the console output generated for getting that message: Begin mail message delivery. Abort mail message delivery. Begin mail message delivery. GetDiskSpaceAvailable returned: -1371414528 bytes Incorporate message begin: uidl string: <1095800990-21088@excite.com> Incorporate message complete. End mail message delivery. WARNING: NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) failed, file c:/mozilla_src/mozilla/mailnews/db/msgdb/src/nsMsgDatabase.cpp, line 160 Begin mail message delivery. Abort mail message delivery. Begin mail message delivery. Abort mail message delivery. Without doing Empty Trash I waited until I was notified of new nessages, did Get New Messages which got two messages, both of which were automatically marked as junk and moved to Trash. These two DO appear when I look in Trash but the previous message still does not appear - opening Trash in an editor confirms that all 3 messages are there. Here is the console output for getting these two message: Begin mail message delivery. GetDiskSpaceAvailable returned: -1371426816 bytes Incorporate message begin: uidl string: <1095801996-28240@excite.com> Incorporate message complete. Incorporate message begin: uidl string: <wkv4_mhn4-1823697035-2697354367-78-47049.7262474792@jimenez6> Incorporate message complete. End mail message delivery. Begin mail message delivery. Abort mail message delivery. Begin mail message delivery. Abort mail message delivery. Notice that NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) doesn't fail at line 160 this time. I have reproduced the above scenario several times and each time NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) only fails after I've done Empty Trash. Does this tell you anything? Ref comment 21, it does get here, but only when Trash is not empty AFAICT. BTW, LXR is out of date; the two line numbers you mention are 1113 and 2932 in the file pulled by cvs.
Assignee | ||
Comment 23•20 years ago
|
||
yes, it tells me that after the empty trash, there was an error opening the message db, presumably for the trash: WARNING: NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) failed, file c:/mozilla_src/mozilla/mailnews/db/msgdb/src/nsMsgDatabase.cpp, line 160 I see the same warning, but the message ends up visible in the trash...I'll try to think why it might be different for you.
Comment 24•20 years ago
|
||
Just checked something about timestamps. After it has moved messages to Trash, Trash and Trash.msf have the same modified timestamp (to the nearest second) but after doing Empty Trash then Trash.msf has a modification time 3 seconds later than Trash. If the code is checking for Trash being later than Trash.msf _before_ messages have been moved to Trash then it will get it wrong and assume that the .msf file is up to date. Is this likely?
Assignee | ||
Comment 25•20 years ago
|
||
it doesn't compare the file timestamps; it compares a time stamp stored *in* the .msf file vs the file time stamp of the folder. But emptying the trash should create a new, empty, valid msf file, with the correct timestamp stored in it. I'll try to figure out why it doesn't...
Comment 26•20 years ago
|
||
Re #24 Peeps, don't you have the ability to lock the "update data file update msf file" operation, so you can be sure the files are synchronized and not have to check time stamps? I have no idea what language you're writing in...
Assignee | ||
Comment 27•20 years ago
|
||
*** Bug 259149 has been marked as a duplicate of this bug. ***
Comment 28•20 years ago
|
||
This happens to me on the 0.8 milestone. I'm using global inbox with three mail accounts. When I receive new mail and decide to delete it with or without reading, the first message disappears into the global thrash folder but the next messages will show up in the folder. Same thing happens with mail automatically marked as junk and moved into the junk folder, the first one will dissapear and the rest will be visible. If I then delete all messages from junk and thrash folders and compact both folders and then go into my profile directory and delete the junk.msf and thrash.msf files for global inbox the lost messages will reappear in the appropriate folders.
Assignee | ||
Comment 29•20 years ago
|
||
re timestamps, the issue is other programs altering our mail store, while we're running or not running...though they cause a lot more grief than they're probably worth... The warning I'm seeing happens when we empty the trash, but we create the trash.msf immediately after, and then it's valid. After you empty the trash, is the Trash.msf file 0 bytes, or 2k (1207 bytes, actually)?
Comment 30•20 years ago
|
||
I posted already on mz, I've also send my two trash files to you David (hope you got them)... I came across another interesting behaviour... After I moved my mails to trash (with 'del' key), the disappeared message reappeared - the reappeared message had the same subject as the just deleted messages (generated mail from bugzilla). Could be related to the threading bug - Delete bug in thread view; #231846? My Trash.msf file has 1.220 bytes after I empty the trash.
Assignee | ||
Comment 31•20 years ago
|
||
I've reproduced this a couple times by changing the timestamp of the trash folder with "touch" after emptying the trash, but even that's not 100% reliable. select the trash folder empty trash touch Trash select another folder and delete a message. The db doesn't get updated, but the Trash folder does. I believe the problem is something like this, however: 1. Trash.msf gets out of date (not sure how that happens, but it seems that empty trash can leave it out of date for some users/scenarios) 2. We move a message into the trash. 3. That causes us to get the db for the trash. Since it's out of date, we get a new db and (perhaps) start to reparse the trash folder. 4. In the meantime, the move finishes, we get the empty db, and we mark the new db as valid, before the reparse finishes. So it's valid w/o any headers.
Status: NEW → ASSIGNED
Comment 32•20 years ago
|
||
Re #31: I'm not sure how many processes(threads, whatever) are involved here, but why is step 3 starting a reparse of the trash file before the move of the message into the trash file has completed? (Or is the data file being parsed in what we oldies used to call core and by move you mean write it to disk?) Why is a reparse of the entire data file necessary on a move in order to update the db(index file?) instead of just inserting the new data in the db? Why is the db being marked valid if you're depending on timestamps to determine validity? Having two mechanisms to indicate validity would seem to be a source of problems. Is it because (not sure if this is possible) some process can start reading the db while you're still modifying it? Why in step 4 is that process marking the db as valid when it has no idea if the db has completed updating?
Comment 33•20 years ago
|
||
Trying to figure this out some more. Is this a buffer management problem, where you need flags for a buffer for valid, read in progress, write in progress, etc. and need to manage the id of the buffer so it is recognizable even while it is being worked on, and a process that wants it has to queue up if there is another process working on the buffer or a read in progress?
Assignee | ||
Comment 34•20 years ago
|
||
If we change the mail folder (e.g., add a message to it), we have to mark the corresponding db as valid, or the time stamps won't match. Obviously, we should never have a db that was previously invalid, make a change to the folder, and then mark the invalid db as valid. But I suspect that's what's happening. It's probably the case that the folder object is caching a pointer to the open db, and the db becomes invalid, and we don't realize that cached db pointer points to a no longer valid db. How the db is becoming invalid in the first place is a mystery to me... Also, I think my previous theory about the reparsing is probably wrong - I can't catch it doing that.
Comment 35•20 years ago
|
||
This doesn't affect just spam emails. check this bug for other details and other related bugs: "message count and file size is wrong for folders on pop3 account" https://bugzilla.mozilla.org/show_bug.cgi?id=259185
Updated•20 years ago
|
Flags: blocking-aviary1.0?
Assignee | ||
Comment 36•20 years ago
|
||
do any of you have virus checkers running that might be holding the trash folder open or modifying it?
Comment 37•20 years ago
|
||
(In reply to comment #36) > do any of you have virus checkers running that might be holding the trash folder > open or modifying it? FWIW I duplicated the problem on a test PC with a fresh image of XP, no virus checker installed, and in fact no other software running.
Comment 38•20 years ago
|
||
I also don't have a virus checker and I'm running win2k (SP4 + latest "fixes").
Assignee | ||
Comment 39•20 years ago
|
||
do you have empty trash on exit set? I'm seeing odd things if I empty trash and then exit, or turn on empty trash on exit.
Comment 40•20 years ago
|
||
I know your question wasn't addessed to me, but I just checked and I don't have empty trash on exit set.
Comment 41•20 years ago
|
||
Hum, I don't use the "empty trash on exit" option. However, I've emptied the trash, then went to my inbox, deleted one mail and it didn't show up in trash - but I can't reproduce this. Update, I've reproduced the bug: 1. compact the trash folder and empty it 2. restart tb 3. delete a message (like mentioned before, to get back the disappearred messages, go to your profile and delete the Trash.msf file)
Assignee | ||
Comment 42•20 years ago
|
||
please try tomorrow's trunk or aviary 1.0 branch build - I've checked in a fix for bug 260981, which I suspect is the same bug, or at least has the same fix.
Comment 43•20 years ago
|
||
Just spun a build with this patch in and it fixes the problem :-) Thanks David. BTW, for anyone else testing this, it may not work the first time if Trash has an "invisible" message in already, but do Empty Trash and it wil then work (obvious really I guess).
Comment 44•20 years ago
|
||
I can confirm that the problem with deleting is fixed in the latest nightly. However, weird things can happen (maybe because of the old Trash.msf file?). Will there be a bugfix release? Anyway, thanks for fixing this David!
Assignee | ||
Comment 45•20 years ago
|
||
I think Scott spins up weekly .8 builds, and of course, there are the nightly tbird .8 builds, as well as the nightly trunk builds - and this fix will be in 1.8a4, for seamonkey users. marking dup of 260981 as the cause and fix seem to be the same. *** This bug has been marked as a duplicate of 260981 ***
Status: ASSIGNED → RESOLVED
Closed: 20 years ago
Resolution: --- → DUPLICATE
Comment 46•20 years ago
|
||
> *** This bug has been marked as a duplicate of 260981 *** Bug 260216 is also a dup, AFAICT. As reporter of that bug, I'd say it is.
Comment 47•20 years ago
|
||
*** Bug 260216 has been marked as a duplicate of this bug. ***
Comment 48•20 years ago
|
||
*** Bug 260370 has been marked as a duplicate of this bug. ***
Comment 49•20 years ago
|
||
(In reply to comment #16) > what does the mozilla status line of the missing e-mail in the trash look like? > If you shutdown tbird, delete the .msf file, launch tbird, and click on the > trash, does the message reappear? Not for me (at least a few days ago).
Comment 50•20 years ago
|
||
(In reply to comment #37) > (In reply to comment #36) > > do any of you have virus checkers running that might be holding the trash folder > > open or modifying it? > FWIW I duplicated the problem on a test PC with a fresh image of XP, no virus > checker installed, and in fact no other software running. > I have Symantec Antivirus 9.0.0.338, but don't know what it is holding open or not.
Comment 51•20 years ago
|
||
(In reply to comment #41) > Hum, I don't use the "empty trash on exit" option. However, I've emptied the > trash, then went to my inbox, deleted one mail and it didn't show up in trash - > but I can't reproduce this. > > Update, I've reproduced the bug: > 1. compact the trash folder and empty it > 2. restart tb > 3. delete a message > > (like mentioned before, to get back the disappearred messages, go to your > profile and delete the Trash.msf file) Like mentioned before, that didn't work for me.
Updated•20 years ago
|
Flags: blocking-aviary1.0?
You need to log in
before you can comment on or make changes to this bug.
Description
•