Closed Bug 262408 Opened 21 years ago Closed 17 years ago

POP3 mail duplicates with Leave Messages on Server - Only on one account from 3!

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: stefe, Assigned: mscott)

References

Details

(Whiteboard: closeme 2008-05-21)

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10 Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10 I have three POP3 accounts, two of which have their own folders. One of these is a gmail - accessible using FreePOPs, the other is a squirrelmail, accessible directly. I also use POPFile for Spam classification (using headers, not the subject line). Both are configured to leave the mail on the server, and never to delete. The gmail is working fine, but hte squirrelmail, which is my default account tends to download the emails left on the server... which results in duplicates. tried copying account settings to gmail - the problem does not arise, with or with POPfile configured (i need to leave FreePops for gmail to work) Reproducible: Always Steps to Reproduce: 1. 2. 3. default theme used. extensions: QuickReply, ClearSearch, External Appl, Chromedit, EM Buttons, Launchy, Mauil Redirect, Minimise totray, Show old Extensions, Open long URL, Shift Delete Controller. None new Also, POPFile v0.22 and FreePOPs
(In reply to comment #0) Forgot to add, this is the 30th September build of TB0.8
I think I have found the problem. Every time the Accounts Setting is viewed, then server settings viewed, then OK clicked ---> Popstate file is re-inistialised, so that TB does not remember which emails it downlaoded. Copying an older version of popstate will solve the problem, until the accounts setting is viewed,
(In reply to comment #2) > I think I have found the problem. > > Every time the Accounts Setting is viewed, then server settings viewed, then OK > clicked > ---> Popstate file is re-inistialised Hm, I tried a dozend times to view (and even edit) the server settings and left with OK. But it never initialised the popstate. Not right after doing this and not after quitting TB.
I am experiencing a similar problem with TB 0.8 (20040913) on WinXp Sp2. Maybe this information will help. There is one pop3 account, the mail server is exim, and i have configured it to leave messages on the server for 3 days. I also have kaspersky AV locally to scan all my incoming pop3 mail. The following extra headers exist in my local copy of the message, compared to the one on the server: X-Account-Key: account1 X-UIDL: 7365077c3bae97c9dd5a894fd45eb636 X-Mozilla-Status: 0203 X-Mozilla-Status2: 00000000 X-Kaspersky: Checked When messages become duplicated (it doesn't happen immediately), they gain the following additional headers in both of the local copies: Status: RO X-Status: X-Keywords: X-UID: 1178
Now that i look at it closer, the messages that are duplicated in my inbox are all older than 3 days. They are still on the server though, so it seems that they are being duplicated instead of being deleted. Maybe its an unrelated bug after all?
Albeit I have an older version of Thunderbird, but I recently observed the same problem when I procured an e-mail account with GCI. Tried to set it up where my wife could access the same e-mail account sharing a static POP account with me. Theoretically, we should each be able to log in to Windows XP, open Thunderbird, I view similarly populated Inboxes. No luck. Even though I specify that no downloads are to occur, they do anyhow, each of us having independent mail.gci.net folders with differently populated databases. It now explains why folks have stated that they have sent a message and I have not seen them.
Confirming here on version 0.9+ (20041111). I have no idea this is related but I think I saw an increase of duplicate msgs when there junk control was enabled for that account (when a move is done to a jun k folder it "forgets" the old mails maybe?
I too suffer from this problem - but until now I thought it was the server. I have 3 mail accounts in Thunderbird - two from my ISP and one university account. I have all servers set to leave mail on the server until I rmeove it from the inbox. Randomly, Thunderbird will re-download the most recent messages from my university account - only the most recent, not all. This happened in versions 0.8 and 0.9. I don't know about 1.0RC yet.
Thunderbird, version 0.9 (20041103), Windows XP SP2 Some issue for me and others!!! (http://forums.mozillazine.org/viewtopic.php?t=173581) Since TB 0.7 (now using 0.9) I receive duplicate mails. I tried to compress the folders, deleted the popstate and *.msf files. No luck. Strange: from one of my three servers I receive duplicate mail altough TB is supposed to delete the mails after downloading ?!?? Now, the next problem appeared. I cleaned out my mail, about 500. I moved them to trash, emptied the trash folder and compressed the folders (I don't want multiple mails, do I *wink* ). Now TB asks me to compress my folders EVERY time I start TB. Please, this needs to be fixed!!!
How reproducible is this? If it's reproducible, can we get some pop3 protocol logs? Here's how to generate a log: http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap replace "protocol" with POP3 in those instructions. Here's the test scenario: 1. Send yourself mail. 2. Retrieve the mail 3. Shutdown, and copy away the protocol log (it gets overwritten each time you run) 4. Startup again and retrieve mail. If you get the duplicate, copy away the log again, and e-mail me or attach both logs, along with info about what mail was retrieved twice, along with your prefs.js, and perhaps the popstate.dat file for the server which has the problem. thx.
testing).(In reply to comment #10) > How reproducible is this? If it's reproducible, can we get some pop3 protocol > logs? That's the question. While for this situation > Strange: from one of my three servers I receive duplicate mail altough TB is > supposed to delete the mails after downloading ?!?? a log of the retrieve that produces the duplicate isn't the interesting one. The one from the retrieve before is, i.e. the question is why the message isn't deleted.
This happens for me as well. I think it's because I went into my account settings and clicked ok. When I do that, the next time I try to get my emails, it "downloads" like 1000 old messages as duplicates and puts them in my inbox. Even emails I deleted like a year ago. I don't have "leave a copy of message on server" on so my only conclusion is that there's a major bug somewhere here.
I've seen this problem since at least 0.7. It occurrs far less often on 1.0. I'll try and get some useful logs for degugging help.
The bug happens for version 1.0 on Linux also. It happend twice for me. The first case happened when the year changed from 2004 to 2005. I also had deleted some mails from the web interface from yahoo mail. The second time it occured was 2 days ago, when the retrival of new messages timed out. The next time I attempted to retrive new mails, thunderbird decided that all messages should be retrived. Very anoying bug, because I had to wait for all messages to download an delete the duplicated messages uname -a Linux localhost 2.6.8 #2 SMP Tue Sep 14 04:24:14 EEST 2004 i686 unknown unknown GNU/Linux No extensions installed
I have experienced two recurrences of this problem with Tb 1.0 recently. In the first instance, as in comment #14, it occured after the previous retrieval attempt timed out while trying to download a particularly large message. Next time i started Tb, it proceded to retrieve messages i already had. In the second instance, it happened after I had not checked my mail for several days and had a good hundred messages waiting to be downloaded. Tb downloaded about half of them, then paused for a few minutes and started downloading from the first one again, duplicating the messages it had already retrieved. It continued to download all messages without problems after this, ending up with a set of unread messages some of which were duplicated and others not.
Thunderbird 1.0 (20041206) on Linux (Suse 9.2, Gnome 2.6). Mailserver is sendmail on another box. One pop3-account here, checking for new mails once a minute. Using Thunderbird for about three weeks now. Had two major incidents so far, where apparently all messages still left on the server where downloaded again. Between the two major "reloads" two or three smaller ones occurred, where only a smaller amount of messages was downloaded again. Side note: After the first time, I told Thunderbird, in Account Settings -> Server Settings to reduce the time to leave messages on the server from seven days to three days. But today (Feb 18, 2005) my mail spool on the server contains all mails starting from Feb 12, 2005. I can't confirm an immediate connection to changes in the server settings or any other user interaction with Thunderbird. I did compress my mail folders a few times as suggested in the FAQ, but not on a daily basis. According to the server's maillog the incidents appear to be preceded by a state of confusion Thunderbird enters at some point. Usually, I start Thunderbird in the morning and exit it in the evening. The last time Thunderbird reloaded old messages right after I started it on Feb 17. In the server's maillog from Feb 16, I find this: Feb 16 14:32:45 hurz ipop3d[22300]: pop3 service init from XXX.XXX.XXX.XXX Feb 16 14:32:45 hurz ipop3d[22300]: Trying to get mailbox lock from process 22288 Feb 16 14:32:59 hurz ipop3d[22300]: Mailbox is open by another process, access is readonly Feb 16 14:32:59 hurz ipop3d[22300]: Message 1681 UID 12 less than 6918 Feb 16 14:32:59 hurz ipop3d[22300]: Error opening or locking INBOX user=obersche host=heinzel.doctronic.de [XXX.XXX.XXX.XXX] Feb 16 14:32:59 hurz ipop3d[22300]: Auth user=obersche host=heinzel.doctronic.de [XXX.XXX.XXX.XXX] no mailbox Feb 16 14:32:59 hurz ipop3d[22300]: Expunge ignored on readonly mailbox Feb 16 14:32:59 hurz ipop3d[22300]: Logout user=obersche host=heinzel.doctronic.de [XXX.XXX.XXX.XXX] nmsgs=1682 ndele=0 ... Feb 16 14:33:45 hurz ipop3d[22317]: pop3 service init from XXX.XXX.XXX.XXX Feb 16 14:33:45 hurz ipop3d[22317]: Trying to get mailbox lock from process 22288 Feb 16 14:33:59 hurz ipop3d[22317]: Mailbox is open by another process, access is readonly Feb 16 14:33:59 hurz ipop3d[22317]: Message 1681 UID 12 less than 6918 Feb 16 14:33:59 hurz ipop3d[22317]: Error opening or locking INBOX user=obersche host=heinzel.doctronic.de [XXX.XXX.XXX.XXX] Feb 16 14:33:59 hurz ipop3d[22317]: Auth user=obersche host=heinzel.doctronic.de [XXX.XXX.XXX.XXX] no mailbox Feb 16 14:33:59 hurz ipop3d[22317]: Expunge ignored on readonly mailbox Feb 16 14:33:59 hurz ipop3d[22317]: Logout user=obersche host=heinzel.doctronic.de [XXX.XXX.XXX.XXX] nmsgs=1683 ndele=0 and so on for every minute Thunderbird checked for new mail until I quit in the evening. Thunderbird did not report any problem to me and I did receive *some* mails, so I did not notice any problem during the afternoon. But after downloading (way too may) mails in the morning of Feb 17 I discovered a lot of mails I should have received in the afternoon of Feb 16 but didn't. Checking for process ID 22288, which is mentioned as blocking the mailbox, I find this Feb 16 14:31:45 hurz ipop3d[22288]: pop3 service init from XXX.XXX.XXX Feb 16 14:31:45 hurz ipop3d[22288]: Message 1681 UID 12 less than 6918 Feb 16 14:31:45 hurz ipop3d[22288]: Auth user=obersche host=heinzel.doctronic.de [XXX.XXX.XXX] nmsgs=1681/1681 logged for the last mail check before the first message above. All previous maillog entries for that day are ok. Digging further, I find a few maillog messages about a blocked mailbox like above later on Feb 17, too. I remember that Thunderbird hung for a few minutes around that time when I opened a message with an attached XML file (that's another story :^). After that, I quit and restarted Thundebird immediately, and no mails where reloaded. Perhaps it does have something to do with timeouts, as supposed in earlier comments. I'll decrease my mail check frequency from "every minute" to "every 3 minutes" or so an see what happens. Anyway, this is at best a serious inconvenience, at worst it holds the risk of losing mail (if not directly due to buggy pop3 retrieval, then indirectly during manual deletion of duplicate mails). I don't know much about mail protocols and such, so that's all I can report on my own. But if there is anything I can do to assist further debugging, feel free to contact me.
I resolved my issue by changing email hosting providers... Not sure why that worked; both are Linux pop3 hosts.... I haven't had this issue re-occur since doing so.
changing to all based on comments 14 and 16. Is this a product=thunderbird or rather some combination of core+component?
OS: Windows XP → All
Hardware: PC → All
In general, this is a core problem. I suspect the reporter's problem had to do with having two accounts pointing at the same local directory, w/o using the global inbox, but that's just a suspicion. But the more general problem is that sometimes if a download is interrupted, or we crash during download, popstate.dat can get emptied out/corrupted so that we don't know what msgs have already been downloaded. In some bug, Christian talked about a way of dealing with this better. I'll try to find that comment.
(In reply to comment #19) > In general, this is a core problem. I suspect the reporter's problem had to do > with having two accounts pointing at the same local directory, w/o using the > global inbox, but that's just a suspicion. > > But the more general problem is that sometimes if a download is interrupted, or > we crash during download, popstate.dat can get emptied out/corrupted so that we > don't know what msgs have already been downloaded. In some bug, Christian talked > about a way of dealing with this better. I'll try to find that comment. suspect you are thinking of one of these https://bugzilla.mozilla.org/show_bug.cgi?id=240049#c2 and also comments 5 and 7 https://bugzilla.mozilla.org/show_bug.cgi?id=263142#c1
I get this problem usually once or twice a week - I use Thunderbird to manage 8 email accounts and it gets very annoying, especially as when it starts to go wrong, it continually download all the messages again every hour or so. The only way I have found to stop it downloading, is to delete the email off the server. I contacted my ISP, but they advised that it was a known problem with Outlook, and that microsoft had issued a patch, however, I was not using Outlook !
I can confirm this on Thunderbird 1.5 RC2, Build ID: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051201 Thunderbird/1.5 ID:2005120115 I happened on a cleaned profile, mail directed to 'Inbox for this Server's account". It is not shared and no other accounts were active or downloading. Steps to repeat: 1. Set up a clean profile on an account with 25 or more emails to download to the servers inbox and leave on server. 2. Create a filter to catch about half the messages on the server and copy to a newly created, empty folder. 3. Initiate download and close Thunderbird completely during download before it completes downloading. I stopped between 3 and 10 messages. 4. repeat 5-10 times. Result: I had Multiple copies of messages in the inbox and the filter destination folders. Always repeatable for the filtered destination folder, 4 out of 10 tries for the Inbox. Jestre
I can confirm this on Portable Thunderbird 1.5 RC1 (20051025), running across a combination of Win XP and Win ME, while trying to access Gmail directly via Pop3 access. I checked the POPSTATE.DAT file, and it was empty, apart from the standard "do not edit this file" header. I was able to "re-remember" what messages I had downloaded by dragging POPSTATE.DAT out of a backup and copy it over the top of my old (corrupted) one - things then returned to normal. I've had quite a few crashes recently, at least a couple while checking mail - everything I've seen is consistent with Comment #19
I have this same intermittent issue on my current release TB 1.5 (20051201), but it has occurred since I first installed, around TB .8 timeframe. It occurs about once every 3 weeeks but I haven't noticed any correlation with failures or setting updates. I have TB running on two Windows XP machine accessing the same set of POP accounts and the issue occurs on both machines. I have multiple POP accounts, my ISP (Road Runner NYC), ipowerweb, and godaddy email. The issue only occurs on ipowerweb and godaddy, not on the NYC RR. That leads me to believe it has some relationship to the POP server. I'll try to run the trace, but when I follow the instructions regarding set NSPR_LOG_MODULES=protocol:5 and NSPR_LOG_FILE, I get a blank log file? In my case, I start TB with "Thunderbird" not mozilla. Is there any difference in trace generation instructions??
I have this problem intermittently with Thunderbird for both Linux and Mac with two WestHost.com provider accounts.
Concerning comment 25, that bug was reported against a Yahoo mail account. This is odd, since i have the problem with two WestHost.com accounts but not with POP access to my Yahoo account.
sleep909 in comment #24 > I have this same intermittent issue on my current release TB 1.5 (20051201), > but it has occurred since I first installed, around TB .8 timeframe. It occurs > about once every 3 weeeks but I haven't noticed any correlation with failures > or setting updates. I have TB running on two Windows XP machine accessing the > same set of POP accounts and the issue occurs on both machines. I have > multiple POP accounts, my ISP (Road Runner NYC), ipowerweb, and godaddy email. > The issue only occurs on ipowerweb and godaddy, not on the NYC RR. That leads > me to believe it has some relationship to the POP server. I'll try to run the > trace, but when I follow the instructions regarding set > NSPR_LOG_MODULES=protocol:5 and NSPR_LOG_FILE, I get a blank log file? per web page, you must replace "protocol" with "pop3" http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap David in comment #19: > In general, this is a core problem. I suspect the reporter's problem had to do > with having two accounts pointing at the same local directory, w/o using the > global inbox, but that's just a suspicion. is warning given when user attempts to configure in this manner? > But the more general problem is that sometimes if a download is interrupted, or > we crash during download, popstate.dat can get emptied out/corrupted so that we > don't know what msgs have already been downloaded. In some bug, Christian talked > about a way of dealing with this better. I'll try to find that comment. Which bug#? Mark in comment #23: >... > I checked the POPSTATE.DAT file, and it was empty, apart from the standard "do > not edit this file" header. I was able to "re-remember" what messages I had > downloaded by dragging POPSTATE.DAT out of a backup and copy it over the top of > my old (corrupted) one - things then returned to normal. > > I've had quite a few crashes recently, at least a couple while checking mail - > everything I've seen is consistent with Comment #19 Mark does bug 237131 match your problem? jestre, what's your situation?
> is warning given when user attempts to configure in this manner? I thought so..."This directory is already used by an other server. Please pick a different directory." bug 240049 is one of the bugs about the duplicate pop3 messages.
(In reply to comment #28) > > jestre, what's your situation? > Wayne, Works for me on version 2.0.0.0 (20070326) PFF on a HDD, as of today. Still have messages archived on server for 10 days and filters caught and moved many of them but no duplicates were created this time. I will retest with PFF off a thumb drive by next week. Jestre
Also (In reply to comment #28) > > jestre, what's your situation? > Continued from comment #30... Retested with PTB, Portable Thunderbird (Not PFF, typo) on USB drive as well as HDD. Does not fail on a clean shutdown via either the abort or close program buttons. any more, it had previously had failed every on every shutdown during a download. Also a timeout will not cause message duplication if TB is not frozen. However, on either the USB or the HDD it fails every time when downloading during a fault, crash or process termination. I was able to download multiple copies of the same emails each time the download process was interrupted by a non TB UI initiated shutdown, crash or abort. Jestre
QA Contact: general
Comment 30 & 31 seem to indicate a WFM situation. However, there was a fix checked in (bug 352998) for a similar problem, after those comments were posted -- see bug 240049 comment 28 & 29. I guess this bug should be either WFM'd or duped. Bug 207109 seems to be another earlier report of this same symptom, which should get a similar disposition.
Anybody care to speculate whether laptops connected by wireless might be vulnerable to connection issues, resulting in popstate corruption and duplicate mail? One of my friends is experiencing this problem.
we've made several fixes for these kinds of issues in 2.0.0.x - and yes, bad wireless connections in the past could have caused us to redownload pop3 messages.
David, thanks for the amazingly fast reply. I'll certainly recommend my friend to upgrade. Chris
Who else still sees this problem? Chris, is this problem gone now that bug 240049 is fixed?
Whiteboard: closeme 2008-05-21
Last week i hit this problem me again. Mozilla Thunderbird version 2.0.0.9 (20071205)
I stopped using pop3 some time ago, so I no longer have any input on this.
Wayne, I am not aware of this problem arising since my friend upgraded. But it was only very intermittent in the first place, so other environmental issues (such as quality of wireless signal) could also be at play here. Thanks anyway for thinking to ask me specifically.
Haven't seen this in months, using the branch nightlies on Linux.
Jestre and Roma, suggest you file new bugs. Jestre's regarding USB sounds like a duplicate. closing WFM as suggested in comment 33
Status: NEW → RESOLVED
Closed: 17 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.