User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Dealio Toolbar 3.1.1) Build Identifier: 2008-04-14-03-trunk Please implement the following two preferences, modeled on the behavior of the *nix 'fetchmail' program: Fetchlimit is a number of messages after which, no matter what may be left on the server, the program sends QUIT, tears down the connection, and waits for the "query server every N minutes" clock. Expunge is a number of messages after which the program sends QUIT -- allowing the server to actually delete the messages -- them more-or-less immediately cycles back to re-authorize. That would also be a good time to make sure everything is 'synced' and all the information that's useful has actually been written to disk. BACKGROUND/RATIONALE: Because of workstation down-time, it is quite possible for a POP server to accumulate far more messages than would be expected in a "normal" batch. A number of factors seem to render this situation quite pathological. While I have not yet been able to detect whether the resultant failures are caused by the server (ISP) deliberately "throttling" the amount of information transmitted in a session or in a unit time, or whether the client (T-Bird) somehow becomes overwhelmed by the volume, the scenario winds up in the same horrible place: C: AUTH, etc to initialize the conversation S: +OK C: LIST S: +OK S: 1 1000 S: 2 2000 ... continuing thru S: 15000 2049 S: . C: UIDL S: [I won't show the UID list here as it is not at issue] S: . C: DELE 1 ## T-Bird immediately deletes messages which the UID shows were retrieved previously S: +OK C: RETR 2 S: +OK 2000 octets S: header-line S: header-line S: S: body-line S: body-line S: . [and so forth, until the fatal limit manifests and C: ****** Deems the connection to have timed-out Provided the original retrieval was initiated by the timer, rather than by the user (another bug), after a waiting period specified by preference, the process starts again. Problem! if the backlog gets "high enough" -- and I'm now working through 16000 mails -- the LIST and UIDL transmissions virtually exhaust the limitations on the connection/session, or maybe we can retrieve two or three mails. That's slower than the arrival of more stuff at the server, and the problem grows and grows. Problem! Even a great program like T-Bird can fail. Maybe the power goes out, maybe my wife pushes the wrong switch, maybe an illegal BASE64 message arrives and locks up the computer (still another bug). If this happens, QUIT is never (never, never) sent to the server. All those messages we've been sending DELE'tes against are not removed and we will have to deal with them again (and again, and again). In the worst case, some or all of the previously-retrieved UID's do not get written to disk and the message has to be retrieved again. The fetchmail program offers a mechanism which has successfully enabled be to get past this horrible situation. "Fetchlimit" specifies a number of messages, after which the client sends QUIT, tears down the connection, and takes its timeout -- even though many messages may remain on the server. And "Expunge" specifies a number of messages, after which the client sends QUIT, takes a quick breath, and immediately cycles back to re-authorize. Both these permit the server to actually drop the messages which have been successfully downloaded. Of course, fetchmail doesn't know whether the message has been successfully dealt with locally -- just that the local sendmail has accepted responsibility for it -- but that isn't an issue here, in fact any decent sendmail will either deposit or save anything and not lose the message. After my most-recent week of horror, the only way in which I've been able to drain the swamp (my ISP's pop server) is by setting the "expunge" all the way down to 200! If I try to retrieve much more, the timeouts start biting me. Fetchmail doesn't really cope well with the timeouts, it winds up getting duplicate messages -- not your problem -- but with the lower setting I have actually plunked the whole terrible volume into T-Bird's movemail Inbox. Reproducible: Always Steps to Reproduce: 1. 2. 3.
Oh, RAT! please pardon the wierd line breaks. I composed this in a textfile so I could put it up quickly ('cause I'm at work).
Looks good, and I'm not seeing a duplicate, so confirming.
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Linux → All
Hardware: PC → All
You need to log in before you can comment on or make changes to this bug.