663761 - keeps redownloading the same messages over IMAP

Status: VERIFIED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Jozsef Fejes]

User-Agent:       Mozilla/5.0 (Windows NT 5.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10

For a long time I've used POP to download my messages. One day I decided to switch to IMAP. I created an IMAP account for the same email address and uploaded all my locally stored messages from the POP account to the IMAP account. Ever since Thunderbird keeps redownloading all my messages. I went as far as deleting all offline mailbox data, but it's still happening. I figure there's something wrong with the headers, so I'll attach a sample. First please tell me what I should do to stop this behavior, then move on to why it happened. I have access to the raw courier files on the server via FTP, so I might even try editing the messages on the server with a script.

Reproducible: Didn't try

Comment 1

[Jozsef Fejes]

Attachment: this is how a sample mail looks like on the server

Comment 2

[Jozsef Fejes]

Attachment: downloaded copy, headers only

I copy-pasted it from the Thunderbird mailbox, so it may not be byte-accurate. You can see three new lines. This may cause the continuous redownloading.

Comment 3

[Ludovic Hirlimann [:Usul]]

can we get an imap log as described at https://wiki.mozilla.org/MailNews:Logging so we can see what's going on ?

Comment 4

[Jozsef Fejes]

Attachment: imap log

When this was made, there were no new messages, I downloaded everything before. I waited until it downloaded around a hundred messages then I quit. I filtered the message bodies and account information, which sadly TB didn't do.

Comment 5

[David :Bienvenu]

Attachment: proposed fix

I noticed this happening to a few of my messages, and the all had a "From " line as the first line returned by the imap server. Technically, the imap server shouldn't do that, but mine does, as does the reporter's, so we should just allow it.

I thought about stripping the "From " off in the imap protocol code, but it's best not to remove any of the message content, even if it's not kosher.

Comment 6

[[e-mail]]

Comment on attachment 544898 [details] [diff] [review]
proposed fix

>+          if (findPos != -1 && (startOfMsg[findPos] == ':' ||
>+              !(strncmp(startOfMsg, "From ", 5))))
Indentation is confusing, it makes it look as if you want to check for "From" when findPos is -1, which would be incorrect. Try

if (findPos != -1 && (startOfMsg[findPos] == ':' ||
                      !(strncmp(startOfMsg, "From ", 5))))

Or

if (findPos != -1 &&
    (startOfMsg[findPos] == ':' || !(strncmp(startOfMsg, "From ", 5))))

Or

if (findPos != -1 &&
    (startOfMsg[findPos] == ':' ||
     !(strncmp(startOfMsg, "From ", 5))))

Comment 7

[David :Bienvenu]

fixed in trunk, http://hg.mozilla.org/comm-central/rev/e6a7df2f0c20, with line reformatted to the first option.

Comment 8

[David :Bienvenu]

wouldn't hurt to consider this for tb 7 - it's a simple fix.

Comment 9

[Jozsef Fejes]

I wouldn't mind that either :). Anyway, please tell me when and how you decide so that I can test it.

Comment 10

[Mark Banner (:standard8)]

Comment on attachment 544898 [details] [diff] [review]
proposed fix

Yep, let's take this for 7.

Comment 11

[Mark Banner (:standard8)]

Comment on attachment 544898 [details] [diff] [review]
proposed fix

Err, get the right flag this time.

Comment 12

[David :Bienvenu]

fixed for TB 7 - http://hg.mozilla.org/releases/comm-aurora/rev/851aad78b9c6

There are still cases where this will happen, when the server returns a really corrupted message, but those cases should be rarer.

Comment 13

[Jozsef Fejes]

It works for me too, thank you.