272988 - IMAP rfc822.size isn't parsed correctly with Dovecot

Status: RESOLVED INVALID

(MailNews Core :: Networking: IMAP, defect)

Description

[Timo Sirainen]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/125.5.5 (KHTML, like Gecko) Safari/125.11
Build Identifier: 

Dovecot 1.0-tests nowadays return BODY[..] fields before other requested fields, which causes 
Thunderbird to parse them somehow wrong. The fetched message size is given to next message 
instead of the current one, causing size-column to contain wrong message sizes and issuing broken 
fetch commands (bug #263904).


Reproducible: Always
Steps to Reproduce:

Comment 1

[David :Bienvenu]

can you get me access to such a server so I can test this?

Comment 2

[Timo Sirainen]

dovecot.org, username anonymous, empty password.

Comment 3

[Nicolas Stransky]

I confirm the bug is still present in thunderbird 1.0.

Comment 4

[David :Bienvenu]

Attachment: proposed fix

I've added some state to the parser to tell if we're received the headers first
or the rfc822 size, when downloading headers. If we receive the size second,
we'll send the NormalMessageEndDownload after we've parsed the size, so we'll
set the right size on the msg hdr.

This seems to work better with the dovecot test server, but I ran into some
problems that look like index corruption on the server, such that fetching a
message gives me the end of one message and the start of the next message.
Despite the fact that this particular size handling bug was a client bug, I
think the other problems I was seeing are a server problem, and it would be
great if someone else could verify that...

Comment 5

[Timo Sirainen]

It's quite possible that the server is giving buggy answers. I know it has
several problems but I haven't had time to look at them. (read-only mboxes are a
bit troublesome especially if they contain broken headers)

Comment 6

[Michael Goerz]

In the new TB1.5b1 this bug is a major problem as it screws up reading mails
completely.
See http://forums.mozillazine.org/viewtopic.php?t=316327

Comment 7

[David :Bienvenu]

Have you tried today's build? I checked in a fix for the kerio server yesterday
that might help with dovecot.

Comment 8

[Michael Goerz]

(In reply to comment #7)
> Have you tried today's build? I checked in a fix for the kerio server yesterday
> that might help with dovecot.

I compiled from the CVS a minute ago, and the problem persists unchanged.

Comment 9

[Timo Sirainen]

Note that nowadays (since last March) Dovecot returns BODY[] fields after others, since Thunderbird 
wasn't the only client having problems with it. So this isn't a problem with Dovecot anymore..

Comment 10

[Isriya Paireepairit]

I can confirm this bug with Kerio MailServer 6.10 [1] too. This happens with Tb
1.5 branch (not in Tb 1.0) on all platform (Win/Mac/Linux).

[1] http://www.kerio.com/kms_home.html 

Comment 11

[David :Bienvenu]

are these newly download headers, or headers downloaded with a previous version
of thunderbird with the bug? If the latter, can you try deleting your .msf files
and redownload the headers?

Comment 12

[Isriya Paireepairit]

Both. I also tested with whole new user profile. I also tested with another free
IMAP server on the web but it worked fine.

Comment 13

[Russell Odom]

I *think* this is the bug I'm getting connecting to my Dovecot server (dovecot-1.0-0_4_stable20050221 running on Fedora 4). The description in the URL given in comment 6 by michaelgoerz pretty much matches what I'm seeing.

This has only affected me since upgrading from 1.0 to 1.5. I've added "imap_client_workarounds = tb-negative-fetch" to my dovecot.conf as per bug 272988 comment 5 (and deleted the .msf file for my inbox) but that made no difference. I have tried connecting with IMAPS and IMAP to the server with the same results; I also tried deleting and recreating the account.

I'm running TB on Windows 2000, so changing OS from "Linux" to "All".

If there's any logs I can capture and post that might help (like someone else produced for bug 272988), please let me know.

I will look at upgrading Dovecot as per comment 9 when I have more time; if that makes a difference I'll post here.

Comment 14

[Russell Odom]

(In reply to comment #13)
> I've added
> "imap_client_workarounds = tb-negative-fetch" to my dovecot.conf as per bug
> 272988 comment 5

Sorry, that should be: bug 263904 comment 5.

> If there's any logs I can capture and post that might help (like someone else
> produced for bug 272988), please let me know.

Again, should read: bug 263904.

Comment 15

[Russell Odom]

I upgraded to Dovecot 1.0 beta 2 (Fedora 4 RPMs from http://fedora.ivazquez.net/yum/4/; I've also filed http://bugzilla.atrpms.net/show_bug.cgi?id=727 for ATrpms to build updated packages), and the problem has now gone away. I didn't even have to delete my .msf files - the correct message contents appeared straight away!

Although the underlying Thunderbird problem presumably still exists, at least it's no longer an issue when using it with Dovecot.

Comment 16

[Nikolay Shopik]

Is this still problem on latest TB?

Comment 17

[Wayne Mery (:wsmwk)]

In looking at an old conference report dated Feb 2014, it was implied that this issue might still be valid. Do you think that might be still true today?

https://mzl.la/3tqO7FV are the imap fixes since Jan 2014 where rfc822.size is mentioned in the bug report.
https://mzl.la/3ea2PdV is a larger list of all bugs that mention rfc822.size sinc that time

Comment 19

[gene smith]

(In reply to Wayne Mery (:wsmwk) from comment #17)

In looking at an old conference report dated Feb 2014, it was implied that this issue might still be valid. Do you think that might be still true today?

I assume you are referring to where someone in the comments down below referenced TB bugs authored by "Timo S." They then referenced this bug as an example. I don't think whoever entered the comment was implying the bug is still valid.

https://mzl.la/3tqO7FV are the imap fixes since Jan 2014 where rfc822.size is mentioned in the bug report.
https://mzl.la/3ea2PdV is a larger list of all bugs that mention rfc822.size sinc that time

Problems regarding rfc822.size have been resolved I think. I noticed patches regarding this while working on the chunking and memory cache issues (probably some from your searches above). So if that's what this fixes I would say it is now N/A and can be closed as INVALID.

Comment 20

[Wayne Mery (:wsmwk)]

My impression as well. Much has changed. Peryaps a couple other open ones https://mzl.la/3svGXyK can be closed?