265739 - IMAP Mail over a certain size (approx 30k?) will not display

Status: RESOLVED INCOMPLETE

(Thunderbird :: Mail Window Front End, defect)

Description

[Luke Jagger]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1
Build Identifier: Thunderbird 0.8

This has been happening for a while, and I couldn't pin down under which
circumstances.  But now I can see that it appears to be related to the size of
the message.  Apologies if this has been reported already, but there seemed to
be so many reports of _similar_ things, that I thought I should at least mention
the possible link to mail size.

Pretty much consistently, if I click on an IMAP message of <30kb, it will
display, and anything over that will not.

Very ocassionally, these mails over 30kb will start to display for no apparent
reason.

If I view its source, it DOES display.   I also see the headers, just not the body.

Reproducible: Always
Steps to Reproduce:
1. Click on an IMAP downloaded mail over 30kb in size

Actual Results:  
Mail will not display in either view pane or when double clicked.

Showing source DOES work.

Headers are visible, just not body.

Expected Results:  
Should display.

Seems to happen in some IMAP folders but not others.
I'm using Mailtraq Free edition for my IMAP server.

Comment 1

[Luke Jagger]

Additionally, if I mark a folder to download for offline use, then ensure it
downloads (via a right click on the folder in the left pane,
properties>Offline>Download Now) it starts to show all these mails over 30Kb !!

It therefore seems to be related to whether the mail is fully downloaded locally
or not, and if not AND it's > 30kb it WON'T display.
(This has nothing to do with the 'do not download mail larger than...' setting,
as I played with that one and it made no odds).

Comment 2

[David :Bienvenu]

a client-side imap protocol log would be helpful.

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

Comment 3

[Luke Jagger]

Attachment: Thunderbird IMAP log

IMAP log for just one message > 30kb failing to display when clicked on.
You'll notice there's an error logged towards the end as it tries to display
it.

Comment 4

[David :Bienvenu]

My guess is that your server is returning invalid syntax for the mime structure
of the message. Without getting access to the server, I can't tell for sure,
however...

But, you can configure thunderbird to not use the mime structure of the message.

set the following pref in prefs.js for your imap server.

user_pref(mail.server.serverXX.mime_parts_on_demand, false);

where serverXX is your imap server (you should see the other prefs for the
server in prefs.js). Let me know if this works for you...

Comment 5

[Luke Jagger]

No, that didn't seem to work.
My server is called 'server' (it's in my hosts file as this), so I added
user_pref(mail.server.server.mime_parts_on_demand, false); to prefs.js under
thunderbird\defaults\profile.  Incidentally, there was nothing else in this file
except a comment line.  Restarted TB, and tried to open the very same message,
but it didn't come up.

The only way I've got it to work is via the setting to download all messages
locally.

Here's what Mailtraq logged at this time, IMAP server side:
-------------------------------------------------------------
+ 0000004E 192.168.77.3 [24/10/2004 09:42]
00001000 00000000 24/10/2004 09:42:53 IMAP: (Accept) Receiving connection from
192.168.77.3
00000004 0000004E 24/10/2004 09:42:53 1 authenticate plain  --->  1 OK
AUTHENTICATE completed  
00000004 0000004E 24/10/2004 09:42:53 2 select "INBOX"  --->  2 OK [READ-WRITE]
SELECT completed  
00000004 0000004E 24/10/2004 09:42:53 3 UID fetch 1:* (FLAGS)  --->  3 OK FETCH
completed  
00000004 0000004E 24/10/2004 09:42:54 4 UID fetch 384 (BODYSTRUCTURE)  --->  4
OK FETCH completed  
00000004 0000004E 24/10/2004 09:42:54 5 UID fetch 384 (BODY[HEADER] BODY[1.MIME]
BODY[1.1.MIME] BODY[1.2.MIME] BODY[2.MIME] BODY[3.MIME])  --->  5 OK FETCH
completed  
00000004 0000004E 24/10/2004 09:42:54 6 logout  --->  6 OK LOGOUT completed  
00000004 0000004E 24/10/2004 09:42:54 192.168.77.3 is disconnecting
- 0000004E
-------------------------------------------------------------

Hope that helps.  I'm not an IMAP expert, but isn't this still getting the MIME
structure, even with the prefs.js change?  Doesn't seem to be any error reported
though either.

Comment 6

[David :Bienvenu]

my bad, it should be:

user_pref("mail.server.server.mime_parts_on_demand", false);

note the quotes...

you could also add the global setting:

user_pref("mail.imap.mime_parts_on_demand", false);

Comment 7

[Luke Jagger]

Ok, tried those too, but they don't solve it.

Comment 8

[David :Bienvenu]

can you attach or e-mail me your prefs.js that doesn't work? If you've set the
right prefs, it will work...

Comment 9

[Luke Jagger]

Attachment: C:\Program Files\Mozilla Thunderbird\defaults\profile\prefs.js

The prefs.js as edited under C:\Program Files\Mozilla
Thunderbird\defaults\profile

Comment 10

[Sturat]

This bug is still happening in version 1.5.0.2 - I posted about it in the support forum (probably the wrong place, i know!)

See here: http://forums.mozillazine.org/viewtopic.php?p=2216855#2216855

Thanks

Comment 11

[Wayne Mery (:wsmwk)]

(In reply to comment #10)
> This bug is still happening in version 1.5.0.2 - I posted about it in the
> See here: http://forums.mozillazine.org/viewtopic.php?p=2216855#2216855

from that thread...

IMAP conversation...

Code:
7 UID fetch 63980 (UID RFC822.SIZE BODY[]<0.10240>)
* 56 FETCH (UID 63980 RFC822.SIZE 15947 BODY[]<0.10240> {10241}
... [lots of email text]
Y2ltZW4gd2hlcmUgdGhlIHNocmlua2FnZSBzdHJlc3MgaXMgZGV0ZXJtaW5lZCBieSB0aGUgdGVz
dGluZyBmYWNpbGl0eSBhbmQgdGhlIHNwZWNpbWVuIGRpbWVuc2lvbnMuIFR3byBleGFtcGxlcyB3
ZXJlIGRpc2N1c3NlZCB0byBpbGx1c3RyYXRlIHRoZSB0ZWNobmlxdWUuIFRoZSBmaXJzdCBleGF)
7 OK FETCH completed

So Thunderbird has successfully retrieved the start of the email. The imap logs say 'CreateNewLineFromSocket' before each line, which I'm assuming means it is adding the data to an internal buffer.

Next, Thunderbird requests more data. Here, the server does not reply properly and gives no data. Then, Thunderbird requests the entire email without specifying the size to give back.

This successfully retrieves the entire email. However, what I am seeing in the email source (see my previous post) shows that the text is being APPENDED to the same buffer the first lot of data was. This results in the display problems I mentioned. 

Comment 12

[Bill Janssen]

From Luke Jagger's example, I believe this is essentially the same bug as 149771.  Wayne Mery's trace indicates that there may be another bug involved, as well.

Comment 13

[Wayne Mery (:wsmwk)]

Bill, thanks for the note.

Comment 14

[Lukasz Stelmach]

I have faced this problem few days ago. What I have alredy observed is that CRs and LFs are interpreded differently when TB retrieves an entire message (<30000 bytes) and when it checks out only its structure  (>=30000 bytes).

Some messages I receive have only LF and some CR-LF as end-of-line. Courier IMAP has a bad habit, though the authors assert it is RFC compliant, of adding CR before every LF. Which in the end gives CR-LF for LF-only messages and CR-CR-LF for CR-LF ones.

When TB retreives big (>=30kB) messages it can't interpret CR-CR-LF sequences properly. At least that is what it looks like after some testing, tcpdumping and fromdosing (fromdos(1)).

I use TB 2.0.0.12 for Windows.

Comment 15

[Wayne Mery (:wsmwk)]

David, thoughts on comment 9, 14?

Comment 16

[David :Bienvenu]

In the original report, we had an error parsing the body structure response from the server: 

932[2452fa8]: 24afc18:server:S-INBOX:CreateNewLineFromSocket:  BODY[1.MIME] (("text" "plain" ("charset" "ISO-8859-1" "format" "flowed") NIL NIL "7bit" 18 2 NIL NIL NIL) ("text" "html" ("charset" "ISO-8859-1") NIL NIL "7bit" 276 11 NIL NIL NIL) "alternative" ("boundary" "------------090007090102070507090808") NIL NIL) BODY[1.1.MIME] ("text" "plain" ("charset" "ISO-8859-1" "format" "flowed") NIL NIL "7bit" 18 2 NIL NIL NIL) BODY[1.2.MIME] ("text" "html" ("char


932[2452fa8]: 24afc18:server:S-INBOX:PARSER:Internal Syntax Error: %s:  BODY[1.MIME] (("text" "plain" ("charset" "ISO-8859-1" "format" "flowed") NIL NIL "7bit" 18 2 NIL NIL NIL) ("text" "html" ("charset" "ISO-8859-1") NIL NIL "7bit" 276 11 NIL NIL NIL) "alternative" ("boundary" "------------090007090102070507090808") NIL NIL) BODY[1.1.MIME] ("text" "plain" ("charset" "ISO-8859-1" "format" "flowed") NIL NIL "7bit" 18 2 NIL NIL NIL) BODY[1.2.MIME] ("text" "html" ("char

I don't know of other reports where we fail to parse the body structure, which makes me think it's something specific to the server, or the particular messages the user is getting.

Comment 17

[Wayne Mery (:wsmwk)]

  Lukasz in comment #14
> I have faced this problem few days ago. What I have alredy observed is that CRs
> and LFs are interpreded differently when TB retrieves an entire message (<30000
> bytes) and when it checks out only its structure  (>=30000 bytes).

  Lukasz, your issue is not bug 149771?

Comment 18

[Wayne Mery (:wsmwk)]

so we need a testcase. 
is this still happening for anyone?
 (especially if you are running version3)

Comment 19

[Joshua Cranmer [:jcranmer]]

RESOLVED INCOMPLETE due to lack of response to last question. If you feel this change was made in error, please respond to this bug with your reasons why.