434054 - IMAP PDF attachments corrupted on download, affected by mime_parts_on_demand

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: IMAP, defect)

Description

[Jason Merrill]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.14) Gecko/20080416 Fedora/2.0.0.14-1.fc8 Firefox/2.0.0.14
Build Identifier: thunderbird-2.0.0.14-1.fc8

Recently I've had trouble with Thunderbird being unable to view or save certain PDF attachments from the IMAP server (Cyrus I think, not sure).  One was sent from Thunderbird 2.0.0.12 under Windows, one was sent by Apple Mail.  In both cases, right clicking the attachment and selecting "Save As" creates the same incomprehensible 27 byte file.

I am able to download the attachments successfully using a webmail interface.  

This doesn't happen for all .pdf attachments.  Furthermore, mailing myself the second of the failing attachments via Thunderbird (Linux) or Gmail (Firefox, Linux or Windows) does not reproduce the problem; saving the attachments that I sent to myself works fine.

od -x of the 27 byte file:

0000000 184e 6eac 7287 aaa5 c2ed 6529 e76d c268
0000020 6879 d769 a29d 5e77 a999 00dd


Reproducible: Always

Steps to Reproduce:
1. Select the affected message in my Inbox
2. Right-click the .pdf attachment
3 [review]. Select Open or Save As or Save All
Actual Results:  
I get this odd 27 byte file instead of the attachment

Expected Results:  
The real attachment

Comment 1

[Jason Merrill]

The thread at http://forums.mozillazine.org/viewtopic.php?p=3293233 seemed like possibly the same bug, but the suggested fix of setting mail.server.default.fetch_by_chunks to False didn't work for me.

Comment 2

[Wolfgang Rosenauer [:wolfiR]]

Not quite sure if that is really the same issue since the pref workaround fixed that problem:
https://bugzilla.novell.com/show_bug.cgi?id=392876

Comment 3

[Ludovic Hirlimann [:Usul]]

Jason does thunderbird 3.0 beta 2 still show the issue ? If so could you log imap (as explained at https://wiki.mozilla.org/MailNews:Logging) and attach the log here ?

Comment 4

[JN]

Attachment: ran thunderbird with normal settings in Terminal Service

Comment 5

[JN]

Thunderbird 3 Release Candidate 2

log.bat
==>>
set NSPR_LOG_MODULES=imap:5
set NSPR_LOG_FILE=u:\imap.log
"C:\Program Files (x86)\Mozilla Thunderbird\thunderbird.exe"

started program, 
clicked on corrupted message, 
downloaded attachment, 
closed program

Comment 6

[Phil Lacy]

of note: the second fetch returned the correct number of octets

11376[6628d00]: 65d7400:hmail:S-INBOX:CreateNewLineFromSocket: * 163 FETCH (UID 233 BODY[2] {196135}

part 1.1 has issues. Since the body[1.1.mime] is blank and there is a mime data in the body[1.1] then there must be a blank line after the boundary.

can you post the offending message source without personal information and you can remove the body part of the pdf file to reduce the upload file size. Just try to leave in first and last lines of body parts.

that way we can make up a similiar test message.

Comment 7

[asudbury]

I've noticed the same problem.  I use Thunderbird with IMAP from GoDaddy mail client ( https://login.secureserver.net )

Documents download fine with webmail interface but on Thunderbird they come out corrupted.

Comment 8

[Wayne Mery (:wsmwk)]

This is not exchange, right?
Do any of bugs in the following list look like a good match? 

https://bugzilla.mozilla.org/buglist.cgi?type1-0-0=substring&keywords=dataloss&type0-1-0=nowordssubstr&keywords_type=allwords&short_desc=attachment%20body%20pdf&field0-1-0=component&field0-0-0=short_desc&type0-0-1=substring&field0-0-1=keywords&type1-0-1=allwordssubstr&resolution=---&resolution=FIXED&classification=Client%20Software&classification=Components&chfieldto=Now&query_format=advanced&chfieldfrom=2009-1-01&value0-1-0=compos%20import%20toolbar%20address&short_desc_type=anywordssubstr&type0-0-0=nowordssubstr&value0-0-0=exchange%20tags%20import%20junk%20Detach%20undo&field1-0-0=short_desc&product=MailNews%20Core&product=Thunderbird&field1-0-1=short_desc

Comment 9

[JN]

I've added the following three lines to my configs and have since not experienced any problems with attachments.

// ATTACHMENTFIX
// fix for corrupted attatchments Thunderbird
///////////////////////////////////////////////////////////////////
lockPref("mail.imap.fetch_by_chunks", false);
lockPref("mail.server.default.fetch_by_chunks", false);
lockPref("mail.server.default.mime_parts_on_demand", false);
///////////////////////////////////////////////////////////////////

Comment 10

[tumbes2000]

I have the same issue except what seems to be a determining factor is file size, which are typically PDFs in my case, but also some  larger doc files as well.  Sometimes they download fine, but other times they do not.  I am using Go daddy imap on ubuntu 11.04 64 bit thunderbird 3.1.1.  My email client on my xoom has no problems downloading and displaying the file same with the web client.

Comment 11

[TT]

I'm having the same issue on SeaMonkey 2.0.14 on Linux.  Occasionally, I will save a PDF that can't be opened, even though it looks okay.

Deleting the "ImapMail" folder fixes this problem for me, at least for a while.  Eventually, the problem will start happening again, and I'll need to delete ImapMail again.

Comment 12

[Parish Mills]

I too was having problems with PDF's that were coming in via my Imap server from Godaddy. About half of them were corrupted when I tried to open them through Thunderbird (v 14.0). I had no problems opening the same file through via webmail on my Android phone. This told me it had something to do with Thunderbird.

THE FIX (Special thanks to JN above on Comment 9:

1. In Thunderbird, go to Tools > Advanced > Config Editor > Accept that you might void your warranty :)
2. In the search bar, type "fetch_by_chunks". I had 1 result pop up, "mail.server.default.fetch_by_chunks". The original value was set to true. Double click on this, so that it changes to false.
3. Now clear the search bar.
4. In the search bar, type "mail.server.default.mime_parts_on_demand". The original value was set to true. Double click on this, so that it changes to false.

By changing these 2 values, my PDF's are coming through as they should.

Comment 13

[Wayne Mery (:wsmwk)]

Is there reason to believe this would be gone in a recent release?  
ANd if not, is this a duplicate bug, or do we confirm?

Comment 14

[:Irving Reid (No longer working on Firefox)]

This is more likely related to mime parts on demand than to fetch by chunks; there are quite a few bugs filed about mime part downloading. I don't have a particular preference for any one of them as a master bug for assigning duplicates.

Comment 15

[Wayne Mery (:wsmwk)]

(In reply to Irving Reid (:irving) from comment #14)
> This is more likely related to mime parts on demand than to fetch by chunks;
> there are quite a few bugs filed about mime part downloading. I don't have a
> particular preference for any one of them as a master bug for assigning
> duplicates.

Parish, can you find one from this list?
https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced;list_id=5236062;short_desc=mime%20part;short_desc_type=allwordssubstr;type0-0-0=nowords;value0-0-0=count%20counts;resolution=---;product=MailNews%20Core;product=Thunderbird

Comment 16

[Makoto Itani]

I use Thunderbird 17 on Japanese Windows 7 (64bit) with IMAP.
(Our IMAP server is "DEEPSoft MailSuite".
 http://www.deepsoft.co.jp/ )
Sometimes, I received mails with corrupted attached files.
(not every time with same file.
 When I run "repair folder" command, I can get correct attached file.)

I wrote some comments at Bug 815730.
And, when I set “mail.server.default.fetch_by_chunks” parameter to “false”, I can get correct attached file every time with IMAP.

I got Thunderbird IMAP log when I received a test mail from myself with a corrupted attached file.

https://bugzilla.mozilla.org/attachment.cgi?id=688564
about IMAP log:
 mail.imap.chunk_size: 393216
 mail.server.default.fetch_by_chunks: true
 uncheck the checkbox "keep messages for this account on this computer"

* thunderbird_20121204-02.log
  I received a mail with a corrupted attached file.

* thunderbird_20121204-03.log
  When I restarted Thunderbird once, I got a correct attached file.

I hope this information helps to debug...

Comment 17

[Wayne Mery (:wsmwk)]

so one of these? https://bugzilla.mozilla.org/buglist.cgi?type1-0-0=substring;list_id=5716486;field0-0-0=short_desc;bug_severity=critical;bug_severity=major;bug_severity=normal;type0-0-1=anywordssubstr;field0-0-1=short_desc;resolution=---;query_format=advanced;value1-0-0=mime_parts_on_demand;value0-0-1=mime%20part;type0-0-0=anywordssubstr;value0-0-0=%20down%20load;field1-0-0=longdesc;product=MailNews%20Core;product=Thunderbird

Comment 18

[Wayne Mery (:wsmwk)]

(Parish writes "I am still using Thunderbird with no issues.")

Comment 19

[Wayne Mery (:wsmwk)]

Jason writes "I haven't seen it in years, though my IMAP server has also changed."