1670787 - Thunderbird fails to fetch some attachments via IMAP

Status: RESOLVED INCOMPLETE

(MailNews Core :: Networking: IMAP, defect)

Description

[Steve Edmonds]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0

Steps to reproduce:

Click on the attached PDF at the bottom in attachments list to open it or right click "Save As" on attachment.

Actual results:

Opens as empty PDF, saves as zero byte PDF

Expected results:

PDF saved or opened.

Comment 1

[Steve Edmonds]

Connection is TB 82.0b2 via IMAP to a gmail account. No local storage (sync locally off).
This is all PDFs from one supplier, starting from a certain date. All other attachments seem fine. The attachment opens fine in Gmail. I can "Save As" the complete email to the HD and open it in Thunderbird, the attachment is stored and opens.

It seems as though the IMAP fetch for the attachment from the mail reading pane is failing or the base64 conversion.
Starting Thunderbird from the terminal results in no messages, Thunderbird provides no messages.

Comment 2

[Steve Edmonds]

Is there a way to log the IMAP conversation from Thunderbird, without using something like wireshark.
Comparing the 2 emails saved locally (working/not working) there is no structural difference, but then both locally saved emails work fine in Thunderbird. It is only trying to load the attachment from the sever via IMAP than has an issue.

Comment 3

[Magnus Melin [:mkmelin] (away until Aug 10)]

Yes, create an imap log: https://wiki.mozilla.org/MailNews:Logging

Comment 4

[Steve Edmonds]

I just added the account into another client (Kmail) setup IMAP only and the attachments not loading from Thunderbird load fine in Kmail.

Comment 5

[Steve Edmonds]

I just created a new profile in TB and the attachments open fine that don't open in the other profile (new last week).
Logging the IMAP conversation I see in both cases the attachments are fetched line by line.
I will try logging another module to see if I can track why the attachments will not open or save.

Comment 6

[Steve Edmonds]

The PDFs in question are financial account information so I am not too keen to share the contents in bugzilla.
I have created a MIME log and with the PDFs in question the logging terminates.

As I mentioned the PDFs show in one new profile but not in another new profile (empty file).
In both cases logging the MIME module terminates without completion of the log with the problem PDFs. File attached showing the termination point, first a log of a successful decode, then the failed decode.

Comment 7

[Steve Edmonds]

Attachment: TB-attachmentFailsToOpen-MIMElog.txt

Part of the log of a successful attachment open and a failed attachment open.

Comment 8

[Steve Edmonds]

Please advise the module name if logging another module is needed and should the other protocol be logged concurrent with the MIME log to determine why the MIME logging terminates.

Comment 9

[gene smith]

Sounds a lot like this: bug 1673093 depending on how big the pdf attachment is. >=25M is a problem.
Try setting mail.server.default.mime_parts_on_demand to true and restart tb.
If that doesn't help an IMAP:5 log will be needed. You can edit out anything private. Probably pdf in log is unreadable since base64 encoded in separate lines. You can delete a few random lines from the log to ensure it's not recoverable.

Comment 10

[Steve Edmonds]

The whole email is 45kb, so may not be this issue. The PDF is 27k
Setting mail.server.default.mime_parts_on_demand to true made no difference.

I created the log, it is 34MB and 278000 lines in the first 20 seconds it takes for Thunderbird to come up and click to open the attachment.
Disabled checking at startup for other accounts, it is now 7.4MB and 62k lines. Looking through it might take a little while to anonymise.
But, if I take the base64 lines for the attachment from the IMAP log, these will decode to the PDF attachment with base64 -d.

There seems no problem with fetching the attachment, Thunderbird just won't decode/open it.
Where does the attachment go on download, RAM, Cache? Although ctl-U shows content and the base64 in that decodes OK with base64 -d.

Comment 11

[Steve Edmonds]

Just to repeat, this bug is in a new profile with 7 IMAP accounts. When I created a new profile with just the 1 account, this bug does not show.
This bug is only presenting on attachments from one sender between 12 August and 3 October. Attachments from the same sender before and after open correctly.
The byte count is the same and there is no diff between the B64s from the IMAP logs of the profile that opens and the profile that doesn't.

Comment 12

[gene smith]

Yes, not the same problem that I thought it might be. I have about 14 accounts for testing and don't see a problem with small PDFs. I mostly run with no offline store so same as you.
With no offline store files, most recently accessed messages are stored in ram cache for quicker re-access. Same for attachments.
If you could provide a log it still might be helpful. Please include these types in the log: IMAP:5,IMAPCache:5,timestamp
Just note in the log where you have removed stuff. Also, in the comment please explain exactly what you were doing while recording the log.

Comment 13

[Wayne Mery (:wsmwk)]

(In reply to gene smith from comment #12)

Yes, not the same problem that I thought it might be. I have about 14 accounts for testing and don't see a problem with small PDFs. I mostly run with no offline store so same as you.
With no offline store files, most recently accessed messages are stored in ram cache for quicker re-access. Same for attachments.
If you could provide a log it still might be helpful. Please include these types in the log: IMAP:5,IMAPCache:5,timestamp
Just note in the log where you have removed stuff. Also, in the comment please explain exactly what you were doing while recording the log.

Steve?

Comment 14

[Wayne Mery (:wsmwk)]

Steve? Please see comment 12

Comment 15

[Phoenix]

Resolved per whiteboard