1361422 - Attachment saved as base64 encoded file if attachment is the only part of the message

Status: VERIFIED FIXED

(MailNews Core :: MIME, defect)

Description

[Robby]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0
Build ID: 20170413192749

Steps to reproduce:

1. Update to Thunderbird 52.1.0.
2. Try to open emails with ZIP attachments, or save them as files to disk and then try to open them.


Actual results:

Error from archival program that the ZIP can not be opened due to it being malformed, damaged or corrupted.

Upon further inspection it turns out that Thunderbird 52.1.0 is saving the ZIP files to disk in base64 encoding instead of properly decoding it. I confirmed this by opening the ZIP file in notepad and saw the base64 encoding.


Expected results:

ZIP files should open normally like in previous Thunderbird versions prior to the 52.1.0 update.

Comment 1

[Robby]

After more testing I have narrowed this problem down a bit. As we have SPF/DKIM/DMARC records for our domain name, I have enabled DMARC reports, so I regularly get DMARC reports. It turns out that *only* the reports sent by Google (from [e-mail]) are affected, I have a couple of older reports still in my inbox whose ZIP attachments also do not want to open anymore due to this problem. Other emails who contain other type of attachment continue to work as normal.

Comment 2

[Robby]

After yet some more testing I have found a workaround so that in the meantime I'm still able to open these ZIP attachments. By clicking Forward in the email, a new window opens where the ZIP attachment is visible in the upper right corner. Double-clicking the file over there will in fact open the ZIP file normally.

Comment 3

[Jorg K (CEST = GMT+2)]

This is most likely a regression from bug 1350080, which was uplifted to TB 52.1 (so it's not in TB 52.0.1):
https://hg.mozilla.org/releases/comm-esr52/rev/10a70478cc88e31fec40585d41112cc6eaee04b0

I was considering a "safer" version of the code, see bug 1350080 comment #21, but in the end we didn't deem it necessary since we could not find any example that would require it.

I can actually open a ZIP archive simply attached to a message, so the messages failing for you must have an "unexpected" MIME structure, as you said: *only* the reports sent by Google. Can you please save such a report as .eml file and attach it here. Or zip it up and e-mail it to me privately.

Alfred, can you please take a look.

Comment 4

[Jorg K (CEST = GMT+2)]

I really need a sample to take this further.

Comment 5

[Alfred Peters [:infofrommozilla]]

I found one:
https://lists.inverse.ca/sogo/arc/users/2015-06/msg00327.html

| Content-Type: application/zip;
|  name="google.com!xxxxxx.dk!1435536000!1435622399.zip"
| Content-Disposition: attachment;
|  filename="google.com!xxxxxx.dk!1435536000!1435622399.zip"
| Content-Transfer-Encoding: base64

Your "safer version" from Bug 1350080 Comment 18 works.

Comment 6

[Jorg K (CEST = GMT+2)]

Attachment: Test message

Test message assembled from https://lists.inverse.ca/sogo/arc/users/2015-06/msg00327.html.

Alfred, can you please attach a patch to fix this.

Comment 7

[Alfred Peters [:infofrommozilla]]

Attachment: Bug1361422_v1.patch

Decode (base64/quoted-printable) when saving "application/*" message attachments

As far as I've tested it, forwarded messages are still encoded.

Comment 8

[Alfred Peters [:infofrommozilla]]

(In reply to Alfred Peters from comment #7)

> As far as I've tested it, forwarded messages are still encoded.

Of course. The entire message is forwarded, not just the attachment.

Comment 9

[Robby]

Reading from the above comments, do you still require an EML file from me? Or is that no longer needed?

Comment 10

[Jorg K (CEST = GMT+2)]

Well, a real example rather than a fake one wouldn't go astray ;-)

Comment 11

[Jorg K (CEST = GMT+2)]

Attachment: Bug1361422 (v2).

How about this? Since there are may text types: text/plain, text/html, text/richtext, text/enriched, text/calendar, I'd like to cover them all.

Comment 12

[Robby]

I just emailed you a real sample. :)

Comment 13

[Jorg K (CEST = GMT+2)]

That works now, too. Thanks.

Comment 14

[Alfred Peters [:infofrommozilla]]

Comment on attachment 8864291 [details] [diff] [review]
Bug1361422 (v2).

Does work as well.

Comment 15

[Alfred Peters [:infofrommozilla]]

(In reply to Jorg K (GMT+2) from comment #10)
> Well, a real example rather than a fake one wouldn't go astray ;-)

I had no problem with v1.
But yes, v2 is more flexible.

Comment 16

[Jorg K (CEST = GMT+2)]

https://hg.mozilla.org/comm-central/rev/a4cdf9a9fe666b2b1e045d6fae82406130ea1398

Thanks for you help, Alfred.

Comment 17

[Jorg K (CEST = GMT+2)]

Comment on attachment 8864291 [details] [diff] [review]
Bug1361422 (v2).

[Approval Request Comment]
Regression caused by (bug #): Bug 1350080

Comment 18

[Jorg K (CEST = GMT+2)]

Beta (TB 54):
https://hg.mozilla.org/releases/comm-beta/rev/88a73f0bbf8f659b984270a0fcbb641b510b950f

Comment 21

[Wayne Mery (:wsmwk)]

Robby, Hungerburg, yup,

Is your problem gone, with no ill effects, if you use the current nightly build from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/
thunderbird-55.0a1.en-US.win32.installer.exe windows
thunderbird-55.0a1.en-US.linux-x86_64.tar.bz2 linux

Comment 22

[Hungerburg]

Hello Wayne, I did download the linux executable, installed to and launched it from /opt/, skipped the integration and found myself running "Daily", which opened both my own Test.eml and the (business like rude) offending SAP content-type: application/pdf only message just fine. No ill effects became apparent during this short exercise.

Comment 24

[Jorg K (CEST = GMT+2)]

TB 52 ESR:
https://hg.mozilla.org/releases/comm-esr52/rev/b7187d663ec0830b057a0a863f06df58a0d6b508

Comment 25

[ruga]

We confirm the bug and temporary workaround on TB 52.1.0 ESR, 
and look forward to install 52.1.1 ESR asap.

Comment 29

[Richard Leger]

We do encounter issue with 52.1 where some attachments when saved to drive, from email with body text, end up to be only 27 bytes in size... and therefore cannot be opened because data is missing. But when forwarded they seems fine. 

Would this patch sort such issue? 

When is the 52.1.1 update would more likely be published and available to end-users?

Comment 30

[Richard Leger]

If there is any way for me to get the patch and test it, I would be glad to help but I may need some instructions and help to know how to apply it manually for testing purpouse...

Comment 31

[Jorg K (CEST = GMT+2)]

We're working on TB 52.1.1, hopefully within a few days. Meanwhile try a Daily ESR build from here:
http://ftp.mozilla.org/pub/thunderbird/nightly/2017/05/2017-05-10-03-02-03-comm-esr52/

The 27 bytes are bug 570914, right? That will be included in a later version of TB 52.x.

Comment 32

[Robby]

I can also confirm this issue is now fixed. Thank you.

Comment 33

[Wayne Mery (:wsmwk)]

v.fixed per comment 32

(In reply to Richard Leger from comment #29)
> We do encounter issue with 52.1 where some attachments when saved to drive,
> from email with body text, end up to be only 27 bytes in size... and
> therefore cannot be opened because data is missing. But when forwarded they
> seems fine. 
> 
> Would this patch sort such issue? 
> 
> When is the 52.1.1 update would more likely be published and available to
> end-users?

52.1.1 is out now.  It would be good if you could comment in one or two of your crash reports about results with 52.1.1

Comment 34

[Johann]

There has been a possible regression here. I'm on Thunderbird 60.5.0, 32-bit, Windows 10. The email headers are as follows for the message with an attachment and no body:

Content-Type: text/plain; name=test.txt
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=test.txt
From: [e-mail]
To: [e-mail]
Subject: Test
Message-ID: <[e-mail]>
Date: Mon, 04 Feb 2019 19:35:59 +0000
MIME-Version: 1.0

VGhpcyBpcyB0aGUgdGV4dApmaWxlIDMu

Comment 35

[Jorg K (CEST = GMT+2)]

Yes, the attachment is not decoded properly when opened, however, it's decoded properly when shown inline.

Can you please file a new bug and CC me on that bug.

Comment 36

[Johann]

Added https://bugzilla.mozilla.org/show_bug.cgi?id=1525120