Closed Bug 1361422 Opened 4 years ago Closed 4 years ago

Attachment saved as base64 encoded file if attachment is the only part of the message

Categories

(MailNews Core :: MIME, defect)

x86_64
Windows 7
defect
Not set
normal

Tracking

(thunderbird_esr5253+ fixed, thunderbird53 wontfix, thunderbird54 fixed, thunderbird55 fixed)

VERIFIED FIXED
Thunderbird 55.0
Tracking Status
thunderbird_esr52 53+ fixed
thunderbird53 --- wontfix
thunderbird54 --- fixed
thunderbird55 --- fixed

People

(Reporter: robbyke, Unassigned)

References

Details

(Keywords: regression, Whiteboard: [regression:TB52.1.0,TB53])

Attachments

(2 files, 1 obsolete file)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0
Build ID: 20170413192749

Steps to reproduce:

1. Update to Thunderbird 52.1.0.
2. Try to open emails with ZIP attachments, or save them as files to disk and then try to open them.


Actual results:

Error from archival program that the ZIP can not be opened due to it being malformed, damaged or corrupted.

Upon further inspection it turns out that Thunderbird 52.1.0 is saving the ZIP files to disk in base64 encoding instead of properly decoding it. I confirmed this by opening the ZIP file in notepad and saw the base64 encoding.


Expected results:

ZIP files should open normally like in previous Thunderbird versions prior to the 52.1.0 update.
After more testing I have narrowed this problem down a bit. As we have SPF/DKIM/DMARC records for our domain name, I have enabled DMARC reports, so I regularly get DMARC reports. It turns out that *only* the reports sent by Google (from noreply-dmarc-support@google.com) are affected, I have a couple of older reports still in my inbox whose ZIP attachments also do not want to open anymore due to this problem. Other emails who contain other type of attachment continue to work as normal.
OS: Unspecified → Windows 7
Hardware: Unspecified → x86_64
Summary: ZIP attachments are saved as base64 encoded files which results in unable to open them → ZIP attachments from Google DMARC reports are saved as base64 encoded files which results in being unable to open them
After yet some more testing I have found a workaround so that in the meantime I'm still able to open these ZIP attachments. By clicking Forward in the email, a new window opens where the ZIP attachment is visible in the upper right corner. Double-clicking the file over there will in fact open the ZIP file normally.
This is most likely a regression from bug 1350080, which was uplifted to TB 52.1 (so it's not in TB 52.0.1):
https://hg.mozilla.org/releases/comm-esr52/rev/10a70478cc88e31fec40585d41112cc6eaee04b0

I was considering a "safer" version of the code, see bug 1350080 comment #21, but in the end we didn't deem it necessary since we could not find any example that would require it.

I can actually open a ZIP archive simply attached to a message, so the messages failing for you must have an "unexpected" MIME structure, as you said: *only* the reports sent by Google. Can you please save such a report as .eml file and attach it here. Or zip it up and e-mail it to me privately.

Alfred, can you please take a look.
Flags: needinfo?(robbyke)
Flags: needinfo?(infofrommozilla)
I really need a sample to take this further.
I found one:
https://lists.inverse.ca/sogo/arc/users/2015-06/msg00327.html

| Content-Type: application/zip;
|  name="google.com!xxxxxx.dk!1435536000!1435622399.zip"
| Content-Disposition: attachment;
|  filename="google.com!xxxxxx.dk!1435536000!1435622399.zip"
| Content-Transfer-Encoding: base64

Your "safer version" from Bug 1350080 Comment 18 works.
Flags: needinfo?(infofrommozilla)
Attached file Test message
Test message assembled from https://lists.inverse.ca/sogo/arc/users/2015-06/msg00327.html.

Alfred, can you please attach a patch to fix this.
Flags: needinfo?(robbyke)
Attachment #8864261 - Attachment mime type: message/rfc822 → text/plain
Attached patch Bug1361422_v1.patch (obsolete) — Splinter Review
Decode (base64/quoted-printable) when saving "application/*" message attachments

As far as I've tested it, forwarded messages are still encoded.
Attachment #8864267 - Flags: review?(jorgk)
(In reply to Alfred Peters from comment #7)

> As far as I've tested it, forwarded messages are still encoded.

Of course. The entire message is forwarded, not just the attachment.
Reading from the above comments, do you still require an EML file from me? Or is that no longer needed?
Well, a real example rather than a fake one wouldn't go astray ;-)
Attached patch Bug1361422 (v2).Splinter Review
How about this? Since there are may text types: text/plain, text/html, text/richtext, text/enriched, text/calendar, I'd like to cover them all.
Attachment #8864267 - Attachment is obsolete: true
Attachment #8864267 - Flags: review?(jorgk)
Attachment #8864291 - Flags: review+
Attachment #8864291 - Flags: feedback?(infofrommozilla)
I just emailed you a real sample. :)
That works now, too. Thanks.
Comment on attachment 8864291 [details] [diff] [review]
Bug1361422 (v2).

Does work as well.
Attachment #8864291 - Flags: feedback?(infofrommozilla) → feedback+
(In reply to Jorg K (GMT+2) from comment #10)
> Well, a real example rather than a fake one wouldn't go astray ;-)

I had no problem with v1.
But yes, v2 is more flexible.
https://hg.mozilla.org/comm-central/rev/a4cdf9a9fe666b2b1e045d6fae82406130ea1398

Thanks for you help, Alfred.
Status: UNCONFIRMED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 55.0
Comment on attachment 8864291 [details] [diff] [review]
Bug1361422 (v2).

[Approval Request Comment]
Regression caused by (bug #): Bug 1350080
Attachment #8864291 - Flags: approval-comm-esr52?
Attachment #8864291 - Flags: approval-comm-beta+
Duplicate of this bug: 1362660
Duplicate of this bug: 1362781
Robby, Hungerburg, yup,

Is your problem gone, with no ill effects, if you use the current nightly build from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/
thunderbird-55.0a1.en-US.win32.installer.exe windows
thunderbird-55.0a1.en-US.linux-x86_64.tar.bz2 linux
Flags: needinfo?(yup)
Flags: needinfo?(robbyke)
Flags: needinfo?(pch)
Whiteboard: [regression:TB52.1.0,TB53]
Hello Wayne, I did download the linux executable, installed to and launched it from /opt/, skipped the integration and found myself running "Daily", which opened both my own Test.eml and the (business like rude) offending SAP content-type: application/pdf only message just fine. No ill effects became apparent during this short exercise.
Flags: needinfo?(pch)
Duplicate of this bug: 1363236
Summary: ZIP attachments from Google DMARC reports are saved as base64 encoded files which results in being unable to open them → Attachment saved as base64 encoded file if attachment is the only part of the message
Attachment #8864291 - Flags: approval-comm-esr52? → approval-comm-esr52+
We confirm the bug and temporary workaround on TB 52.1.0 ESR, 
and look forward to install 52.1.1 ESR asap.
Duplicate of this bug: 1363340
Duplicate of this bug: 1363985
Duplicate of this bug: 1364055
We do encounter issue with 52.1 where some attachments when saved to drive, from email with body text, end up to be only 27 bytes in size... and therefore cannot be opened because data is missing. But when forwarded they seems fine. 

Would this patch sort such issue? 

When is the 52.1.1 update would more likely be published and available to end-users?
If there is any way for me to get the patch and test it, I would be glad to help but I may need some instructions and help to know how to apply it manually for testing purpouse...
We're working on TB 52.1.1, hopefully within a few days. Meanwhile try a Daily ESR build from here:
http://ftp.mozilla.org/pub/thunderbird/nightly/2017/05/2017-05-10-03-02-03-comm-esr52/

The 27 bytes are bug 570914, right? That will be included in a later version of TB 52.x.
I can also confirm this issue is now fixed. Thank you.
Flags: needinfo?(robbyke)
v.fixed per comment 32

(In reply to Richard Leger from comment #29)
> We do encounter issue with 52.1 where some attachments when saved to drive,
> from email with body text, end up to be only 27 bytes in size... and
> therefore cannot be opened because data is missing. But when forwarded they
> seems fine. 
> 
> Would this patch sort such issue? 
> 
> When is the 52.1.1 update would more likely be published and available to
> end-users?

52.1.1 is out now.  It would be good if you could comment in one or two of your crash reports about results with 52.1.1
Status: RESOLVED → VERIFIED
Flags: needinfo?(yup)
Component: Untriaged → MIME
Product: Thunderbird → MailNews Core
Version: 52 Branch → 52

There has been a possible regression here. I'm on Thunderbird 60.5.0, 32-bit, Windows 10. The email headers are as follows for the message with an attachment and no body:

Content-Type: text/plain; name=test.txt
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=test.txt
From: from@localhost.com
To: me@mydomain.com
Subject: Test
Message-ID: <c7a2664e-6803-c144-1c70-96dc0941e3a1@localhost.com>
Date: Mon, 04 Feb 2019 19:35:59 +0000
MIME-Version: 1.0

VGhpcyBpcyB0aGUgdGV4dApmaWxlIDMu

Yes, the attachment is not decoded properly when opened, however, it's decoded properly when shown inline.

Can you please file a new bug and CC me on that bug.

You need to log in before you can comment on or make changes to this bug.