Closed Bug 1607872 Opened 6 years ago Closed 1 year ago

Attachments missed from complex MIME (posta certificata)

Categories

(MailNews Core :: Security: S/MIME, defect)

Product:
MailNews Core
Component:
Security: S/MIME
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: vesely, Unassigned)

References

(Regression)

Details

(Keywords: regression)

Attachments

(3 files)

redacted mail file
18.32 KB, text/plain
Details
Open saved message result
11.49 KB, image/png
Details
An email containing posta certificata as an attachment.
31.88 KB, application/octet-stream
Details
Attached file redacted mail file —

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

Open a message

Actual results:

Displays "2 attachments"

Expected results:

The file has a complex structure (rfc6109) displayed below.

I saved the file, changed boundaries, made sure there's an empty line before end-boundaries. Then redacted some info and shortened long parts so as to reach a reasonable size.

I recall I used to read the attachments from "posta certificata", it was perhaps TB 60. With the new version I'm unable to read the old messages too.

structure display:
ale@pcale:~/tmp$ reformime < postacert2-bis.eml
1
1.1
1.1.1
1.1.1.1
1.1.1.2
1.1.2
1.1.2.1
1.1.2.1.1
1.1.2.1.2
1.1.2.1.2.1
1.1.2.1.2.1.1
1.1.2.1.2.1.1.1
1.1.2.1.2.1.1.1.1
1.1.2.1.2.1.1.1.2
1.1.2.1.2.1.1.2
1.1.2.1.2.1.1.3
1.1.2.1.2.1.1.3.1
1.1.2.1.2.1.1.3.1.1
1.1.2.1.2.1.1.3.1.2
1.1.2.1.2.1.1.3.1.3
1.1.2.1.2.1.2
1.1.3
1.2

Hard to tell what happened here. I see two attachments, postacert.eml and copia.eml. What is the expected result? Can you make the unredacted message available? Maybe as ZIP file with a password and you mail me the password?

We've seen those S/MIME encoded POSTA CERTIFICATA messages before, they used to cause some IMAP trouble, but we fixed that.

BTW, in TB 60 I see seven attachments. Maybe we should look for the regression then.

Alice, can you help us here. Import the message attached and display it. How many attachments do you see? I see 2 in TB 68 and 7 in TB 60. When did that change?

Flags: needinfo?(alice0775)

60.9.1 -- see 7 attachments
68.0 -- see 7 attachments
68.1.0 -- see 7 attachments
68.1.1. -- see 2 attachments
Trunk -- see 2 attachments

Thought maybe something I broke but couldn't find anything. Guess the above "regression window" by Alice points to same mid-September changes that went into 68.1.1; some major mime changes.

It's bug 1240290 as I had already assumed. Thank you so much for finding/confirming it, Alice.

Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(kaie)
Keywords: regression
Regressed by: CVE-2019-11755

Alessandro, is the message that doesn't work the original POSTA CERTIFICATA message, or a product of attaching such a message to another message. Certainly we need to make sure that POSTA CERTIFICATA messages still work, but it would be a lesser problem if some forwarding of such a message had broken.

We should check whether our "original" POSTA CERTIFICATA sample message in attachment 8676772 [details] of bug 1216951 still works.

(In reply to Jorg K (GMT+1) (PTO to 26th Jan 2020, sporadically reading bugmail) from comment #6)

Alessandro, is the message that doesn't work the original POSTA CERTIFICATA message, or a product of attaching such a message to another message. Certainly we need to make sure that POSTA CERTIFICATA messages still work, but it would be a lesser problem if some forwarding of such a message had broken.

The forwarding leaves something to be desired. E.g. it has a quoted-printable .xml attachment encoded like so:

                       [...large snip ...]<DescrizioneAttachment>Informazioni A=
    llegate</DescrizioneAttachment><Attachment>JVBERi0xLjQKJcfsj6IKNSAwIG9iago8=
    PC9MZW5ndGggNiAwIFIvRmlsdGVyIC9GbGF0ZURlY29k
    ZT4+CnN0cmVhbQp4nK1X227bRhB951fMSxG5NTd732X6pNhKoFZxHEs2UARFwUiMwUISY4m2Ub8=
    l
    X+JP6Cf1U3p2ScqSrTQtUBkWudeZOXPm7OqKOBPEw1/7nC6S52eOLtfJFUkvbOw0XJPLODmubTP=
    D
    [... continuing with one extra byte per line and ending with no line terminator ...]
    NTNBMUFBQUQ3OURCQUVBNkYyPjw3QTYwNUYxNUNGRTBFQzUzQTFBQUFENzlEQkFFQTZGMj5dCj4=
    +
    CnN0YXJ0eHJlZgoyMjIyOQolJUVPRgo=3D
    </Attachment></Allegati></FatturaElettronicaBody></p:FatturaElettronica>
    ------=_Part_74792_-2133189196.1578458197275
    Content-Type: text/plain; charset=UTF-8
    Content-Transfer-Encoding: quoted-printable
    [...]

However, that's still legal.

We should check whether our "original" POSTA CERTIFICATA sample message in attachment 8676772 [details] of bug 1216951 still works.

I can send the original file privately. The attachments in the cannibalized copy I uploaded have invalid content. However, TB should be able to show them, save them, and attempt to open them possibly failing gracefully.

Component: Untriaged → Security: S/MIME
Product: Thunderbird → MailNews Core

OK, I received the original message and also forwarded it to Kai. It behaves the same as the cut-down version, only two message attachments (postacert.eml and copia.eml), none of the other parts visible in TB 60.

If I understand the background correctly, there's an "official" billing system in Italy where providers sent their bills via e-mail (with attachments) and which in turn forwards the bills to the customers using the format of the messages provided by Alessandro. So basically we just damaged TB and users in Italy can't read that sort of message any more. Is that correct?

Alessandro, do I interpret it correctly that you are the provider sending the message as well as the recipient receiving it via POSTA CERTIFICATA. In which case, could you also supply the original that was sent to the billing system before forwarding it.

Just an observation: If you save postacert.eml and copia.eml and then drag them into a mail folder again, then you see the attachments. Bug 1240290 disabled the decoding of messages in parts, that is, anything but the top level. The complainer in bug 1524750 was told: WONTFIX with a reference to bug 1576655.
EDIT: See my comment in bug 1576655 comment #2.

For the Italian case, AIUI, it's not S/MIME but independently signed attachments passed around.

Now I'm getting astonished. Yes, I tried and save "copia.eml" instead of opening.it, then open the saved message. In that case, I get the attachments.

Enigmail (2.1.5) enabled seems to worsen TB behavior. A number of times (but not always) opening those messages while Enigmail is enabled results in an empty message with no attachments.

Alessandro, do I interpret it correctly that you are the provider sending the message as well as the recipient receiving it via POSTA CERTIFICATA.

No. W.r.t. rfc6109, the Italian law requires 1 million nominal capital to run a "POSTA CERTIFICATA" mailbox provider. At the same time, companies are required —private individuals just encouraged— to rent a mailbox at those providers. The law that mandates to send invoices through a governmental agency is newer. The agency wants a "POSTA CERTIFICATA" mailbox to forward the invoice.

Signatures are discussed in the rfc.

Attachment #9119508 - Attachment mime type: application/x-extension-eml → text/plain

Sorry for the ignorance. The message you supplied has a To: at tana.it, so it looks like the message was sent to you. So far so good. The From: is "Per conto di: alessandro.vesely@", so that looks like it was sent by you as well. But that's apparently not the case. It comes from a business sending you an invoice.

Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-1; 
	boundary="boundary-1"
--boundary-1
    Content-Type: multipart/mixed; 
      boundary="boundary-2"
    --boundary-2
        Content-Type: multipart/alternative; 
          boundary="boundary-3"
        --boundary-3
            Content-Type: text/plain; charset="ISO-8859-1"
        --boundary-3
            Content-Type: text/html; charset="ISO-8859-1"
        --boundary-3--
    --boundary-2
        Content-Type: message/rfc822; name=postacert.eml
        Content-Type: multipart/mixed; 
          boundary="boundary-4"
        --boundary-4
            Content-Type: text/plain; charset=us-ascii
        --boundary-4
            Content-Type: message/rfc822; name=copia.eml
            Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg="sha-256"; boundary="boundary-5"
            --boundary-5
                Content-Type: multipart/mixed; boundary="boundary-6"
                --boundary-6
                    Content-Type: multipart/alternative;
                     boundary="boundary-7"
                    --boundary-7
                        Content-Type: text/plain; charset="iso-8859-1"
                    --boundary-7
                        Content-Type: text/html; charset="iso-8859-1"
                    --boundary-7--
                --boundary-6
                    Content-Type: application/xml; name="daticert.xml"
                --boundary-6
                    Content-Type: message/rfc822; name="postacert.eml"
                    Content-Type: multipart/mixed; 
                      boundary="boundary-8"
                    --boundary-8
                        Content-Type: application/octet-stream; name=IT07543230960_0eK7i_MT_001.xml
                    --boundary-8
                        Content-Type: application/octet-stream; name=IT07543230960_0eK7i.xml
                    --boundary-8
                        Content-Type: text/plain; charset=UTF-8
                    --boundary-8--
                --boundary-6--
            --boundary-5
                Content-Type: application/pkcs7-signature; name="smime.p7s"
            --boundary-5--
        --boundary-4--
    --boundary-2
        Content-Type: application/xml; charset=UTF-8; name=daticert.xml
    --boundary-2--
--boundary-1
    Content-Type: application/pkcs7-signature; name=smime.p7s; smime-type=signed-data
--boundary-1--

For the above structure, I used the attached message.
For all the following statements, I used the original email, which I've been sent by Jörg.

I see the following differences in behavior:

(1) attachments inline setting

I see that we display different contents, based on the setting "view / display attachments inline".

With that setting disabled, both TB 60 and TB 68 behave the same.
They show 2 attachments, postacert.eml and daticert.xml

When trying to save them, both can successfully save postacert.eml.
However, trying to save daticert.xml, only TB 60 can save it.
TB 68 fails, it saves an empty file.

Looking at the structure, I see there are two embedded parts, in different places, that both contain a filename daticert.xml. I edited the original message and renamed the second of them (from the outer message), but that didn't help, the saved file was still empty. I don't know why.
-> regression in tb 68, fails to save outer attachment daticert.xml

With the "attachments inline" setting, enabled, 60 and 68 behave differently. I think all previous comments in this bug were made with that setting enabled.

60 shows 7 attachments.
68 shows 2 attachments, postacert.eml and copia.eml

When saving these 2, both produce identical files.
However, the display of 68 has a bug, it claims that copia.eml is 1.5 KB, but the saved file is > 50 KB.
-> regression in tb 68, check why smaller size is shown

(2) Signature status

With TB 60, the message is shown with a valid signature.
Both for the top level, and also, when double clicking the copia.eml attachment, that inner message is shown with a valid signature.

With TB 68, we don't show any signature status for the top level.

IIRC, with 68, when parsing the nested contents, and we detect that there's an outer level signature, plus an inner level signature:

  • we refuse to show a signature status for the top level, as those multiple signatures message might be an attempt to trick thunderbird
  • we completely refuse to process the inner signed message, which is why the contents of copia.eml aren't shown

(3) amount of attachments

The outermost message is of type multipart/signed, has an alternative text or html content (that is shown), and two attachments, postacert.eml and daticert.xml - Postacert.eml is of type "message/rfc822".

The Postacert message is "multipart/mixed" with two parts: text, and another rfc822 named copia.eml

copia.eml is a signed message. Because of the recent fix, we refuse to process this mime part at all.

When "attachments inline" is disabled, (1), we don't process the outer postacert.eml, and therefore we don't even see copia.eml, that's why we don't display it in the attachments list, and don't offer to save it.

With "attachments inline" enabled, we do process postacert.eml, and arrive at copia.eml, and show it as an attachment.
However, we don't show the daticert.xml part from the outer part, I think that's a bug.
-> tb 68 regression, outer daticert.xml isn't shown

Message part copia.eml contains one alternative text or html part, another daticert.xml, another postacert.eml, which contains two xml attachments. That's 4 non-text parts. I think it's correct that we don't display those 4 parts, because of the intended change to not process copia.eml

Conclusion:
It seems we have a few regressions, but the primary issue reported here is intended.

(This bug also demonstrates that we don't have a sufficient number of automated tests for complex MIME messages.)

Flags: needinfo?(kaie)

(In reply to Kai Engert (:KaiE:) from comment #14)

IIRC, with 68, when parsing the nested contents, and we detect that there's an outer level signature, plus an inner level signature:

  • we refuse to show a signature status for the top level, as those multiple signatures message might be an attempt to trick thunderbird
  • we completely refuse to process the inner signed message, which is why the contents of copia.eml aren't shown

What's the rationale of that behavior? When I try to see Bug #1240290 I get access denied...

The rationale is to protect against spoofing attempts, in which an attacker adds an additional signature to a message, trying to override an internal signature (possibly encrypted), with the intention to trick the recipient that a message was originally signed by someone else.

You don't mean EFAIL attacks, do you? Would it make sense to exempt plain text, unencrypted messages?

A signature only means that the signer has had access to the message at hand. It usually entails some responsibility in the forwarding of the message itself. It doesn't grant authorship, nor claim paternity of the content. I can hardly understand how safety is improved by hiding signatures.

Did you observe the complete failure to display the message when Enigmail is enabled? Is it related to that hiding?

(In reply to Alessandro Vesely from comment #17)

You don't mean EFAIL attacks, do you?

It's a similar scenario.

Would it make sense to exempt plain text, unencrypted messages?

Maybe. But we'd have to carefully check our implementation, to ensure we do the right thing. However, what's the right thing to do, if there are multiple signatures present? What if there are two MIME parts present at the same nesting level, both having a signature? Which one should be shown, which one should be ignored? If we wanted to handle this in a more advanced way, we'd have to carefully analyze the scenarios we'd want to support, and which ones we'd want to reject - and ensure our implementation can handle those signatures at various layers in the MIME tree. WIth the current implementation, that processes all parts in a streaming approach, one by one, that's tricky to do, and would likely require enhancements to collect data during processing, and then do a final decision phase after processing. Right now, we disable processing on ambiguity, to avoid that we do something that's incorrect.

Did you observe the complete failure to display the message when Enigmail is enabled? Is it related to that hiding?

That would be a bug in Enigmail. I haven't looked at that yet.

I'd consider this bug a duplicate of bug 1576655.
This might include an option to double-click nested messages, and treat them as top-level messages in their own window, thereby allowing signatures to be processed inside that separate window.

(In reply to Kai Engert (:KaiE:) from comment #19)

I'd consider this bug a duplicate of bug 1576655.

In part, yes. What I find disturbing is that saving an eml attachment to a file and then reopening it produces a different result than double clicking on the same attachment. Isn't the relevant function implemented once?

Perhaps, this is not even the occasion to treat ".eml" as message/rfc822, as stated by /etc/mime.types...

(In reply to Alessandro Vesely from comment #20)

In part, yes. What I find disturbing is that saving an eml attachment to a file and then reopening it produces a different result than double clicking on the same attachment. Isn't the relevant function implemented once?

At the time the attachment is double-clicked, that message part is still loaded in the context of the original message, which causes it to notice that there are multiple signatures.

When opening the attachment separately, no additional signatures are found in that message.

To get you the same behavior, we'd have to implement smarter filtering for the ambiguity detection. When deciding "is there another signature", we'd have to ignore other signatures that aren't visible in the shown window.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE

Reopening per bug 978570 comment 5.

Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---

I think Magnus means bug 1607872 comment 5

(In reply to Wayne Mery (:wsmwk) from comment #24)

I think Magnus means bug 1607872 comment 5

Perhaps bug 1576655 comment 5. I was just trying to understand fixing strategies...

This is an email that contains a "posta certificata" .eml as an attachment.
If you open this file you'll see the attachment. If you open that attachment you'll see it empty. If you forward this message then you'll be able to open the attached .eml file correctly.
I hope it can be useful.

Severity: normal → S3
Duplicate of this bug: 1855391

(In reply to alessandro from comment #27)

Created attachment 9230206 [details]
An email containing posta certificata as an attachment.

This is an email that contains a "posta certificata" .eml as an attachment.
If you open this file you'll see the attachment. If you open that attachment you'll see it empty. If you forward this message then you'll be able to open the attached .eml file correctly.
I hope it can be useful.

I saved this .eml file and dragged it to an imap folder for a server. Then opened the email and I see both attachment not empty.
Do I have to actually receive this email as it is normally sent via SMTP to see the problem?

The first attachment above by Alesandro V. (redacted mail file), I'm not sure what I should be seeing and if dragging into a folder is a way to see the issue. Edit add: Here I see 7 attachment when inline a 2 when not inline.

Edit add: Note also that the number of attachment shown differs depending on if showing attachments inline or not. With inline I see 7 attachments listed at the bottom. With not inline there are only 2. That's because show inline displays all the visible attachment as flat list (but doesn't show the .xml attachments).
Note also, with version >= 110 there is no fetching by parts at all, which may or not be significant.

I see the issue is solved in version 115.3.1

Flags: needinfo?(vesely)

Yup, I can open "copia.eml" directly, without having to save it first. Thanks.

Flags: needinfo?(vesely)

Magnus, Wayne, do you understand why this is now fixed?

Much has change in two years. Maybe related to fixing bug 1630688 or bug 1854592?

Yep, hard to say.

WFM per comment 31. Thanks.

Status: REOPENED → RESOLVED
Closed: 5 years ago → 1 year ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: