1690603 - PGP/MIME messages fail to verify because of modified boundary preamble line

Status: RESOLVED WORKSFORME

(MailNews Core :: Security: OpenPGP, defect)

Description

[sian]

Attachment: Capture d’écran du 2021-02-03 09-20-20.png

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0

Steps to reproduce:

I sign my emails with an openPGP key (RSA 4096 bits, current validity, successful imported in TDB) generated with gpg --full-gen-key (Linux:debian buster).

Actual results:

I send an email in TDB.
I open the mail I sent in my "Sent" folder : the signing is validated.
I open the mail as the recipient : the signing is not validated.

Same result with each of my email accounts in Thunderbird.

My email provider allows me to use my PGP keys on the webmail. So I tried to sign an email with that key on the webmail : successful. Both in the webmail and in TDB the signing is validated (in the "Sent" folder and as the recipient).

My key seems to be correct and valid. So TDB fails to sign properly.

Expected results:

Signing should be validated.

Comment 1

[Magnus Melin [:mkmelin]]

Can you attach a mail that's claimed to be incorrect (save as .eml). Attach your public key to the mail as well so it can be tested.

Comment 2

[sian]

Attachment: message sent with Thunderbird

Comment 3

[sian]

Attachment: pubkey

Comment 4

[sian]

Attachment: message sent with the same key from the webmail

Comment 5

[sian]

(In reply to Magnus Melin [:mkmelin] from comment #1)

Can you attach a mail that's claimed to be incorrect (save as .eml). Attach your public key to the mail as well so it can be tested.

I Magnus,

Thanks for your message.
Files attached to the bug report.
+++

Comment 6

[sian]

ADD : same result when I sign an email in TDB on my other computer (win10) with my professional PGP key and email account.

Comment 7

[Kai Engert (:KaiE:)]

(In reply to sian from comment #0)

I send an email in TDB.
I open the mail I sent in my "Sent" folder : the signing is validated.
I open the mail as the recipient : the signing is not validated.

This means the bug isn't with Thunderbird.

The message that we send out is exactly what we store in the Sent folder.

If the received message is different thatn what you have in your sent folder, then a mail transport agent modified the message, causing the integrity check to fail.

I suggest that you compare the sent and the received message to understand what is different. Then you could ask your ISP if they are responsible for that modification.

Comment 8

[sian]

(In reply to Kai Engert (:KaiE:) from comment #7)

(In reply to sian from comment #0)

I send an email in TDB.
I open the mail I sent in my "Sent" folder : the signing is validated.
I open the mail as the recipient : the signing is not validated.

This means the bug isn't with Thunderbird.

The message that we send out is exactly what we store in the Sent folder.

If the received message is different thatn what you have in your sent folder, then a mail transport agent modified the message, causing the integrity check to fail.

I suggest that you compare the sent and the received message to understand what is different. Then you could ask your ISP if they are responsible for that modification.

Sorry :KaiE: but

• I have the same problem with two other providers when I send mails from TDB ;
• when I use the provider's webmail, there is no problem with the PGP signing : well verified in TDB (both in sent and received mails) ;
• when I send emails from my smartphone (K-9 mail with openkeychain) with that same provider, no problem too when verifying signing in TDB.

So the provider don't seem to be involved.

Comment 9

[sian]

Attachment: using webmail : diff sent/received mails

Comment 10

[sian]

Attachment: using android : diff sent/received mails

Not the same key used. Generated with K9-mail and openkeychain.

Comment 11

[sian]

Attachment: using Thunderbird : diff sent/received mails

Received mail : signing not validated.
Note : 2 "DKIM-signature" in the received mail.

Comment 12

[sian]

(In reply to Kai Engert (:KaiE:) from comment #7)

I suggest that you compare the sent and the received message to understand what is different. Then you could ask your ISP if they are responsible for that modification.

I attached 3 diffs. In the mail sent/received in TDB, I see 2 "DKIM-Signature" in the header. I don't observe this when the mail is sent from Android or the webmail.

Comment 13

[Kai Engert (:KaiE:)]

sian, I apologize that nobody was able to look at this earlier, I was out sick for a longer period, but now I'm back.

I looked at your diffs, thanks for providing them.

Most of the changes are no problem, because they are outside the signed payload. However, one the changes in the third attachment 9203108 [details] is problematic and can explain the failing signature verification.

The "multipart/signed" part is the start of the critical part. The header lines that follow immediately are ignored for validation. However, everything that changes inside the payload part (with boundary 3ziGT3oz6tmqdWdtf3sDTiD1Hs0sspwd4) will cause a mismatch.

Some transport agent removed the line
"27 This is a multi-part message in MIME format."
and this is likely the cause of the verificaiton failure.

Comment 14

[Kai Engert (:KaiE:)]

According to the rules, I'd have to mark this bug as invalid, because the failure is introduced outside the scope of the Thunderbird application.

I wish we could find ways to make this more robust.

Bug 1731109 is another example where a transport agent modifies messages and causes signature checks to fail.

Comment 15

[Kai Engert (:KaiE:)]

Actually, I'm reopening this for exploring an idea.

Would it be OK to ignore all preamble lines that appear inside a signed message, if it's in outermost structure?
https://datatracker.ietf.org/doc/html/rfc2046#section-5.1.1

   NOTE TO IMPLEMENTORS:  Boundary string comparisons must compare the
   boundary value with the beginning of each candidate line.  An exact
   match of the entire candidate line is not required; it is sufficient
   that the boundary appear in its entirety following the CRLF.

   There appears to be room for additional information prior to the
   first boundary delimiter line and following the final boundary
   delimiter line.  These areas should generally be left blank, and
   implementations must ignore anything that appears before the first
   boundary delimiter line or after the last one.

   NOTE:  These "preamble" and "epilogue" areas are generally not used
   because of the lack of proper typing of these parts and the lack of
   clear semantics for handling these areas at gateways, particularly
   X.400 gateways.  However, rather than leaving the preamble area
   blank, many MIME implementations have found this to be a convenient
   place to insert an explanatory note for recipients who read the
   message with pre-MIME software, since such notes will be ignored by
   MIME-compliant software.

Comment 16

[Kai Engert (:KaiE:)]

This is the complete structure of an example message that has both html and text parts:

    Content-Type: multipart/signed; micalg=pgp-sha256;
     protocol="application/pgp-signature";
     boundary="------------3wzKMz7BTwms4UFJqWTR5Vlf"

    This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
    --------------3wzKMz7BTwms4UFJqWTR5Vlf
    Content-Type: multipart/mixed;
     boundary="------------Zmj7gnD7pdVYiMLC09dM402C";
     protected-headers="v1"
    From: Kai Engert <[e-mail]>
    To: Kai Engert <[e-mail]>
    Message-ID: <[e-mail]>
    Subject: sig to myself html

    --------------Zmj7gnD7pdVYiMLC09dM402C
    Content-Type: multipart/alternative;
     boundary="------------FeeXD10On4R08MY7Xsd0mzZN"

---> preamble line inside signed content
    --------------FeeXD10On4R08MY7Xsd0mzZN
    Content-Type: text/plain; charset=UTF-8; format=flowed
    Content-Transfer-Encoding: base64

    KmlqaSpqMzQNCg0KDQo=
    --------------FeeXD10On4R08MY7Xsd0mzZN
    Content-Type: text/html; charset=UTF-8
    Content-Transfer-Encoding: quoted-printable

    <html>...
    --------------FeeXD10On4R08MY7Xsd0mzZN--


    --------------Zmj7gnD7pdVYiMLC09dM402C--

    --------------3wzKMz7BTwms4UFJqWTR5Vlf
    Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
    Content-Description: OpenPGP digital signature
    Content-Disposition: attachment; filename="OpenPGP_signature"

    -----BEGIN PGP SIGNATURE-----

    wsF5BAABCAAjFiEEIdFuZ....
    -----END PGP SIGNATURE-----

    --------------3wzKMz7BTwms4UFJqWTR5Vlf--

Comment 17

[Kai Engert (:KaiE:)]

Refer to the line that is marked above with --->

This is the line that contains the modification in sian's third diff.

I see a difference in what we produce in TB 78 and TB 91.

With TB 78, the line contains what we see in sian's diff:
This is a multi-part message in MIME format.

With TB 91, the line is empty.

sian, the above change might not have been intended, it was likely caused by a change in the code that we use for creating new messages in TB 91.

However, this changed could have the side effect to produce emails that no longer fail in your scenario.
Do you want to test again?

Comment 18

[Kai Engert (:KaiE:)]

(In reply to Kai Engert (:KaiE:) from comment #15)

Would it be OK to ignore all preamble lines that appear inside a signed message, if it's in outermost structure?
https://datatracker.ietf.org/doc/html/rfc2046#section-5.1.1

Answering myself:
No, ignoring those lines won't work. If the sending agent has added preamble lines, then the sending agent very likely has included that text in the message hash calculation.

If we ignore the line, but the sender included it, then our verification will fail - even if we have received the exact message as it was sent.

I think the best we can do is to produce empty preamble lines - based on the observation that some agents strip them - and hope that no transport agents will add their own preamble lines.

With TB 91, it appears we already implemented the "empty preamble", caused as a side effect of other changes.

Comment 19

[Kai Engert (:KaiE:)]

I'm marking this fixed for now, in the hope that we now fully avoid such preamble lines in the middle of signed emails. Should we discover that TB still adds them in certain scenarios, we should add code to omit them reliably.

Comment 20

[Magnus Melin [:mkmelin]]

-> WFM (no know patch, probably the rewritten sending or smtp backend)