1729221 - PGP-encrypted attachments in unencrypted messages can't be opened or saved (when missing MDC)

Status: NEW

(MailNews Core :: Security: OpenPGP, defect, P3)

Description

[Binarus]

Attachment: bug01.png

+++ This bug was initially created as a clone of Bug #1663169 +++

Steps to reproduce:

Tested with TB 91.0.3 64-bit on Windows 10 Enterprise 64-bit.

When I receive an message which is not encrypted itself, but contains a PGP-encrypted attachment, it is not possible to decrypt and save the attachment. It is also not possible to decrypt and open the attachment.

Actual results:

• When right-clicking the attachment and choosing "Decrypt and Open", a dialog box with an error message immediately appears (see attached images). When closing that error dialog, nothing happens.

• When right-clicking the attachment and choosing "Decrypt and Save as ...", the file name dialog appears (which is expected). After having picked a file name and having closed that dialog, a dialog box with an error message appears (see attached images). When closing that error dialog, nothing happens; notably, no file is created (not even a damaged one - just nothing).

Expected results:

The PGP-encrypted attachment should be decrypted and opened or decrypted an saved, respectively.

Comment 1

[Binarus]

Attachment: bug02.png

Comment 2

[Magnus Melin [:mkmelin]]

Using the normal way works for me. Are you referring to bug 1704820?

Comment 3

[Binarus]

Thank you very much for your fast reply.

No, I don't refer to bug 1704820. I don't use external GnuPG components or keys. All keys needed (including the private ones) are existing in TB's own, native key management.

One thing might be a problem, though: Those attachments have been encrypted by the sender using one of my keys which have expired in the meantime. Could this be the reason? Did you test this case as well?

If you're interested in seeing what's going on, I could offer you a remote session on one of my test PCs where you can examine the situation (preferably TeamViewer, but open for other propositions). I could also help with further logs or screenshots if you tell me in detail what I have to do.

Comment 4

[Magnus Melin [:mkmelin]]

That could easily be what's causing it. Can you send yourself a test message using a non-expired key to verify it?

Comment 5

[Binarus]

Thank you very much!

Thanks for the suggestion. I'll try that although I currently have no clue how to create a PGP-encrypted attachment. Probably it requires GnuPG and the command line. I should be able to learn that, though :-)

If there's an easier way (e.g. using TB only), I would be grateful for a hint.

Comment 6

[Magnus Melin [:mkmelin]]

Unfortunately there's no easy way to do that inside Thunderbird yet, but we discussed adding such a feature.

Comment 7

[Binarus]

No problem with the missing feature; I don't consider it important anyway. I now have tested and can report a surprising result.

With my own encrypted attachment, it works with a correct key as well as with an expired key. What I have done (Windows 10 Enterprise 64-bit with installed Gpg4Win):

• Make a subdirectory which contains one PDF file (file.pdf).
• On the command line, issue gpg --faked-system-time 20200306T152344 -se -r EE4C64E713CE83DD302485157BC95B6787498D06 -u 11634A853BE0373F5C92E907F4E3BB0974AFD89F file.pdf, where EE4C64E713CE83DD302485157BC95B6787498D06 is the expired public key of the intended recipient of the message.
• This creates the file file.pdf.pgp, which is the PDF file, encrypted with the expired public key of the recipient.
• In TB, create a message which is not encrypted, and attach the encrypted file.pdf.pgp to it.
• In TB, send this message in unencrypted form to the recipient (i.e. the message itself is not encrypted, but the attachment of course still is).
• In TB, when the message has arrived in the recipient's inbox, open it, right click on the attachment and choose "Decrypt and Open" or "Decrypt and Save as ...".

---> This works although an expired public key has been used for encryption.

• Repeat the process, this time using a valid public key of the recipient instead of an expired one.

---> This works as well.

That means that decryption of encrypted attachments in unencrypted messages works at least when the encrypted attachments have been created in a conformant way. That's good news.

However, we now have to investigate why we can't decrypt a lot of encrypted attachments from other senders. Perhaps it is due to the fact that these are all encrypted ZIP files (not PDF files). I'll test this next. Another reason may be wrong Content-Type headers. I'll compare the messages from myself (where decryption of attachment works) to those of the other senders (where it doesn't work). Plus, I'll try to see whether the console can help us debug the issue.

Comment 8

[Binarus]

OK, I am quite sure that I've found the culprit.

When trying to decrypt and open or save the problematic attachments, the following message appears in the error console:

bad message, missing MDC

I have a vague idea what this means and that it might be a bad idea to ignore it. However, we urgently need to open those attachments. Is there a way to tell TB to ignore that error and to decrypt the attachment nevertheless?

For the record: If trying to decrypt one of the test attachments we have created ourselves (as described in the previous post), there is no such error message in the error console.

Comment 9

[Magnus Melin [:mkmelin]]

Good investigation! Yes in general missing MDC is a hard failure, but I'd think for the decryption of an attachment case it doesn't really matter.

There seems to exist an UI_IGNORE_MDC_ERROR flag

We call
https://searchfox.org/comm-central/rev/22040823193c06d1aebc27064192dfb45c02e556/mail/extensions/openpgp/content/modules/cryptoAPI/RNPCryptoAPI.jsm#226
-> somewhere around https://searchfox.org/comm-central/rev/22040823193c06d1aebc27064192dfb45c02e556/mail/extensions/openpgp/content/modules/RNP.jsm#1024

No time to check it what to pass from where atm, but the above is probably around the right places.

Comment 10

[Binarus]

Thank you very much!

Your last comment is very instructive. If I had a compilation environment at my hands, I would try to solve it myself. But setting up such an environment is impossible for me at the moment; I guess it would take several days until I have it running, provided that I would be successful at all, which is questionable.

Could you imagine fixing this problem in the foreseeable future? Or would it be considered a new feature which won't be available before the next ESR release anyway?

Comment 11

[Nickolay Olshevsky]

Looks like MDC checking was added as a fix for this issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1638645
From the security point of view, encrypted message with missing MDC should not be considered as safe to decrypt, however from the usability point of view user should be able to do what he wants.

Are these messages generated by modern software or just archived years ago? Solution could be to allow automatic decryption for old messages, and displaying a warning for new ones.

Comment 12

[Binarus]

Thanks for caring.

The messages are from various senders we have long-term relationships with. At the beginning of the relationship, we uploaded our public key to our partner so that they could PGP-encrypt attachments and send them to us. The situation has not changed since.

Obviously, the senders didn't update their systems which batch-create the attachments. Those attachments are usually not created and encrypted "manually", but with the help of custom, automated software which probably would be quite expensive to update. One of the senders surely has millions of customers, and if I would be forced to give a number I would estimate that at least 100.000 of them have opted for PGP-encrypted attachments (invoices with sensitive data in this case).

In other words, we are still receiving such attachments (a few per month). I know of other persons and companies who receive such attachments, too. I can't tell which software exactly is used to create the attachments, because the senders under no circumstances will tell us about it, and it is very likely that each sender has a different system in place. As outlined above, I am nearly sure that it is custom software.

Comment 13

[Nickolay Olshevsky]

Thanks for the detailed explanation.
Good case where security hits the real world.

Looks like warning message This message is encrypted without the MDC, which may be insecure. If you trust sender, press Ok to decrypt it anyway with checkbox [x] do not display for this sender again could be a good solution.

At the time when we have a way to add a self-signed certificate to the browser this would not be more harmful anyway.

Comment 14

[Magnus Melin [:mkmelin]]

I don't quite understand. MDC is supposed to detect an adversary's modifications to the encrypted data. When decrypting an attachment in a plain text message - well you never display anything secure about who it's from anyway: the encryption is just to hide the data, not protect it from modification. If an adversary wants to change the content "all" they need to do is change the attachment and put in their own attachment in there.

Why would it be any less secure to decrypt? Encryption is just a technical matter in this case, and not really about security.

Comment 15

[Nickolay Olshevsky]

(In reply to Magnus Melin [:mkmelin] from comment #14)

Why would it be any less secure to decrypt? Encryption is just a technical matter in this case, and not really about security.

Hm, agree on this, didn't take in account that message itself is plaintext.

Comment 16

[Binarus]

I don't agree here (unless I have seriously misunderstood something, in which case it would be nice if you could correct me):

Being able to verify that a PGP-encrypted attachment has not been tampered with is as important as being able to do this with normal encrypted messages. There are many reasons for this, including legal ones, but let's focus on security here:

Suppose I get a PGP-encrypted attachment which contains an invoice, and suppose that the attachment is missing the MDC. Then an adversary could have tampered with the attachment and could have replaced the bank account of the legitimate payment recipient by his own (after having increased the total payment sum by a factor of 10). We can easily imagine more drastic examples, e.g. the results of a medical examination being faked to trigger a wrong medication of a patient to kill him, contracts being faked, and so on. Another class of attacks is possible if the attachment is opened (as with "Decrypt and Open"): Suppose the said invoice is in HTML format, and due to the missing MDC, the attacker has injected malicious Javascript code into it. That code would then be executed when "Decrypt and Open"-ing the attachment. And so on ...

Therefore, it is of utmost importance that we can verify the authenticity of PGP-encrypted attachments, even if they are in a message which is not encrypted or signed itself, and that Thunderbird by default refuses decrypting such attachments (there is no doubt that the error message should be more instructive, though). In other words, IMHO it would be a bad idea to remove the MDC check completely.

However, in cases like ours, where we have made sure (by other means) that we can trust the attachments, ignoring the missing MDC should be up to the user. As nearly always, we wouldn't need a luxury solution with a fancy UI; instead, a config variable or even an entry in a JSON file which sets a flag "Ignore missing MDCs in attachments" would be sufficient for us. We could set this flag before decrypting the problematic attachments and reset it afterwards to be safe again.

Further hint: If you decide to address the problem, please make sure that you don't disable the check for an existing MDC being violated. A missing MDC would be acceptable in a few cases and if the user knows what he does, but a "wrong" MDC definitely means that somebody has faked the attachment. I am just writing this because I eventually vaguely remember that exactly this mistake already has happened (or at least, almost) elsewhere.

Comment 17

[Nickolay Olshevsky]

(In reply to Binarus from comment #16)

Being able to verify that a PGP-encrypted attachment has not been tampered with is as important as being able to do this with normal encrypted messages. There are many reasons for this, including legal ones, but let's focus on security here:

This should be done via digital signature. MDC doesn't bind contents to the sender in any way, i.e. I may create any fake attachment (knowing invoice structure, for instance), encrypt it to your public key with MDC, and send it to you, faking all other metadata (sender, subject, headers and so on).

Comment 18

[Magnus Melin [:mkmelin]]

Exactly, you can't use encrypted attachments of a plain email and then expect it to be secure. It doesn't matter if we check the signature either: it's easy to add a signature that is technically correct. If you need security, you should have them send you secure email, not emails with "hidden content".

Comment 19

[Binarus]

Thank you very much.

All encrypted messages we get, and all encrypted attachments in unencrypted messages, have a PGP signature. We can verify the signatures, because we have the public key of the sender, and we have checked that it is the correct public key by verifying its fingerprint.

MDC doesn't bind contents to the sender in any way

Yes, that has been clear to me.

i.e. I may create any fake attachment (knowing invoice structure, for instance), encrypt it to your public key with MDC, and send it to you, faking all other metadata (sender, subject, headers and so on).

You could try that. But if the attachment does not contain a PGP signature, we don't trust it. If the attachment does contain a PGP signature, but that signature has been created using a private key whose public counterpart we haven't imported and trusted yet, we also don't trust the attachment.

I strongly hope that Thunderbird will throw warnings in big red letters if a PGP-encrypted attachment does not contain a PGP signature, or a signature from a sender whose public key we haven't imported or trusted yet. But I admit that I didn't test this yet.

Anyway, provided that TB behaves reasonably and checks for and verifies the signature in the attachment, we stumble across that the attachment is not signed correctly.

Exactly, you can't use encrypted attachments of a plain email and then expect it to be secure. It doesn't matter if we check the signature either: it's easy to add a signature that is technically correct.

When there is a signature which is technically correct, its verification will fail and TB will hopefully present a warning in big red letters, unless we have trusted the respective public key. Perhaps we have a different definition of what checking a signature means. In my opinion, this is a two-step-process:

1. Check whether the attachment has a technically correct signature at all. If not, throw a warning in big red letters.
2. Check whether we already have imported and trusted the public PGP key which corresponds to the private PGP key the signature is based on. If not, throw a warning in big red letters.

If you need security, you should have them send you secure email, not emails with "hidden content".

I still believe that there is not much difference between a regular PGP-encrypted message and a PGP-encrypted file or attachment, respectively. Messages as well as files can be signed, and in both cases, the signature can be verified in identical manner. I just have done some tests with GnuPG on the command line which confirm that statement. If you're interested in the details, I can provide the log from the terminal, which is very instructive.

If the method of verifying encrypted attachments' signatures differs from the method of verifying whole encrypted messages' signatures in Thunderbird, then this is due to the implementation, not due to the protocol or data formats.

Comment 20

[Zoe Martin]

https://github.com/Betterbird/thunderbird-patches/blob/main/91/bugs/1729221-ignore_missing_mdc.patch
This allows decryption of some sample attachment we received from Binarus. Sadly it's unclear what rnp_op_verify_get_protection_info() really checks here https://searchfox.org/comm-central/source/mail/extensions/openpgp/content/modules/RNP.jsm#1007 and why any validation error here https://searchfox.org/comm-central/source/mail/extensions/openpgp/content/modules/RNP.jsm#1018 is interpreted asEnigmailConstants.MISSING_MDC here https://searchfox.org/comm-central/source/mail/extensions/openpgp/content/modules/RNP.jsm#1024.

Even looking at the RNP libraries C code doesn't help much:
https://searchfox.org/comm-central/rev/6cc04e8018b418764f7eb1346cae4b6953b621f2/third_party/rnp/src/lib/rnp.cpp#3137
Looks like the function only checks a member of op which was populated by a prior call here:
https://searchfox.org/comm-central/rev/6cc04e8018b418764f7eb1346cae4b6953b621f2/mail/extensions/openpgp/content/modules/RNP.jsm#953

If any validation error really always means "missing MDC" then ignoring the validation result would be valid to accept "missing MDC" for decrypting attachments.

BTW, the MDC code was introduced in bug 1638645 in this changeset:
https://hg.mozilla.org/comm-central/rev/cdef6999ca32#l1.209

Comment 21

[Binarus]

Thank you very much for fixing the problem. We'll test today or tomorrow and report the result.

Another question: I just noticed that the status of the bug is still "Unconfirmed". Should I provide a test message (i.e. an unencrypted email message with an encrypted attachment)? Or a standalone encrypted attachment?

Comment 22

[Magnus Melin [:mkmelin]]

For verifying the attachment is signed correctly there is (separately) the context menu "Verify Signature".
It seems for this special use case you should use that if you want to be sure about the signer, and leave the pure decryption to do just that.

If you can create a sample email with an encrypted attachment missing the MDC, that could be useful. (Use alice's keys so it possible to test.)

Comment 23

[Binarus]

For verifying the attachment is signed correctly there is (separately) the context menu "Verify Signature".

Could you please explain in short where that context menu is? When right-clicking on a PGP-encrypted attachment, there are the following entries in the context menu: Open, Save As ..., Detach ..., Delete, Decrypt and Open, Decrypt and Save As .... Do I have to change a config variable or something like that to enable the Verify Signature menu item?

Comment 24

[Binarus]

If you can create a sample email with an encrypted attachment missing the MDC, that could be useful. (Use alice's keys so it possible to test.)

I have added a saved email message to this bug report which you can use to reproduce the problem (Test-Attachment-without-MDC.eml). However, I have used Bob's key to encrypt the attachment in that message (sorry about that, but I already had it in that form): https://searchfox.org/comm-central/source/mail/test/browser/openpgp/data/keys/[e-mail].

Steps to reproduce (using TB 91 64-bit in Windows 10 Enterprise 64-bit, unless noted otherwise):

• Using your browser, save the file I have just attached to this bug report somewhere to your file system (e.g. c:\Test-Attachment-without-MDC.eml)

• In TB, Open the Error Console (Tools -> Developer Tools -> Error Console)

• In TB, load the file you have just saved (File -> Open -> Saved Message ...)

• A new window opens in TB, showing that message; please note that it has an attachment (test.txt.bad.2.pgp)

• In that new windows, right-click on the attachment and choose Decrypt and Open from the context menu

• Note that a dialog box appears, stating Error - decryption failed, and that at the same time the following line appears in the error console: bad message, missing MDC; that line obviously is triggered by decryption.jsm in line 511.

Comment 25

[Binarus]

Attachment: Test-Attachment-without-MDC.eml

Comment 26

[Binarus]

@Zoe Martin

Thank you very much for having fixed that problem!

We just have downloaded BB 91.1.1-BB-14 (64-bit) and have conducted some tests. It works as expected - we finally can conveniently decrypt the PGP-encrypted attachments without MDC which we are still receiving from a few sources. This will save a lot of hassle, time and nerves (until now, users needed to save these attachments and decrypt them on the command line using GnuPG, which mostly was not possible without the help of an admin).

For the record, and for people who are coming to this page from a search engine:

BB 91.1.1-BB-14 introduces a new config setting (a variable you can set in the config editor): mail.openpgp.ignore_missing_mdc. You must set this variable to true to enable the new behavior. Without this, even BB 91.1.1-BB-14 won't decrypt PGP-encrypted attachments which lack the MDC.

This is the correct approach, because ignoring a missing MDC can be dangerous. Having to take an extra step to enable this behavior is a security measure.

However, in our case, there is no way around it (except the command line, which is not acceptable for normal users), because we can't do without the data from those attachments, and there is definitely no way to make the senders create them the correct way (i.e. with MDC). So I guess we'll leave that config variable permanently set to true :-)

Comment 27

[Kai Engert (:KaiE:)]

I think it is very very confusing for everyone that you discuss forks of Thunderbird in this system.

If users here read this, they might falsely conclude that the comments apply to Thunderbird.

Comment 28

[Kai Engert (:KaiE:)]

Binarus, FYI, I see agreement from Wayne on my statement (comment 27) - he has hidden your comment 26 as "off-topic".

I think we must mark all similar bugzilla comments as off-topic, when we see them.

Comment 29

[Binarus]

Thanks for the notification. I'm fine with that. I had written this post just to confirm that the patch worked, which I thought in turn would help you solve the problem as well.

Best regards,
Binarus