1665497 - OpenPGP. If encryption subkey expired, but primary not, show better error than "Sending of the message failed"

Status: RESOLVED FIXED

(MailNews Core :: Security: OpenPGP, defect, P1)

Description

[gunter.zielke]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36

Steps to reproduce:

upgraded to TB 78
Migrated my enigmail data to the new e2ee
enabled openpgp for account [e-mail]
try to send an encrypted test message to [e-mail]

Actual results:

I am getting a "Sending of the message failed" message box with no indication WHAT might have gone wrong. This setup had worked with the previous version of TB and Enigmail.

Expected results:

the message is encrypted and sent to the recipient, or at least more information is given what might have gone wrong.

Comment 1

[Magnus Melin [:mkmelin]]

Well, does sending an unencrypted mail work? It sounds like there was just a "normal" send failure, for whatever reason.

Comment 2

[gunter.zielke]

Naturally, I tried to send unencrypted mail - and that has no problems.

Comment 3

[Magnus Melin [:mkmelin]]

Do you get an error in the Error Console?

Comment 4

[gunter.zielke]

I did not even know there was an error console in TB! And yes there seems to be quite a bit of info that might help in tracking down the problem. Below the output in the different tabs for the failed send event:

In the debug:
16:04:28.873 in getEncryptionFlags, gSendEncrypted=true, gSendSigned=true enigmailMsgComposeOverlay.js:1697:13
16:04:28.915 getCryptParams parameters: from=0x002A601E8FDE6FF6, to=[e-mail], bcc=, hash=SHA256, flags=20673, ascii=0, errorObj=
Object { value: "" }
, logObj=
Object { }
encryption.jsm:86:13
16:04:28.916 getCryptParams, got: to=[e-mail], bcc= encryption.jsm:125:13
16:04:28.916 getCryptParams returning: encryption.jsm:178:13
16:04:28.917
Object { sender: "0x002A601E8FDE6FF6", sign: true, signatureHash: "SHA256", sigTypeClear: false, sigTypeDetached: true, encrypt: false, encryptToSender: false, armor: true, senderKeyIsExternal: false, to: (1) […], … }
encryption.jsm:179:13
16:04:29.231 sendFlags=000050c1 encryption.jsm:358:13
16:04:29.231 getCryptParams parameters: from=0x002A601E8FDE6FF6, to=[e-mail], bcc=, hash=SHA256, flags=20674, ascii=0, errorObj=
Object { value: "" }
, logObj=
Object { }
encryption.jsm:86:13
16:04:29.232 getCryptParams, got: to=[e-mail], bcc= encryption.jsm:125:13
16:04:29.232 getCryptParams returning: encryption.jsm:178:13
16:04:29.232
Object { sender: "0x002A601E8FDE6FF6", sign: false, signatureHash: "", sigTypeClear: false, sigTypeDetached: false, encrypt: true, encryptToSender: true, armor: true, senderKeyIsExternal: false, to: (1) […], … }
encryption.jsm:179:13
16:04:29.246 sendFlags=000050c2 encryption.jsm:358:13
16:04:29.246 Error: failure in finishCryptoEncapsulation
finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:588
CompleteGenericSendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4685
GenericSendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4587
SendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4855
doCommand chrome://messenger/content/messengercompose/MsgComposeCommands.js:967
doCommand chrome://messenger/content/messengercompose/MsgComposeCommands.js:1163
goDoCommand chrome://global/content/globalOverlay.js:101
oncommand chrome://messenger/content/messengercompose/messengercompose.xhtml:1
mimeEncrypt.jsm:603:15

​In the Logs tab:
16:04:29.245 CryptoAPI.sync() failed result: Error: no suitable subkey found for encrypt
addSuitableEncryptKey chrome://openpgp/content/modules/RNP.jsm:1971
encryptAndOrSign chrome://openpgp/content/modules/RNP.jsm:2099
sync chrome://openpgp/content/modules/cryptoAPI/interface.js:51
encryptMessageStart chrome://openpgp/content/modules/encryption.jsm:341
finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:567
CompleteGenericSendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4685
GenericSendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4587
SendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4855
doCommand chrome://messenger/content/messengercompose/MsgComposeCommands.js:967
doCommand chrome://messenger/content/messengercompose/MsgComposeCommands.js:1163
goDoCommand chrome://global/content/globalOverlay.js:101
oncommand chrome://messenger/content/messengercompose/messengercompose.xhtml:

In the Error tab:
16:04:23.222
GenericSendMessage FAILED: [Exception... "Component returned failure code: 0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS) [nsIMsgCompose.SendMsg]" nsresult: "0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)" location: "JS frame :: chrome://messenger/content/messengercompose/MsgComposeCommands.js :: CompleteGenericSendMessage :: line 4685" data: no] MsgComposeCommands.js:4693
16:04:29.247 mimeEncrypt.js: caught exception: Error
Message: 'failure in finishCryptoEncapsulation'
File: chrome://openpgp/content/modules/mimeEncrypt.jsm
Line: 588
Stack: finishCryptoEncapsulation@chrome://openpgp/content/modules/mimeEncrypt.jsm:588:15
CompleteGenericSendMessage@chrome://messenger/content/messengercompose/MsgComposeCommands.js:4685:17
GenericSendMessage@chrome://messenger/content/messengercompose/MsgComposeCommands.js:4587:29
SendMessage@chrome://messenger/content/messengercompose/MsgComposeCommands.js:4855:21
doCommand@chrome://messenger/content/messengercompose/MsgComposeCommands.js:967:11
doCommand@chrome://messenger/content/messengercompose/MsgComposeCommands.js:1163:9
goDoCommand@chrome://global/content/globalOverlay.js:101:18
oncommand@chrome://messenger/content/messengercompose/messengercompose.xhtml:1:12

16:04:29.247
Error: failure in finishCryptoEncapsulation mimeEncrypt.jsm:588:15
16:04:32.334
GenericSendMessage FAILED: [Exception... "Component returned failure code: 0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS) [nsIMsgCompose.SendMsg]" nsresult: "0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)" location: "JS frame :: chrome://messenger/content/messengercompose/MsgComposeCommands.js :: CompleteGenericSendMessage :: line 4685" data: no]

Comment 5

[Kai Engert [:KaiE:]]

To make sense of the line numbers, I need to know what version you were running.
I assume you were on 78.2.2 (released 2020-09-10), because that was the latest version the day you posted the logs.

Comment 6

[gunter.zielke]

That is correct - 78.2.2
And I have some more info that might give some more insight:
I started with a fresh install of 78.2.2 without any migration. Then I created a new key for the gunter.zielke@gm account and manually imported the key for the gz0001@gm account after exporting it from Kleopatra.
I was then able to send an encrypted email from gz0001 to gunter.zielke.
Then I imported manually the gunter.zielke key from Kleopatra and I was no longer able to send those encrypted emails.
I then removed the just imported gunter.zielke key so that I ended up with only the initially created key for gunter.zielke - and I still could not send encrypted emails.
I then deleted the initial key from gunter.zielke and created a new one. tried to send from gz0001 to gunter.zielke - failed.
Then, just for the fun of it, tried to send from gunter.zielke with the newly created key to gz0001 - that failed too.

Comment 7

[Kai Engert [:KaiE:]]

I can see at which action it is failing:

It is trying to prepare encryption to yourself. (We encrypt to yourself, in addition to the recipients, to allow you to decrypt the copy of the message in the send folder.)

I attempts to find a suitable encryption (sub) key related to the key ID that you have configured for yourself, but it cannot find a key usable for encryption.

Comment 8

[Kai Engert [:KaiE:]]

(In reply to gunter.zielke from comment #6)

I started with a fresh install of 78.2.2 without any migration. Then I created a new key for the gunter.zielke@gm account

You created this key, inside Thunderbird, let's call it K1.

and manually imported the key for the gz0001@gm account after exporting it from Kleopatra.
I was then able to send an encrypted email from gz0001 to gunter.zielke.

Secret key or just the personal key for gz0001 ?
Let's call it K2.

Then I imported manually the gunter.zielke key from Kleopatra and I was no longer able to send those encrypted emails.

You say you imported a different key for gunter.zielke, so now you had 3 secret keys in Thunderbird.

I assume this imported key was for the same email address as the new key you created inside Thunderbird.

I assume you used the default selection "accept as personal key" at the time of import.

Let's call this K3.

I assume K1 and K3 are different keys with different fingerprints.

It's not clear if you have changed the configured personal openpgp key for the email account to K3.

If you have kept the account's key selection at K1, then nothing should have changed.

If have have kept the account's key selection at K1, but you could no longer encrypt to yourself with K1, then it would mean that there is some kind of conflict/similarity between K1 and K3 that the RNP library cannot handle (just my theory).

If you have changed the accounts key select to use K3, then K3 might not have a valid encryption key?

I then removed the just imported gunter.zielke key so that I ended up with only the initially created key for gunter.zielke - and I still could not send encrypted emails.

You said you removed K3. If previously K3 was selected for your email account (in account settings, end-to-end encryption pane), then removing K3 might not have updated the account configuration.

If the account configuration still pointed to K3, then the code might have searched for K3 (for encrypt to yourself), and failed to find any key.

I then deleted the initial key from gunter.zielke and created a new one. tried to send from gz0001 to gunter.zielke - failed.

Now you no longer had any key for your sender email address, so it couldn't work.

Then, just for the fun of it, tried to send from gunter.zielke with the newly created key to gz0001 - that failed too.

You said you imported K1 again from a backup?

It's difficult to say what went wrong. I think as a first step, each time after you have deleted a key, please go back to account settings, to check that a present key is selected as your email account's openpgp key.

When in doubt, please close the account settings pane, and open it again (just to make sure the listed keys refresh correctly).

Comment 9

[Kai Engert [:KaiE:]]

In short:

Each time you delete a key in key manager, please check that the account settings configuration still has a correct openpgp key configured.

It's necessary to have your own openpgp key configured. It's necessary to be able to send encrypted emails, because Thunderbird wants to encrypt to yourself, in addition to other recipients.

If you have made sure this is correctly configured, then we must try to analyze potential overlaps of K1 and K3. Please explain if there are any special properties of K3, like offline primary key, expired subkeys etc.

Comment 10

[gunter.zielke]

With the hints, you gave me above, I ran the following tests:

• get rid of all TB keys in the manager
• generate key pair K1 for email gunter.zielke (E1)
• generate key pair K2 for email gz0001 (E2)
• sending encrypted email from K2 to K1 is successful
• import key pair K3 for email E2 which was exported from Kleopatra (on import I entered the password as I had it set for K3 in enigmail)
• sending encrypted email from K2 to K1 is successful (without changing the private key used - so it was still using K2)
• changed E2 to use K3
• sending mail failed
  So, it appears that there is something wrong with the imported key K3 because if I go back to K2 for E2, I can send encrypted messages again.

From the console I get the following output:
Debug Tab:
00:37:16.983 getCryptParams parameters: from=0x002A601E8FDE6FF6, to=[e-mail], bcc=, hash=SHA256, flags=20673, ascii=0, errorObj=
Object { value: "" }
, logObj=
Object { }
encryption.jsm:86:13
00:37:16.983 getCryptParams, got: to=[e-mail], bcc= encryption.jsm:125:13
00:37:16.983 getCryptParams returning: encryption.jsm:178:13
00:37:16.983
Object { sender: "0x002A601E8FDE6FF6", sign: true, signatureHash: "SHA256", sigTypeClear: false, sigTypeDetached: true, encrypt: false, encryptToSender: false, armor: true, senderKeyIsExternal: false, to: (1) […], … }
encryption.jsm:179:13
00:37:17.274 sendFlags=000050c1 encryption.jsm:358:13
00:37:17.274 getCryptParams parameters: from=0x002A601E8FDE6FF6, to=[e-mail], bcc=, hash=SHA256, flags=20674, ascii=0, errorObj=
Object { value: "" }
, logObj=
Object { }
encryption.jsm:86:13
00:37:17.274 getCryptParams, got: to=[e-mail], bcc= encryption.jsm:125:13
00:37:17.274 getCryptParams returning: encryption.jsm:178:13
00:37:17.274
Object { sender: "0x002A601E8FDE6FF6", sign: false, signatureHash: "", sigTypeClear: false, sigTypeDetached: false, encrypt: true, encryptToSender: true, armor: true, senderKeyIsExternal: false, to: (1) […], … }
encryption.jsm:179:13
00:37:17.300 sendFlags=000050c2 encryption.jsm:358:13
00:37:17.300 Error: failure in finishCryptoEncapsulation
finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:588
CompleteGenericSendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4685
GenericSendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4587
SendMessage chrome://messenger/content/messengercompose/MsgComposeCommands.js:4855
doCommand chrome://messenger/content/messengercompose/MsgComposeCommands.js:967
doCommand chrome://messenger/content/messengercompose/MsgComposeCommands.js:1163
goDoCommand chrome://global/content/globalOverlay.js:101
oncommand chrome://messenger/content/messengercompose/messengercompose.xhtml:1
mimeEncrypt.jsm:603:15

Error Tab:
00:37:17.300 mimeEncrypt.js: caught exception: Error
Message: 'failure in finishCryptoEncapsulation'
File: chrome://openpgp/content/modules/mimeEncrypt.jsm
Line: 588
Stack: finishCryptoEncapsulation@chrome://openpgp/content/modules/mimeEncrypt.jsm:588:15
CompleteGenericSendMessage@chrome://messenger/content/messengercompose/MsgComposeCommands.js:4685:17
GenericSendMessage@chrome://messenger/content/messengercompose/MsgComposeCommands.js:4587:29
SendMessage@chrome://messenger/content/messengercompose/MsgComposeCommands.js:4855:21
doCommand@chrome://messenger/content/messengercompose/MsgComposeCommands.js:967:11
doCommand@chrome://messenger/content/messengercompose/MsgComposeCommands.js:1163:9
goDoCommand@chrome://global/content/globalOverlay.js:101:18
oncommand@chrome://messenger/content/messengercompose/messengercompose.xhtml:1:12

00:37:17.300
Error: failure in finishCryptoEncapsulation mimeEncrypt.jsm:588:15
00:37:20.836
GenericSendMessage FAILED: [Exception... "Component returned failure code: 0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS) [nsIMsgCompose.SendMsg]" nsresult: "0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)" location: "JS frame :: chrome://messenger/content/messengercompose/MsgComposeCommands.js :: CompleteGenericSendMessage :: line 4685" data: no] MsgComposeCommands.js:4693

Comment 11

[Kai Engert [:KaiE:]]

(In reply to Kai Engert (:KaiE:) from comment #9)

Please explain if there are any special properties of K3, like offline primary key, expired subkeys etc.

Comment 12

[gunter.zielke]

I created that key with Kleopatra some five years ago and I don't think I did anything out of the ordinary. I have no problem giving you that key to take a look. I only needed that key to maintaining access to a bigger number of emails, but in the meantime, I managed to find a solution to permanently decode those messages, so I don't really need that key anymore. I will send the key to you in an email to [e-mail]

Comment 13

[Kai Engert [:KaiE:]]

Thank you for sending me the key.
The encryption subkey of your key has expired in February 2020.
That's why TB cann

You can see this yourself, by using OpenPGP key manager, open the details of your key, and click the "structure" tab.

I agree that Thunderbird should show a better error message in this scenario.

Comment 14

[Kai Engert [:KaiE:]]

I think this needs a high priority, because we show misleading information about the usability/validity of keys, when displaying the details of a key, or when displaying the security information for a message composer.

We currently use different code for computing the user feedback and when sending the message. The latter is more strict.

Comment 15

[gunter.zielke]

Thank you Kai, that was extremely helpful. TB78 does not yet allow to change the expiration of the subkey, but I found good instructions on how to change this at https://sites.lafayette.edu/newquisk/archives/504 (just in case somebody runs into this problem and lands here).
I changed the expiration date of my key from the command line and re-imported it into TB78, and - voila! - mail sending works!

[I have worked in software development for many years, so I can really appreciate your help here :-) ]

Comment 16

[Kai Engert [:KaiE:]]

I'd suggest to fix this bug in the following way.

We calculate the "effective" expiration timestamps for both "can use for encryption" and "can use for signing". If the best subkey for a purpose expires sooner than the primary key, than that subkey's expiration is the effective expiration date for that purpose.

Whenever we display or check the expiration date for a correspondent's public key, we use the effective expiration timestamp for the encryption purpose.

And when displaying or checking the expiration of a personal key, we require that it must be usable for both signing and expiration, so the effective expiration is the earliest of the signing and encryption key's effective expiration.

The effective dates should be used when showing the expiration date in the key manager list, in the key details dialog, when checking and reporting the validity of correspondent keys in composer and message security info, and also when checking a personal key.

Comment 17

[Kai Engert [:KaiE:]]

Attachment: Bug 1665497 - Use effective expiration date instead of primary key's expiration. r=PatrickBrunschwig

Comment 18

[Kai Engert [:KaiE:]]

The fix was incomplete. The code related to composer, that checks for validity of a key, also needs to consider that a subkey might be expired. There are two places that need checking, so I'm refactoring it into a comment function in keyRing.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/e60cefc36651
Use effective expiration date instead of primary key's expiration. r=PatrickBrunschwig DONTBUILD

Comment 20

[Kai Engert [:KaiE:]]

Comment on attachment 9178195 [details]
Bug 1665497 - Use effective expiration date instead of primary key's expiration. r=PatrickBrunschwig

[Approval Request Comment]
Regression caused by (bug #): no
User impact if declined: misleading behavior (failures to use some keys, despite most UI claiming it's fine)
Testing completed (on c-c, etc.): none, want beta testing
Risk to taking this patch (and alternatives if risky): medium risk, changes general OpenPGP code

Comment 21

[Rob Lemley [:rjl]]

Comment on attachment 9178195 [details]
Bug 1665497 - Use effective expiration date instead of primary key's expiration. r=PatrickBrunschwig

[Triage Comment]
By wsmwk via Matrix.

Comment 22

[Rob Lemley [:rjl]]

Thunderbird 82.0b2:
https://hg.mozilla.org/releases/comm-beta/rev/8168bed90d01

Comment 23

[Magnus Melin [:mkmelin]]

Comment on attachment 9178195 [details]
Bug 1665497 - Use effective expiration date instead of primary key's expiration. r=PatrickBrunschwig

[Approval Request Comment]
Avoid OpenPGP confusion.

Comment 24

[Rob Lemley [:rjl]]

Comment on attachment 9178195 [details]
Bug 1665497 - Use effective expiration date instead of primary key's expiration. r=PatrickBrunschwig

[Triage Comment]
Approved for 78.3.3 per wsmwk via Matrix

Comment 25

[Rob Lemley [:rjl]]

Thunderbird 78.3.3:
https://hg.mozilla.org/releases/comm-esr78/rev/558ad432702b