1741604 - OpenPGP fails to sign/encrypt new messages in Fedora 35 (decryption works correctly)

Status: RESOLVED WORKSFORME

(MailNews Core :: Security: OpenPGP, defect)

Description

[Mat Kowalski]

Steps to reproduce:

When trying to send a new message that is either signed or signed+encrypted with OpenGPG using External GnuPG Key, the "Sending of the message failed" error message appears and the message cannot be sent.

The only way to proceed is to disable the encryption and signature.

The error does not affect decryption of the messages that are already in the mailbox (those can be decrypted correctly). It only disables the ability to encrypt.

Environment

• Fedora 35
• Thunderbird 91.3.0 (20211109081331), RPM-based installation
• gpg (GnuPG) 2.3.3

This is a regression between Fedora 34 and Fedora 35. Same setup without any configuration changed worked correctly before upgrading to F35.

Actual results:

enigdbug shows the following output

2021-11-17 13:53:40.884 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.sendMessageListener
2021-11-17 13:53:40.885 [DEBUG] keyRing.jsm: getKeyById: 0x759E193F489DC659
2021-11-17 13:53:40.885 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.prepareSendMsg: msgSendType=0, gSendSigned=true, gSendEncrypted=false
2021-11-17 13:53:40.886 [DEBUG] encryption.jsm: determineOwnKeyUsability: sendFlags=36865, sender=759E193F489DC659
2021-11-17 13:53:40.886 [DEBUG] keyRing.jsm: getKeyById: 759E193F489DC659
2021-11-17 13:53:40.887 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.determineMsgRecipients: currentId=[nsIMsgIdentity: id1], [e-mail]
2021-11-17 13:53:40.887 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.determineMsgRecipients:gMsgCompose=[xpconnect wrapped nsIMsgCompose]
2021-11-17 13:53:40.889 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.attachOwnKey: 759E193F489DC659
2021-11-17 13:53:40.889 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.extractAndAttachKey: 
2021-11-17 13:53:40.889 [DEBUG] keyRing.jsm: EnigmailKeyRing.extractKey: 0x759E193F489DC659
2021-11-17 13:53:40.906 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.prepareSecurityInfo(): Using PGP/MIME, flags=37057
2021-11-17 13:53:40.907 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.prepareSecurityInfo: oldSecurityInfo = [xpconnect wrapped nsIMsgComposeSecure]
2021-11-17 13:53:40.907 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.prepareSecurityInfo: securityInfo = [object Object]
2021-11-17 13:53:40.907 [DEBUG] enigmailMsgComposeOverlay.js: Enigmail.msg.prepareSendMsg: enabled forceMsgEncoding
2021-11-17 13:53:40.924 [DEBUG] mimeEncrypt.js: requiresCryptoEncapsulation
2021-11-17 13:53:40.925 [DEBUG] mimeEncrypt.js: beginCryptoEncapsulation
2021-11-17 13:53:40.925 [DEBUG] mimeEncrypt.js: startCryptoHeaders
2021-11-17 13:53:40.925 [DEBUG] mimeEncrypt.js: signedHeaders1
2021-11-17 13:53:40.926 [DEBUG] funcs.jsm: getHeaderData: multipart/mixed; boundary="------------L1ADrFI2wwtfeSnsXwHWI0if"
2021-11-17 13:53:40.926 [DEBUG] funcs.jsm: getHeaderData: boundary = "------------L1ADrFI2wwtfeSnsXwHWI0if"
2021-11-17 13:53:40.947 [DEBUG] mimeEncrypt.js: finishCryptoEncapsulation
2021-11-17 13:53:40.947 [DEBUG] encryption.jsm: encryptMessageStart: uiFlags=9, from 0x759E193F489DC659 to m@REDACTEDz, hashAlgorithm=SHA256 (000090c1)
2021-11-17 13:53:40.947 [DEBUG] encryption.jsm: getCryptParams: hashAlgorithm=SHA256
2021-11-17 13:53:41.576 [DEBUG] mimeEncrypt.js: finishCryptoEncapsulation: exitCode = -1
2021-11-17 13:53:41.576 [ERROR] mimeEncrypt.js: caught exception: Error
Message: 'failure in finishCryptoEncapsulation, exitCode: -1'
File:    chrome://openpgp/content/modules/mimeEncrypt.jsm
Line:    580
Stack:   finishCryptoEncapsulation@chrome://openpgp/content/modules/mimeEncrypt.jsm:580:15
createMessageFile@resource:///modules/MimeMessage.jsm:86:27

No errors in SELinux or any systemd unit at the time when the error happens.

Comment 1

[Kai Engert [:KaiE:]]

I'm not up to date what Fedora ships.

Does Fedora ship RNP and Botan ? It's required for public key operations like encryption. (Only secret key operations decrypt and sign will be diverted to the optionally configured external gnupg.)

Comment 2

[Kai Engert [:KaiE:]]

When in doubt about Fedora's behavior, please download a Thunderbird 91.x binary from thunderbird.net and check if that works correctly.

By performing this test, you can help us confirm if a bug might be in the code that Fedora distributes.

Comment 3

[Mat Kowalski]

When in doubt about Fedora's behavior, please download a Thunderbird 91.x binary from thunderbird.net and check if that works correctly.
By performing this test, you can help us confirm if a bug might be in the code that Fedora distributes.

Using Thunderbird 91.3.2 binary from thunderbird.net I do not see the issue, so it indeed looks like something Fedora-related.

Does Fedora ship RNP and Botan ? It's required for public key operations like encryption. (Only secret key operations decrypt and sign will be diverted to the optionally configured external gnupg.)

I'm not sure this is correct. The problematic action is signing a message which is a secret key operation. For this reason, I can't see a clear reason why decryption would work in Fedora-shipped version but signing would not.

Still, because of the successful test with 91.3.2 binary, it looks like either a regression in Fedora or something that's changed between 91.3.0 and 91.3.2

Comment 4

[Kai Engert [:KaiE:]]

If you want to ensure the problem isn't related to the version difference, you could download the official Thunderbird 91.3.0 binary from here:
https://ftp.mozilla.org/pub/thunderbird/releases/91.3.0/

Comment 5

[Kai Engert [:KaiE:]]

Well, the official 91.3.0 binary would quickly attempt to automatically update itself, so you'd have to disable that.

I'll attach an archive.

First, extract thunderbird 91.3.0
Second, extra the attached file into the thunderbird directory
(the distribution directory must be a subdirectory of the thunderbird directory)

Third, start Thunderbird.
This should prevent automatic update. (Help about will tell you that updates are disabled.)

Comment 6

[Kai Engert [:KaiE:]]

Attachment: prevent-disable-update.tar.gz

Comment 7

[Magnus Melin [:mkmelin]]

WFM per comment 3.