Closed Bug 1753214 Opened 3 years ago Closed 3 years ago

OpenPGP: Cannot sign if the secret part of the preferred signing subkey is missing

Categories

(MailNews Core :: Security: OpenPGP, defect)

Thunderbird 91
x86_64
Linux
defect

Tracking

(thunderbird_esr91+ fixed)

RESOLVED FIXED
100 Branch
Tracking Status
thunderbird_esr91 + fixed

People

(Reporter: raffi.enficiaud, Assigned: KaiE)

References

Details

Attachments

(16 files)

39.79 KB, image/png
Details
15.88 KB, text/plain
Details
13.74 KB, text/plain
Details
95.75 KB, image/png
Details
78.65 KB, image/png
Details
88.93 KB, image/png
Details
97.22 KB, image/png
Details
102.79 KB, image/png
Details
89.34 KB, image/png
Details
48 bytes, text/x-phabricator-request
Details | Review
125.18 KB, image/png
Details
74.50 KB, image/png
Details
89.34 KB, image/png
Details
95.85 KB, image/png
Details
64.52 KB, image/png
Details
48 bytes, text/x-phabricator-request
Details | Review

Steps to reproduce:

  1. import my GPG key and configure my account to use it
  2. prepare an email, activate the signing of the message, disable the encryption of the subject
  3. send the email

Actual results:

  1. A popup/modale window with a message indicating that there was a failure while sending the email
  2. The email has not been sent
  3. I was sent back to editing my email
  4. Several error messages in the "error window", see below

The error messages related to this failure (almost verbatim: I changed my email address):

14:46:51,656 in getEncryptionFlags, gSendEncrypted=false, gSendSigned=true enigmailMsgComposeOverlay.js:1610:13
14:46:51,690 getCryptParams parameters: from=0xD4DCF656B16691C4, to=redacted@redacted.redacted, bcc=, hash=SHA256, flags=193, ascii=0, errorObj= 
Object { value: "" }
 , logObj= 
Object {  }
encryption.jsm:73:13
14:46:51,692 getCryptParams, got: to=redacted@redacted.redacted, bcc= encryption.jsm:113:13
14:46:51,693 getCryptParams returning: encryption.jsm:190:13
14:46:51,693
Object { sender: "0xD4DCF656B16691C4", sign: true, signatureHash: "SHA256", sigTypeClear: false, sigTypeDetached: true, encrypt: false, encryptToSender: false, armor: true, senderKeyIsExternal: false, to: (1) […], … }
encryption.jsm:191:13
14:46:51,733 CryptoAPI.sync() failed result:  Error: rnp_op_sign_add_signature failed
    encryptAndOrSign chrome://openpgp/content/modules/RNP.jsm:2429
    sync chrome://openpgp/content/modules/cryptoAPI/interface.js:56
    encryptMessageStart chrome://openpgp/content/modules/encryption.jsm:410
    finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:559
    createMessageFile resource:///modules/MimeMessage.jsm:86
interface.js:46:17
14:46:51,734 sendFlags=000000c1 encryption.jsm:426:13
14:46:51,735 Error: failure in finishCryptoEncapsulation, exitCode: -1
    finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:580
    createMessageFile resource:///modules/MimeMessage.jsm:86
mimeEncrypt.jsm:597:15
14:46:51,735
mailnews.send: 
Exception { name: "NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS", message: "[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]'[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]' when calling method: [nsIMsgComposeSecure::finishCryptoEncapsulation]", result: 2153185313, filename: "resource:///modules/MimeMessage.jsm", lineNumber: 86, columnNumber: 0, data: XPCWrappedNative_NoHelper, stack: "createMessageFile@resource:///modules/MimeMessage.jsm:86:27\n", location: XPCWrappedNative_NoHelper }
MessageSend.jsm:131:27
14:46:51,735 mimeEncrypt.js: caught exception: Error
Message: 'failure in finishCryptoEncapsulation, exitCode: -1'
File:    chrome://openpgp/content/modules/mimeEncrypt.jsm
Line:    580
Stack:   finishCryptoEncapsulation@chrome://openpgp/content/modules/mimeEncrypt.jsm:580:15
createMessageFile@resource:///modules/MimeMessage.jsm:86:27


14:46:51,735
Error: failure in finishCryptoEncapsulation, exitCode: -1 mimeEncrypt.jsm:580:15
14:46:51,737
mailnews.send: Sending failed; , exitCode=2153185313, originalMsgURI= MessageSend.jsm:330:27
14:46:51,759 Error: rnp_op_sign_add_signature failed RNP.jsm:2429:17

Expected results:

  1. email sent with proper signature
  2. no error message in the error console
Attached image key structure

The exact version of Thunderbird is 91.5.0, installed on Ubuntu 21.04 from the regular package manager (not snap). I have 91.5.1 on macOS and it works without issue. I imported my GPG key and it contains several subkeys (maybe relevant) while on macOS I imported only the relevant subkeys.

The structure of the key used is attached to the ticket.

Component: Untriaged → Security: OpenPGP
Product: Thunderbird → MailNews Core

You are using the same Thunderbird version on macOS and Ubuntu, you use the same personal/secret key on both machines, and you perform the same action on both machines, but only TB on Ubuntu fails?

This is strange. Could you attempt to run an official Linux from thunderbird.net and check if it has the same problem? Maybe it's a bug in the Ubuntu package?

Flags: needinfo?(raffi.enficiaud)
Dear @Kai I am using 91.5.0 on Ubuntu and 91.5.1 on macOS. Here is the installed version on Ubuntu ``` raffi@ze-computer:~$ dpkg -l | grep thunderbird ii thunderbird 1:91.5.0+build1-0ubuntu0.20.04.1 ii thunderbird-locale-en 1:91.5.0+build1-0ubuntu0.20.04.1 ii thunderbird-locale-en-gb 1:91.5.0+build1-0ubuntu0.20.04.1 ii thunderbird-locale-en-us 1:91.5.0+build1-0ubuntu0.20.04.1 ``` I just did the following at your request: 1. downloaded 91.5.1 for Linux, deflated it in `/data/tmp`, 1. made a copy of `~/.thunderbird` into `/data/tmp` as well 1. and ran Thunderbird from the command line with `'/data/tmp/thunderbird/thunderbird' --profile /data/tmp/.thunderbird/7x2b05m2.default/`. Unfortunately the result is exactly the same. I did the same run with the Throubleshooting mode "on". Here is the trace of the Error console: ``` 17:42:06.667 Unexpected event profile-after-change URLQueryStrippingListService.jsm:224 17:42:06.921 Unknown Collection "thunderbird/query-stripping" RemoteSettingsClient.jsm:160 17:42:15.974 Successfully loaded OpenPGP library librnp.so version 0.15.2+git20210806.dd923a4e.MZLA from /data/tmp/thunderbird/librnp.so RNPLib.jsm:92:15 17:42:16.172 Found 7 public keys and 3 secret keys (3 protected, 0 unprotected) RNPLib.jsm:288:15 17:42:16.286 Trying to load /data/tmp/thunderbird/libotr.so OTRLib.jsm:64:11 17:42:16.287 Successfully loaded OTR library /data/tmp/thunderbird/libotr.so OTRLib.jsm:72:13 17:42:16.353 tb.account.size_on_disk - Trying to set an unsigned scalar to a negative number. 72 17:42:16.357 services.settings: thunderbird/hijack-blocklists has signature disabled RemoteSettingsClient.jsm:1027 17:42:17.068 Failed to create WebGL context: WebGL is currently disabled. 2 Troubleshoot.jsm:663:21 17:42:17.360 Relative positioning of table rows and row groups is now supported. This site may need to be updated because it may depend on this feature having no effect. preferences.js:261:2 17:42:18.328 Calendar: [calCachedCalendar] replay action failed: <unknown>, uri=https://calendar.google.com/calendar/ical/REDACTED@REDACTED/private-936d3e50d9738bfe57adba0b5d616f99/basic.ics, result=2147500037, operation=[xpconnect wrapped calIOperation] calCachedCalendar.js:346 17:43:07.548 in getEncryptionFlags, gSendEncrypted=false, gSendSigned=true enigmailMsgComposeOverlay.js:1610:13 17:43:07.592 getCryptParams parameters: from=0xD4DCF656B1LASTPARTREDACTED, to=REDACTED@REDACTED, bcc=, hash=SHA256, flags=193, ascii=0, errorObj= Object { value: "" } , logObj= Object { } encryption.jsm:73:13 17:43:07.593 getCryptParams, got: to=REDACTED@REDACTED, bcc= encryption.jsm:113:13 17:43:07.593 getCryptParams returning: encryption.jsm:190:13 17:43:07.593 Object { sender: "0xD4DCF656B1LASTPARTREDACTED", sign: true, signatureHash: "SHA256", sigTypeClear: false, sigTypeDetached: true, encrypt: false, encryptToSender: false, armor: true, senderKeyIsExternal: false, to: (1) […], … } encryption.jsm:191:13 17:43:07.641 CryptoAPI.sync() failed result: Error: rnp_op_sign_add_signature failed encryptAndOrSign chrome://openpgp/content/modules/RNP.jsm:2429 sync chrome://openpgp/content/modules/cryptoAPI/interface.js:56 encryptMessageStart chrome://openpgp/content/modules/encryption.jsm:410 finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:559 createMessageFile resource:///modules/MimeMessage.jsm:86 interface.js:46:17 17:43:07.642 sendFlags=000000c1 encryption.jsm:426:13 17:43:07.642 Error: failure in finishCryptoEncapsulation, exitCode: -1 finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:580 createMessageFile resource:///modules/MimeMessage.jsm:86 mimeEncrypt.jsm:597:15 17:43:07.643 mailnews.send: Exception { name: "NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS", message: "[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]'[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]' when calling method: [nsIMsgComposeSecure::finishCryptoEncapsulation]", result: 2153185313, filename: "resource:///modules/MimeMessage.jsm", lineNumber: 86, columnNumber: 0, data: XPCWrappedNative_NoHelper, stack: "createMessageFile@resource:///modules/MimeMessage.jsm:86:27\n", location: XPCWrappedNative_NoHelper } MessageSend.jsm:131:27 17:43:07.643 mailnews.send: Sending failed; , exitCode=2153185313, originalMsgURI= MessageSend.jsm:332:27 17:43:07.643 mimeEncrypt.js: caught exception: Error Message: 'failure in finishCryptoEncapsulation, exitCode: -1' File: chrome://openpgp/content/modules/mimeEncrypt.jsm Line: 580 Stack: finishCryptoEncapsulation@chrome://openpgp/content/modules/mimeEncrypt.jsm:580:15 createMessageFile@resource:///modules/MimeMessage.jsm:86:27 17:43:07.643 Error: failure in finishCryptoEncapsulation, exitCode: -1 mimeEncrypt.jsm:580:15 17:43:07.680 Error: rnp_op_sign_add_signature failed RNP.jsm:2429:17 ``` Concerning the keys, the ones I have on macOS are two subkeys of the (full) PGP key I have on Ubuntu. I don't know if this is of any relevance: I don't have Chrome (I am seeing various `chrome://`) but I do have Chromium installed as a Snap package: ``` raffi@raffi-work:~$ snap list Name Version Rev Tracking Publisher Notes 0ad 0.0.25b-alpha 251 latest/stable play0ad✓ - bare 1.0 5 latest/stable canonical✓ base chromium 98.0.4758.80 1899 latest/stable canonical✓ - core 16-2.54.2 12603 latest/stable canonical✓ core core18 20211215 2284 latest/stable canonical✓ base core20 20220114 1328 latest/stable canonical✓ base gnome-3-28-1804 3.28.0-19-g98f9e67.98f9e67 161 latest/stable canonical✓ - gnome-3-34-1804 0+git.3556cb3 77 latest/stable canonical✓ - gnome-3-38-2004 0+git.cd626d1 87 latest/stable canonical✓ - gnome-system-monitor 41.0-5-g91e67f7982 174 latest/stable/… canonical✓ - gtk-common-themes 0.1-59-g7bca6ae 1519 latest/stable/… canonical✓ - gtk2-common-themes 0.1 13 latest/stable canonical✓ - librepcb 0.1.6-1 621 latest/stable librepcb✓ - p7zip-desktop 16.02.2 220 latest/stable ernytech - snap-store 3.38.0-66-gbd5b8f7 558 latest/stable/… canonical✓ - telegram-desktop 3.4.3 3544 latest/stable telegram.desktop - ``` If relevant I am also joining the troubleshooting information: ```

(sorry for the duplicate message, something bad happened with Bugzilla while pasting the troubleshooting info. I am reposting here with correct formatting).

Dear @Kai

I am using 91.5.0 on Ubuntu and 91.5.1 on macOS. Here is the installed version on Ubuntu

raffi@ze-computer:~$ dpkg -l | grep thunderbird
ii  thunderbird                                       1:91.5.0+build1-0ubuntu0.20.04.1
ii  thunderbird-locale-en                             1:91.5.0+build1-0ubuntu0.20.04.1
ii  thunderbird-locale-en-gb                          1:91.5.0+build1-0ubuntu0.20.04.1
ii  thunderbird-locale-en-us                          1:91.5.0+build1-0ubuntu0.20.04.1

I just did the following at your request:

  1. downloaded 91.5.1 for Linux, deflated it in /data/tmp,
  2. made a copy of ~/.thunderbird into /data/tmp as well
  3. and ran Thunderbird from the command line with '/data/tmp/thunderbird/thunderbird' --profile /data/tmp/.thunderbird/7x2b05m2.default/.

Unfortunately the result is exactly the same. I did the same run with the Throubleshooting mode "on". Here is the trace of the Error console:

17:42:06.667 Unexpected event profile-after-change URLQueryStrippingListService.jsm:224
17:42:06.921 Unknown Collection "thunderbird/query-stripping" RemoteSettingsClient.jsm:160
17:42:15.974 Successfully loaded OpenPGP library librnp.so version 0.15.2+git20210806.dd923a4e.MZLA from /data/tmp/thunderbird/librnp.so RNPLib.jsm:92:15
17:42:16.172 Found 7 public keys and 3 secret keys (3 protected, 0 unprotected) RNPLib.jsm:288:15
17:42:16.286 Trying to load /data/tmp/thunderbird/libotr.so OTRLib.jsm:64:11
17:42:16.287 Successfully loaded OTR library /data/tmp/thunderbird/libotr.so OTRLib.jsm:72:13
17:42:16.353 tb.account.size_on_disk - Trying to set an unsigned scalar to a negative number. 72
17:42:16.357 services.settings: thunderbird/hijack-blocklists has signature disabled RemoteSettingsClient.jsm:1027
17:42:17.068 Failed to create WebGL context: WebGL is currently disabled. 2 Troubleshoot.jsm:663:21
17:42:17.360 Relative positioning of table rows and row groups is now supported. This site may need to be updated because it may depend on this feature having no effect. preferences.js:261:2
17:42:18.328 Calendar: [calCachedCalendar] replay action failed: <unknown>, uri=https://calendar.google.com/calendar/ical/REDACTED@REDACTED/private-936d3e50d9738bfe57adba0b5d616f99/basic.ics, result=2147500037, operation=[xpconnect wrapped calIOperation] calCachedCalendar.js:346
17:43:07.548 in getEncryptionFlags, gSendEncrypted=false, gSendSigned=true enigmailMsgComposeOverlay.js:1610:13
17:43:07.592 getCryptParams parameters: from=0xD4DCF656B1LASTPARTREDACTED, to=REDACTED@REDACTED, bcc=, hash=SHA256, flags=193, ascii=0, errorObj= 
Object { value: "" }
 , logObj= 
Object {  }
encryption.jsm:73:13
17:43:07.593 getCryptParams, got: to=REDACTED@REDACTED, bcc= encryption.jsm:113:13
17:43:07.593 getCryptParams returning: encryption.jsm:190:13
17:43:07.593
Object { sender: "0xD4DCF656B1LASTPARTREDACTED", sign: true, signatureHash: "SHA256", sigTypeClear: false, sigTypeDetached: true, encrypt: false, encryptToSender: false, armor: true, senderKeyIsExternal: false, to: (1) […], … }
encryption.jsm:191:13
17:43:07.641 CryptoAPI.sync() failed result:  Error: rnp_op_sign_add_signature failed
    encryptAndOrSign chrome://openpgp/content/modules/RNP.jsm:2429
    sync chrome://openpgp/content/modules/cryptoAPI/interface.js:56
    encryptMessageStart chrome://openpgp/content/modules/encryption.jsm:410
    finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:559
    createMessageFile resource:///modules/MimeMessage.jsm:86
interface.js:46:17
17:43:07.642 sendFlags=000000c1 encryption.jsm:426:13
17:43:07.642 Error: failure in finishCryptoEncapsulation, exitCode: -1
    finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:580
    createMessageFile resource:///modules/MimeMessage.jsm:86
mimeEncrypt.jsm:597:15
17:43:07.643 mailnews.send: 
Exception { name: "NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS", message: "[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]'[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]' when calling method: [nsIMsgComposeSecure::finishCryptoEncapsulation]", result: 2153185313, filename: "resource:///modules/MimeMessage.jsm", lineNumber: 86, columnNumber: 0, data: XPCWrappedNative_NoHelper, stack: "createMessageFile@resource:///modules/MimeMessage.jsm:86:27\n", location: XPCWrappedNative_NoHelper }
MessageSend.jsm:131:27
17:43:07.643 mailnews.send: Sending failed; , exitCode=2153185313, originalMsgURI= MessageSend.jsm:332:27
17:43:07.643 mimeEncrypt.js: caught exception: Error
Message: 'failure in finishCryptoEncapsulation, exitCode: -1'
File:    chrome://openpgp/content/modules/mimeEncrypt.jsm
Line:    580
Stack:   finishCryptoEncapsulation@chrome://openpgp/content/modules/mimeEncrypt.jsm:580:15
createMessageFile@resource:///modules/MimeMessage.jsm:86:27


17:43:07.643 Error: failure in finishCryptoEncapsulation, exitCode: -1 mimeEncrypt.jsm:580:15
17:43:07.680 Error: rnp_op_sign_add_signature failed RNP.jsm:2429:17

Concerning the keys, the ones I have on macOS are two subkeys of the (full) PGP key I have on Ubuntu.
I don't know if this is of any relevance: I don't have Chrome (I am seeing various chrome://) but I do have Chromium installed as a Snap package:

raffi@raffi-work:~$ snap list 
Name                  Version                     Rev    Tracking         Publisher         Notes
0ad                   0.0.25b-alpha               251    latest/stable    play0ad✓          -
bare                  1.0                         5      latest/stable    canonical✓        base
chromium              98.0.4758.80                1899   latest/stable    canonical✓        -
core                  16-2.54.2                   12603  latest/stable    canonical✓        core
core18                20211215                    2284   latest/stable    canonical✓        base
core20                20220114                    1328   latest/stable    canonical✓        base
gnome-3-28-1804       3.28.0-19-g98f9e67.98f9e67  161    latest/stable    canonical✓        -
gnome-3-34-1804       0+git.3556cb3               77     latest/stable    canonical✓        -
gnome-3-38-2004       0+git.cd626d1               87     latest/stable    canonical✓        -
gnome-system-monitor  41.0-5-g91e67f7982          174    latest/stable/…  canonical✓        -
gtk-common-themes     0.1-59-g7bca6ae             1519   latest/stable/…  canonical✓        -
gtk2-common-themes    0.1                         13     latest/stable    canonical✓        -
librepcb              0.1.6-1                     621    latest/stable    librepcb✓         -
p7zip-desktop         16.02.2                     220    latest/stable    ernytech          -
snap-store            3.38.0-66-gbd5b8f7          558    latest/stable/…  canonical✓        -
telegram-desktop      3.4.3                       3544   latest/stable    telegram.desktop  -

If relevant I am also attaching the troubleshooting information.

Flags: needinfo?(raffi.enficiaud)

... and sorry for the title: it is Ubuntu 20.04.

Summary: Thunderbird failed to send message when GPG signing is on: Ubuntu 21.04 → Thunderbird failed to send message when GPG signing is on: Ubuntu 20.04
OS: Unspecified → Linux
Hardware: Unspecified → x86_64

(In reply to raffi.enficiaud from comment #4)

Concerning the keys, the ones I have on macOS are two subkeys of the (full)
PGP key I have on Ubuntu.

You say your Ubuntu and macOS machines have a different set of keys installed in Thunderbird. That's very relevant information.

You say you have the full key (all secret keys) installed on Ubuntu, and Ubuntu isn't working?

You say you have only a subset of keys installed on macOS, and macOS is working? With subset, does that mean, only for the subkeys are the secret keys installed?

I don't know if this is of any relevance: I don't have Chrome (I am seeing
various chrome://) but I do have Chromium installed as a Snap package:

Not relevant, the term chrome doesn't refer to the chromium browser, rather chrome refers to the user interface elements of the application (not rendered messages).

Are you using an "Thunderbird with external gnupg" configuration?

(In reply to Kai Engert (:KaiE:) from comment #8)

Are you using an "Thunderbird with external gnupg" configuration?

No, you aren't, because TB calls an internal RNP function for signing.
(Just a clarification for myself.)

I would add that most likely rnp_op_sign_add_signature() fails due the absence of corresponding secret key. Do you have your secret key imported to the Thunderbird, and don't have any uncommon configuration like offline primary signing key, absence of signing secret subkey or so on?
gpg --list-packets your-key.asc could be helpful in these investigations.

Current TB implementation attempts to prefer the use of subkeys for signing. It doesn't currently check for which subkeys the secret key is available or isn't. If TB decides to use a particular (sub)key for signing, and it isn't available, TB will fail.

Maybe RNP prints an error to the console when it fails? You could follow the instructions regarding "RNP log" from here:
https://wiki.mozilla.org/Thunderbird:OpenPGP#Debugging_.2F_Tracing

For testing I've used GnuPG to create a test key with the same structure as your key. I'm able to use it with a local 91.x build on Linux to sign a message.

Flags: needinfo?(raffi.enficiaud)

You say you have only a subset of keys installed on macOS, and macOS is working? With subset, does that mean, only for the subkeys are the secret keys installed?

Yes, more precisely I extracted 2 subkeys for Thunderbird and installed only those.

@Nickolay

gpg --list-packets your-key.asc could be helpful in these investigations.

For this:

  1. I saved the key to a file from Thunderbird key manager
  2. ran the command above

The result is in a file I will attach in the next comment.

Also, I am not seeing any complain by GPG when importing the key

raffi@raffi:/data/tmp/.thunderbird$ gpg --dry-run --import secret.asc
gpg: Total number processed: 1
gpg:       secret keys read: 1

Listing the keys shows this:

raffi@raffi:/data/tmp/.thunderbird$ gpg --show-keys --with-fingerprint --with-subkey-fingerprint secret.asc
sec   rsa2048 2014-10-23 [SC] [expires: 2022-10-31]
      EA29 15DE 9DEE 7F2C FC17  F36D D4DC F656 B166 91C4
uid                      Raffi Enficiaud <raffi.enficiaud@email1.com.redacted>
uid                      Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi@email2.com.redacted>
uid                      Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@email3.com.redacted>
uid                      [jpeg image of size 203959]
uid                      Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@email4.com.redacted>
uid           [ revoked] Raffi Enficiaud <raffi.enficiaud@email4.com.redacted>
ssb   rsa2048 2014-10-23 [E] [expires: 2022-10-31]
      EAE3 D605 8678 6846 3A63  0C94 5134 53DF C850 67FF
ssb   rsa4096 2015-06-28 [S] [expired: 2019-06-28]
      B8BE 2B4E 1905 53E3 B0B7  C414 542B 960F F636 ACBA

@Kai
Running thunderbird from the terminal after having set the RNP logging produces this:

raffi@raffi:/data/tmp/.thunderbird$ /data/tmp/thunderbird/thunderbird
IPDL protocol error: Handler returned error code!

###!!! [Parent][DispatchAsyncMessage] Error: PClientManager::Msg_ExpectFutureClientSource Processing error: message was deserialized, but the handler returned false (indicating failure)

IPDL protocol error: Handler returned error code!

###!!! [Parent][DispatchAsyncMessage] Error: PClientManager::Msg_ForgetFutureClientSource Processing error: message was deserialized, but the handler returned false (indicating failure)


###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost

raffi@raffi:/data/tmp/.thunderbird$

and I have

raffi@raffi-work:/data/tmp/.thunderbird$ env | grep RNP
RNP_LOG_CONSOLE=1

For testing I've used GnuPG to create a test key with the same structure as your key. I'm able to use it with a local 91.x build on Linux to sign a message.

Yes, I believe the fact that this is happening on Ubuntu is just incidental and the problem may certainly come from the structure of the key. As I said, it works without any issue on macOS with a subset of that key.

Flags: needinfo?(raffi.enficiaud)
Attached file list_packets output

This is the output of the list-packets command.

@Kai
when setting the RNP_LOG_CONSOLE to 1 in the previous test, I have this on the error console if of any relevance:

22:26:09,008 Unexpected event profile-after-change URLQueryStrippingListService.jsm:224
22:26:09,457 Unknown Collection "thunderbird/query-stripping" RemoteSettingsClient.jsm:160
22:26:10,714 Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience. For more help http://xhr.spec.whatwg.org/ background.js:1478:16
22:26:10,888 [Exception... "Component returned failure code: 0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH) [nsIXPCComponents_Utils.readUTF8URI]"  nsresult: "0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH)"  location: "JS frame :: resource://gre/modules/L10nRegistry.jsm :: L10nRegistry.loadSync :: line 692"  data: no] L10nRegistry.jsm:692:19
22:26:10,889 Trying to load /data/tmp/thunderbird/libotr.so OTRLib.jsm:64:11
22:26:10,891 Successfully loaded OTR library /data/tmp/thunderbird/libotr.so OTRLib.jsm:72:13
22:26:10,930 Successfully loaded OpenPGP library librnp.so version 0.15.2+git20210806.dd923a4e.MZLA from /data/tmp/thunderbird/librnp.so RNPLib.jsm:92:15
22:26:11,219 Found 7 public keys and 3 secret keys (3 protected, 0 unprotected) RNPLib.jsm:288:15
22:26:11,565 tb.account.size_on_disk - Trying to set an unsigned scalar to a negative number. 46
22:26:12,222 services.settings: thunderbird/hijack-blocklists has signature disabled RemoteSettingsClient.jsm:1027
22:26:12,615 Calendar: [calCachedCalendar] replay action failed: <unknown>, uri=https://calendar.google.com/calendar/ical/raffi.enficiaud%40email2.com.redacted/private-936d3e50d9738bfe57adba0b5d616f99/basic.ics, result=2147500037, operation=[xpconnect wrapped calIOperation] calCachedCalendar.js:346
22:26:42,281 [Exception... "Component returned failure code: 0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH) [nsIXPCComponents_Utils.readUTF8URI]"  nsresult: "0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH)"  location: "JS frame :: resource://gre/modules/L10nRegistry.jsm :: L10nRegistry.loadSync :: line 692"  data: no] L10nRegistry.jsm:692:19
22:26:42,282 [Exception... "Component returned failure code: 0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH) [nsIXPCComponents_Utils.readUTF8URI]"  nsresult: "0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH)"  location: "JS frame :: resource://gre/modules/L10nRegistry.jsm :: L10nRegistry.loadSync :: line 692"  data: no] L10nRegistry.jsm:692:19
22:26:52,879 getCryptParams parameters: from=0xD4DCF656B16691C4, to=raffi.enficiaud@email2.com.redacted, bcc=, hash=SHA256, flags=193, ascii=0, errorObj= 
Object { value: "" }
 , logObj= 
Object {  }
encryption.jsm:73:13
22:26:52,880 getCryptParams, got: to=raffi.enficiaud@email2.com.redacted, bcc= encryption.jsm:113:13
22:26:52,880 getCryptParams returning: encryption.jsm:190:13
22:26:52,880
Object { sender: "0xD4DCF656B16691C4", sign: true, signatureHash: "SHA256", sigTypeClear: false, sigTypeDetached: true, encrypt: false, encryptToSender: false, armor: true, senderKeyIsExternal: false, to: (1) […], … }
encryption.jsm:191:13
22:26:52,899 CryptoAPI.sync() failed result:  Error: rnp_op_sign_add_signature failed
    encryptAndOrSign chrome://openpgp/content/modules/RNP.jsm:2429
    sync chrome://openpgp/content/modules/cryptoAPI/interface.js:56
    encryptMessageStart chrome://openpgp/content/modules/encryption.jsm:410
    finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:559
    createMessageFile resource:///modules/MimeMessage.jsm:86
interface.js:46:17
22:26:52,924 sendFlags=000000c1 encryption.jsm:426:13
22:26:52,924 Error: failure in finishCryptoEncapsulation, exitCode: -1
    finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:580
    createMessageFile resource:///modules/MimeMessage.jsm:86
mimeEncrypt.jsm:597:15
22:26:52,924 mailnews.send: 
Exception { name: "NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS", message: "[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]'[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]' when calling method: [nsIMsgComposeSecure::finishCryptoEncapsulation]", result: 2153185313, filename: "resource:///modules/MimeMessage.jsm", lineNumber: 86, columnNumber: 0, data: XPCWrappedNative_NoHelper, stack: "createMessageFile@resource:///modules/MimeMessage.jsm:86:27\n", location: XPCWrappedNative_NoHelper }
MessageSend.jsm:131:27
22:26:52,924 mimeEncrypt.js: caught exception: Error
Message: 'failure in finishCryptoEncapsulation, exitCode: -1'
File:    chrome://openpgp/content/modules/mimeEncrypt.jsm
Line:    580
Stack:   finishCryptoEncapsulation@chrome://openpgp/content/modules/mimeEncrypt.jsm:580:15
createMessageFile@resource:///modules/MimeMessage.jsm:86:27


22:26:52,924 Error: failure in finishCryptoEncapsulation, exitCode: -1 mimeEncrypt.jsm:580:15
22:26:52,925 mailnews.send: Sending failed; , exitCode=2153185313, originalMsgURI= MessageSend.jsm:332:27
22:26:52,947 Error: rnp_op_sign_add_signature failed RNP.jsm:2429:17

Your key has first RSA secret subkey with flags which allows to sign (0x02), and second one is RSA encrypt-only subkey (flags 0x0C = 0x04 + 0x08). Additionally, second subkey is created later then the first one.
My guess is that Thunderbird by mistake picks the second subkey for the signing attempt.
Kai, could this be the case? Which algorithm do you use to select the appropriate subkey?

Btw, RNP has function rnp_key_get_default_key() which takes care of algorithms, flags and subkey creation dates to pick the appropriate subkey or primary key for operation.

Flags: needinfo?(kaie)

Quick addition: sending an encrypted email by disabling the signing works with the exact same setup.

(In reply to raffi.enficiaud from comment #13)

As I said, it works without any issue on macOS with a subset of that key.

Please avoid misunderstandings. When you say something works on macOS, but doesn't work on Ubuntu,. it's easy to conclude that there may be a hidden bug somewhere that is platform specific.

But in your specific case, it sounds like you are using different configuration on different platforms.

If you want to ensure that this isn't specific to a platform, then you could test the same key configuration on both platforms.

If the issue is with different configurations, I suggest to use wording like "it doesn't work on the configuration with a smaller number of available secret subkeys".

Flags: needinfo?(kaie)

Please avoid misunderstandings.

Sorry if this was creating a misunderstanding. I reported verbatim what I have without trying to interpret anything: an Ubuntu machine with a key, and a macOS machine with a subset of that key. I don't know the code to judge if there is a known issue or a different behaviour on those platforms.

The screenshot you have attached shows 4 keys (1 primary, 3 sub). However, the packet list you have attached only contains 3 keys - it contains the secret primary key, and it contains 2 secret subkeys. Apparently the secret key ending with ...D120 is missing.

How were you able to export this subset that excludes ...D120 ?
And why doesn't the packet list show a public key for ...D120 either? Did you export in a way that strips one of the subkeys? Or did you use an older version of your key to create the packet dump, using a version that didn't yet have the ...D120 subkey?

I think you said, the key you have on Ubuntu is the "full" key. Does that mean, on your Ubuntu machine you have a key that includes all 4 secret keys? I think so, based on your screenshot. Because in a configuration with a primary secret key missing, we show (!) in front of the "primary key" label.

(In reply to Nickolay Olshevsky from comment #16)

Your key has first RSA secret subkey with flags which allows to sign (0x02), and second one is RSA encrypt-only subkey (flags 0x0C = 0x04 + 0x08). Additionally, second subkey is created later then the first one.
My guess is that Thunderbird by mistake picks the second subkey for the signing attempt.

Can you please clarify to which subkeys you refer? It's not clear, because the screenshot and the packet list show different subkeys.

In a system that doesn't have the ..D120 subkey available (neither secret nor public, as suggested in the packet list), Thunderbird shouldn't be able to try using ..D120, and therefore it should be fine to use the single signing subkey?

If we're talking about a system that has all 4 secret keys available, why would it be a problem to select the second signing subkey? Both should work, right?

Nickolay, do you suspect that Thunderbird might try to use an encryption-only subkey for signing?

I think the code should avoid that.

We call this function with usage "sign":
https://searchfox.org/comm-esr91/source/mail/extensions/openpgp/content/modules/RNP.jsm#2184

We skip subkeys if rnp_key_allows_usage("sign") says "no".

The code shows that we attempt to use the most recently created subkey for a usage.

If you have public key for ...D120 available, then we'd try to use it. Even if the secret key for ...D120 is missing, we'd still try to use it, and fail.

In other words, if you have public subkey ...D120 available, but secret subkey ...D120 is missing, then yes, we'd fail. I didn't expect this key of configuration.

But from the packet list, it seems that you have neither public nor secret ...D120 available. In that case, the code should use subkey ...ACBA and succeed.

Now I'm noticing this here:

(In reply to raffi.enficiaud from comment #13)


Listing the keys shows this:

raffi@raffi:/data/tmp/.thunderbird$ gpg --show-keys --with-fingerprint --with-subkey-fingerprint secret.asc
sec rsa2048 2014-10-23 [SC] [expires: 2022-10-31]
EA29 15DE 9DEE 7F2C FC17 F36D D4DC F656 B166 91C4
uid Raffi Enficiaud <raffi.enficiaud@email1.com.redacted>
uid Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi@email2.com.redacted>
uid Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@email3.com.redacted>
uid [jpeg image of size 203959]
uid Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@email4.com.redacted>
uid [ revoked] Raffi Enficiaud <raffi.enficiaud@email4.com.redacted>
ssb rsa2048 2014-10-23 [E] [expires: 2022-10-31]
EAE3 D605 8678 6846 3A63 0C94 5134 53DF C850 67FF
ssb rsa4096 2015-06-28 [S] [expired: 2019-06-28]
B8BE 2B4E 1905 53E3 B0B7 C414 542B 960F F636 ACBA

Again, this has ..D120 missing (which means you are using a key that is different from what you showed in the initial screenshot in this message).

The secret signing subkey expired.

The expected behavior would be for Thunderbird to fall back to use the primary key for signing.
Is the secret key for the primary key available in your configuration?

Also, in the screenshot, subkey ...ACBA is valid until 2022-09-21.
However, in the listing I quoted in comment 24, subkey ..ABCA already expired in 2019.

It seems like your key setup is a little chaotic?

I created a test key that matches the structure from comment 24, and I used roughly the same times and expiration dates (modified my system's date).

I am able to send a signed message with that key (signing subkey expired, but signing primary key still valid).
So we still don't know what's wrong on your system.

I think the information you have provided isn't sufficient to analyze your scenario.
We need information from a clean setup.

You could create a new (separate) operating system user on your system, which doesn't have any keys in your gnupg keyring.

Copy the key you want to use to that account, and make sure you use an up to date version of your key.

Create a key list of that key, and also a packet dump.

Import the key into Thunderbird, and test if it works.

If it doesn't work, please provide a screenshot of the key structure of the key you have.

Please make sure that all pieces are matching, if you want further help in investigating your scenario.
(Because I think for the earlier reports, you mixed things up, because the information shown screen shot, packet dump, gnupg text listing, were inconsistent.)

A/ At your request

  1. from thunderbird on my machine causing the issue (Ubuntu): I exported the key using the Thunderbird key manager
  2. I transferred the key to the machine that does not have the issue (macOS), removed all previous keys and installed the key exported in 1/
  3. I then successfully sent an email with signature

But I then inspected the keys:

  • exporting the key on from one machine (Ubuntu) and importing it on another machine (macOS) yields different subset of keys (please see attached screenshots): dates do not correspond and subkeys are different. I did the operation several times with different passphrases to make sure I was doing the operation correctly.

B/ Trying to reproduce on mac
Then on macOS I tried to export the key from GPG directly and import it to Thunderbird (mac), as I don't remember exactly how I did it initially on Ubuntu. The import fails and says "The key you are trying to import might be corrupt or use unknown attributes". It suggest to import parts that are correct but fails as well. This may be because I am using another passphrase for one of the subkeys (I am seeing Cannot unprotect key 542B960FF636ACBA in the error console).

Then on macOS I exported the original GPG key I was using (trying to get as close as possible to the initial report) using the GPG keychain. It had one additional subkey which I removed from the command line. I then imported the key into Thunderbird and send a signed email to myself without issue.

So unable to reproduce the issue on mac at this point.

C/ Back to Ubuntu
I removed the keys from Thunderbird. I then imported the file saved in B/ to Ubuntu thunderbird. I am able to send a signed email.

D/ On Ubuntu again
I removed the keys from Thunderbird, I took the key saved in A/ (the one from the export), imported it in Thunderbird. I am able to send a signed email.
I made an enormous amount of experiments with subkeys etc. trying to get into the same state. For instance, exporting to a file having only the D120 subkey with
secret, that subkey with the main key secret, various combinations of those, etc. It was just for me impossible to get to the same point.

E/ Summing up 1/

  • @Kai this joins exactly what you are saying as the key ending with D120 is missing
  • the setup I have: I reached this using only Thunderbird and GPG, nothing other than those 2 programs.

Because I think for the earlier reports, you mixed things up, because the information shown screen shot, packet dump, gnupg text listing, were inconsistent

I should not be needing to have another user for reproducing the issue. Things may be seen inconsistent but I in no way changed the outputs of anything I did (except for changing the emails). So if there is any inconsistency, it comes from something else.

F/ Now able to reproduce the issue
... but I noticed something fishy ... and this is not about the setup. And I am now able to reproduce the issue. Give me some minutes and I prepare the minimal steps.

Ok, so here is the story.

step 1

I have a key that is like this on Ubuntu/GPG (here I have a clean gpg folder). I thought I have the secret for the subkey D120 on that machine, but I was wrong. This is however not the problem.

The content of the key:

$ gpg --homedir /data/tmp/gpg_tmp --with-subkey-fingerprints --list-secret-keys
gpg: WARNING: unsafe permissions on homedir '/data/tmp/gpg_tmp'
/data/tmp/gpg_tmp/pubring.kbx
-----------------------------
sec   rsa2048 2014-10-23 [SC] [expires: 2022-10-31]
      EA2915DE9DEE7F2CFC17F36DD4DCF656B16691C4
uid           [ unknown] Raffi Enficiaud <raffi.enficiaud@mines-paris.org>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@XXXXX>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@XXXXXX>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi@XXXXXXXX>
uid           [ unknown] [jpeg image of size 203959]
ssb   rsa2048 2014-10-23 [E] [expires: 2022-10-31]
      EAE3D605867868463A630C94513453DFC85067FF
ssb   rsa4096 2015-06-28 [S] [expires: 2022-09-21]
      B8BE2B4E190553E3B0B7C414542B960FF636ACBA
ssb#  rsa4096 2021-09-14 [S] [expires: 2025-09-14]
      5972A22ED75F464D86B7D83F12B5AE439236D120

As you can see from above, I don't have the secret for ...D120. In addition to that, subkey ...CBA has another passphrase. So I edit the key

  1. I remove completely ...D120 from it,
  2. I change the passphrase such that Thunderbird can import it

This is what I have now:

$ gpg --homedir /data/tmp/gpg_tmp --with-subkey-fingerprints --list-secret-keys
gpg: WARNING: unsafe permissions on homedir '/data/tmp/gpg_tmp'
/data/tmp/gpg_tmp/pubring.kbx
-----------------------------
sec   rsa2048 2014-10-23 [SC] [expires: 2022-10-31]
      EA2915DE9DEE7F2CFC17F36DD4DCF656B16691C4
uid           [ unknown] Raffi Enficiaud <raffi.enficiaud@mines-paris.org>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@XXXXX>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@XXXXXX>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi@XXXXXXXX>
uid           [ unknown] [jpeg image of size 203959]
ssb   rsa2048 2014-10-23 [E] [expires: 2022-10-31]
      EAE3D605867868463A630C94513453DFC85067FF
ssb   rsa4096 2015-06-28 [S] [expires: 2022-09-21]
      B8BE2B4E190553E3B0B7C414542B960FF636ACBA

step 2

I then export that key to a file and import it in Thunderbird:

gpg --homedir /data/tmp/gpg_tmp/  --output ~/Documents/test-reproduce.asc --armor --export-secret-keys

I have now in Thunderbird a coherent setup with exactly the structure as above (screenshot on the image step-2-after-import.png).

step 3

I then go to the menu "keyserver" and "discover keys online",

  1. I enter my email in the modal
  2. the proper public keys are found, I import the key

However, I inspect the structure of the key, and it shows the subkey ...D120 as if it has the secret (!) (screenshot on the image step-3-after-public-key-lookup.png), which is impossible.

step 4

  1. I write an email with signature,
  2. sending fails as mentioned initially,
  3. error console shows the same error as before
21:40:15,731 getCryptParams, got: to=raffi.enficiaud@XXXXXXX, bcc= encryption.jsm:113:13
21:40:15,731 getCryptParams returning: encryption.jsm:190:13
21:40:15,731
Object { sender: "0xD4DCF656B16691C4", sign: true, signatureHash: "SHA256", sigTypeClear: false, sigTypeDetached: true, encrypt: false, encryptToSender: false, armor: true, senderKeyIsExternal: false, to: (1) […], … }
encryption.jsm:191:13
21:40:15,732 CryptoAPI.sync() failed result:  Error: rnp_op_sign_add_signature failed
    encryptAndOrSign chrome://openpgp/content/modules/RNP.jsm:2429
    sync chrome://openpgp/content/modules/cryptoAPI/interface.js:56
    encryptMessageStart chrome://openpgp/content/modules/encryption.jsm:410
    finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:559
    createMessageFile resource:///modules/MimeMessage.jsm:86
interface.js:46:17
21:40:15,733 sendFlags=000000c1 encryption.jsm:426:13
21:40:15,733 Error: failure in finishCryptoEncapsulation, exitCode: -1
    finishCryptoEncapsulation chrome://openpgp/content/modules/mimeEncrypt.jsm:580
    createMessageFile resource:///modules/MimeMessage.jsm:86
mimeEncrypt.jsm:597:15
21:40:15,733 mailnews.send: 
Exception { name: "NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS", message: "[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]'[JavaScript Error: \"failure in finishCryptoEncapsulation, exitCode: -1\" {file: \"chrome://openpgp/content/modules/mimeEncrypt.jsm\" line: 580}]' when calling method: [nsIMsgComposeSecure::finishCryptoEncapsulation]", result: 2153185313, filename: "resource:///modules/MimeMessage.jsm", lineNumber: 86, columnNumber: 0, data: XPCWrappedNative_NoHelper, stack: "createMessageFile@resource:///modules/MimeMessage.jsm:86:27\n", location: XPCWrappedNative_NoHelper }
MessageSend.jsm:131:27
21:40:15,733 mimeEncrypt.js: caught exception: Error
Message: 'failure in finishCryptoEncapsulation, exitCode: -1'
File:    chrome://openpgp/content/modules/mimeEncrypt.jsm
Line:    580
Stack:   finishCryptoEncapsulation@chrome://openpgp/content/modules/mimeEncrypt.jsm:580:15
createMessageFile@resource:///modules/MimeMessage.jsm:86:27


21:40:15,733 Error: failure in finishCryptoEncapsulation, exitCode: -1 mimeEncrypt.jsm:580:15
21:40:15,734 mailnews.send: Sending failed; , exitCode=2153185313, originalMsgURI= MessageSend.jsm:330:27
21:40:15,762 Error: rnp_op_sign_add_signature failed RNP.jsm:2429:17
21:43:06,001 in getEncryptionFlags, gSendEncrypted=false, gSendSigned=true enigmailMsgComposeOverlay.js:1610:13
22:03:46,812 searchKeysOnInternet no wkd data for raffi.enficiaud@XXXXXXXX keyLookupHelper.jsm:104:15

Conclusion

The problem does not seem to come from any file operation, but rather when merging the information that the keystore has with the information pulled from the remote server key lookup. The state of that key becomes inconsistent. Whether or not the information shown by the key is consistent or not, I don't know.

Attached image step-2-after-import.png

This is the state of the key after file import, but deleting one of the subkeys completely (...D120).

This is the state of the key after lookup on the keyserver. It shows now the key ...D120 as if it can use it for signing (without the (!) meaning it has the associated secret ... right?).

This seems to be the root cause of the problem.

@Kai @Nickolay
I spent quite some time on this :) I hope this helps. Let me know if you need anything else from my side.

Ok, thanks, I think we have found the cause of the problem.

If I remember correctly, we only show the (!) prefix for the primary key. I didn't consider that a secret key for a subkey could be missing. That seemed like an unusual setup.

After updating from the keyserver, you have only the public for key subkey ..D120.

TB searches through subkeys for one that is usable for "signing", and prefers the one that was most recently created.

That means it attempts to use ..D120, but because the secret key is missing, it fails to sign.

We need to fix the code to only consider subkeys for signing that also have the secret key available (currently the code simply assumes that any key that is configured for signing will have secret keys for all subkeys available).

If I remember correctly, we only show the (!) prefix for the primary key.

As far as I can tell, this is not the case: any subkey not having the associated secret will be shown with a (!). Example on the screenshot test-without-secret-for-subkeys.png. This is why I believe the problem comes on the merge of the key information after fetching the public parts from the keyserver. The logic for selecting the subkey I think can remain the same, the only problem being that the subkey fetched from the keyserver is tagged with having the associated secret.

The setup I want is this one:

-----------------------------
sec#  rsa2048 2014-10-23 [SC] [expires: 2022-10-31]
      EA2915DE9DEE7F2CFC17F36DD4DCF656B16691C4
uid           [ unknown] Raffi Enficiaud <raffi.enficiaud@XXXX>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@XXXX>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@XXXX>
uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi@XXXX>
uid           [ unknown] [jpeg image of size 203959]
ssb   rsa2048 2014-10-23 [E] [expires: 2022-10-31]
      EAE3D605867868463A630C94513453DFC85067FF
ssb   rsa4096 2015-06-28 [S] [expires: 2022-09-21]
      B8BE2B4E190553E3B0B7C414542B960FF636ACBA

which corresponds to the screenshot test-without-secret-for-primary-key.png (no secret for the primary key and only 2 chosen subkeys).

In this example, I removed all the secret parts of the subkeys.

In this example I took a subset of the keys and removed the secret of the primary key.

(In reply to raffi.enficiaud from comment #35)

As far as I can tell, this is not the case: any subkey not having the associated secret will be shown with a (!). Example on the screenshot test-without-secret-for-subkeys.png.

I have further refreshed my memory.

The (!) does not generally mean "secret key not available".
It's indicating a very special state, which is a non-standard gnupg extension, which means:

  • the key CLAIMS to have the secret key available
  • but in reality the secret key is ABSENT

If you have originally imported a set of of keys where ..D120 is completely missing, and then you later import only a public key incliding ..D120, then you do not end up in this special state.

Only if you use a special gnupg command to explicitly delete the secret key material, only then I believe you get into the special state.

Currently Thunderbird does not have UI to show that a secret key is completely missing.

Summary: Thunderbird failed to send message when GPG signing is on: Ubuntu 20.04 → OpenPGP: Cannot sign if the secret part of the preferred signing subkey is missing

The attached patch changes the logic to identify subkeys suitable for signing, to only consider keys that have the secret part available. Previously, this was assumed for all keys with the primary key advertising itself as having a secret key available (including the offline primary scenario). Previously, the code didn't expect that secret subkeys might be missing.

If a primary key advertises itself as having a secret key, with this patch, the key properties will show the (!) marker for all missing secret subkeys, for both absent key material as indicated by GnuPG, and also for fully absent secret keys.

I'm building an experimental binary based on TB 91.x plus this patch here:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=6e19e74417d0e72d0c8ab89c94910d873c92ca7b

Raffi, when the build is complete (in about 1-2 hours), there will be a green "B" (middle right section, after the text "Linux x64 opt"). Click the green B, then click Artifacts in the lower black bar, then click target.tar.bz2 to download the Thunderbird binary. It behaves like a TB 91.x version, so it should be safe to use it with your regular profile. Could you please test it and give feedback if it solves the bug for you?

Sure, I'll do that tomorrow morning! (CET)

Attached image screenshot.png

@Kai I guess the build did not succeed (I cannot find the file you mentioned in the lower bar), correct?

  1. I created the following key, leaving the ...D120 out:

    $ gpg --homedir tmp_gpg --list-secret-keys --with-subkey-fingerprints
    gpg: WARNING: unsafe permissions on homedir '/tmp/tmp_gpg'
    /tmp/tmp_gpg/pubring.kbx
    ------------------------
    sec   rsa2048 2014-10-23 [SC] [expires: 2022-10-31]
        EA2915DE9DEE7F2CFC17F36DD4DCF656B16691C4
    uid           [ unknown] Raffi Enficiaud <raffi.enficiaud@XXXXX>
    uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@XXXXX>
    uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi.enficiaud@XXXXX>
    uid           [ unknown] Raffi Enficiaud (Alias for Raffi Enficiaud) <raffi@XXXXX>
    uid           [ unknown] [jpeg image of size 203959]
    ssb   rsa2048 2014-10-23 [E] [expires: 2022-10-31]
        EAE3D605867868463A630C94513453DFC85067FF
    ssb   rsa4096 2015-06-28 [S] [expires: 2022-09-21]
        B8BE2B4E190553E3B0B7C414542B960FF636ACBA
    
  2. then saved the key and imported it to Thunderbird. The image shows the right set of key/subkeys

This is the same as "Step 2" from https://bugzilla.mozilla.org/show_bug.cgi?id=1753214#c30 above

Now I am fetching the keys from the keyserver. It properly pulls the public part of ...D120 but now shows it with an exclamation mark:

  • according to your definition: the key CLAIMS to have the secret key available, but the secret material is not available
  • from the user perspective, this should now mean the key cannot be used for signing (yet should be usable for verifying signed emails)

I then send a signed email with that account, sending the email succeeds.

The signed email uses another subkey, ...CBA which is the only subkey available for signing.

Finally, exporting the key w. secrets from Thunderbird to a file and inspecting it shows this:

$ gpg --list-packets Raffi\ Enficiaud\ raffi.enficiaud@XXXXX-\(0xD4DCF656B16691C4\)-secret.asc | grep keyid
	keyid: D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
:signature packet: algo 1, keyid D4DCF656B16691C4
	keyid: 513453DFC85067FF
:signature packet: algo 1, keyid D4DCF656B16691C4
	keyid: 542B960FF636ACBA
:signature packet: algo 1, keyid D4DCF656B16691C4

So still no ...D120 as it should be.

I guess now we are all good !

(In reply to raffi.enficiaud from comment #47)

Now I am fetching the keys from the keyserver. It properly pulls the public part of ...D120 but now shows it with an exclamation mark:

  • according to your definition: the key CLAIMS to have the secret key available, but the secret material is not available

CLAIMS was the old behavior. With this suggested patch, the (!) is shown for any absent secret key (if primary claims to be a secret key).

  • from the user perspective, this should now mean the key cannot be used for signing (yet should be usable for verifying signed emails)

Well, this subkey with the (!) cannot be used for signing. But overall, the key should be usable for signing, because the primary key supports signing, and there is even another valid subkey still usable for signing.

I then send a signed email with that account, sending the email succeeds.

Thanks. This means your bug is fixed with this patch.

(In reply to raffi.enficiaud from comment #48)

I guess now we are all good !

Glad to hear.

Will request review for the patch and we can try to backport to 91.

Assignee: nobody → kaie
Attachment #9264282 - Attachment description: WIP: Bug 1753214 - Only consider signing subkeys usable that have the secret part available. Indicate all missing secret (sub)keys in key properties. → Bug 1753214 - Only consider signing subkeys usable that have the secret part available. Indicate all missing secret (sub)keys in key properties. r=mkmelin
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

Test fails without D138959, succeeds with D138959.

Blocks: 1756795

When backporting to esr91, see here for required changes:
https://phabricator.services.mozilla.com/D139463?vs=547699&id=550709#change-YGxqn7Cw4qot

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/e8b4fda8c162
Only consider signing subkeys usable that have the secret part available. Indicate all missing secret (sub)keys in key properties. r=mkmelin
https://hg.mozilla.org/comm-central/rev/8cef3744fa98
Add a test. r=mkmelin

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 100 Branch

Comment on attachment 9264282 [details]
Bug 1753214 - Only consider signing subkeys usable that have the secret part available. Indicate all missing secret (sub)keys in key properties. r=mkmelin

[Approval Request Comment]
Regression caused by (bug #): no
User impact if declined: reduced compatibility
Testing completed (on c-c, etc.): yes
Risk to taking this patch (and alternatives if risky): very low risk

Attachment #9264282 - Flags: approval-comm-esr91?
Attachment #9265121 - Flags: approval-comm-esr91?

Comment on attachment 9265121 [details]
Bug 1753214 - Add a test. r=mkmelin

[Triage Comment]
Approved for esr91

Attachment #9265121 - Flags: approval-comm-esr91? → approval-comm-esr91+

Comment on attachment 9264282 [details]
Bug 1753214 - Only consider signing subkeys usable that have the secret part available. Indicate all missing secret (sub)keys in key properties. r=mkmelin

[Triage Comment]
Approved for esr91

Attachment #9264282 - Flags: approval-comm-esr91? → approval-comm-esr91+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: