1752010 - Better error reporting for invalid external GnuPG configuration

Status: RESOLVED FIXED

(MailNews Core :: Security: OpenPGP, enhancement)

Description

[Kai Engert [:KaiE:]]

Today I helped a TB 91.x user to analyze why their external GnuPG configuration didn't work.

https://support.mozilla.org/en-US/questions/1365757

The reason was, the user had entered an incorrect key ID into the advanced external gnupg configuration.

To my surprise, I learned that the Linux Mint distribution has changed the default for that configuration to on... So more users might try to use that configuration, without having read the howto document :(

I think we need to at least improve the error reporting in this scenario. If we try to use the configured external key for encryption, and it cannot be used, we should tell the user what the cause of the problem is.

Comment 1

[Kai Engert [:KaiE:]]

To be more specific, the configuration had failed, because the user had entered the keyID of a subkey, which wasn't usable to encryption.

Although our documentation (https://wiki.mozilla.org/Thunderbird:OpenPGP:Smartcards) requests to configure the key ID of the primary key, I remember from a past conversation, that another used had configured a specific subkey ID, with the intention to explicitly request which subkey the encryption should target. That works, because we pass the configured key ID to RNP.

From that past conversation I also remember that signing worked despite that configuration. I conclude that when giving the encryption subkey ID to gnupg, and requesting to use that for signing, GnuPG automatically finds a matching associated signing key and uses it.

The main question here is, why didn't we warn?

The reason is, that we have two stages of processing. In the first stage, we use application code (TB/JS) to check if the configured key is good. The second stage is that we ask RNP for processing.

In this scenario here, our first stage had succeeded. Why? Because our initial checks used smarter lookup code that what we use at the RNP processing level.

During the initial check, even if search by subkey ID, we'll find the primary key. Then the checking sees the subkeys and finds that everything required is present.

But the RNP processing didn't do that smart lookup. It attempted to use the given subkey directly, which failed, because a subkey was configured that isn't valid for encryption.

So, we should fix this in a way that doesn't break the existing working scenario, which means, it works to configure an encryption subkey. This means in the first check we shouldn't simply require a primary key ID.

I suggest that in the initial check, we verify that the configured key is either a primary key ID (then we check for any valid encryption subkey, which is what our RNP code will do), or it is a subkey ID. If it's a subkey ID, we check that this specific subkey is usable for encryption. If it is, we're good. If it isn't, we bring up an error message.

Comment 2

[Kai Engert [:KaiE:]]

Attachment: Bug 1752010 - Better error reporting for invalid external GnuPG configuration. r=mkmelin

Comment 3

[Nickolay Olshevsky]

This may be helpful: since v0.15.2 we added a new function to RNP FFI - rnp_key_get_default_key() which would pick key/subkey for the desired operation, in case you don't want to implement your own logic.

Comment 4

[Nickolay Olshevsky]

(incorrect: it was added in v0.15.2).

Comment 5

[Kai Engert [:KaiE:]]

Attachment: Bug 1752010 - Better error reporting for invalid external GnuPG configuration. r=mkmelin

Comment 6

[Kai Engert [:KaiE:]]

esr91 is affected, but not sure we should backport. It's only improving diagnostics.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/222d65a839fa
Better error reporting for invalid external GnuPG configuration. r=mkmelin

Comment 8

[Magnus Melin [:mkmelin]]

Maybe worth taking on 91, to potentially save someone hazzle of investigations.

Comment 9

[Kai Engert [:KaiE:]]

I was considering to ask for esr91 approval, but given the changes, it's not easy to judge the risk. I'd need to rereview the changes to be confident asking.