Closed Bug 1777336 Opened 2 years ago Closed 2 years ago

S/MIME Sending of signed messages fails after upgrade to TB102 due to intermediate cert handling

Categories

(Thunderbird :: Security, defect, P1)

Product:
Thunderbird
Component:
Security
Version:
Thunderbird 102
Type:
defect

Tracking

(thunderbird_esr102+ fixed, thunderbird103 fixed)

Status:
RESOLVED FIXED
Milestone:
104 Branch
Tracking Flags:
Tracking Status
thunderbird_esr102 + fixed
thunderbird103 --- fixed

People

(Reporter: jaltman, Assigned: KaiE)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: regression, Whiteboard: [TM:102.0.3] workaround: Use Settings, Config Editor, set security.intermediate_preloading_healer.enabled to false)

Attachments

(1 file)

Bug 1777336 - Disable automatic intermediate CA cleanup, because it breaks S/MIME. r=rjl
48 bytes, text/x-phabricator-request
wsmwk
: approval-comm-beta+
wsmwk
: approval-comm-esr102+
Details | Review

Steps to reproduce:

Upgraded to Thunderbird 102 from 91.x.

Mail accounts have associated valid S/MIME certificates from IdenTrust with Client Authentication and E-mail Protection EKUs.

Actual results:

Attempts to send mail messages with S/MIME Signing and no Encryption fails.

Sending of the message failed.
Unable to sign message. Please check that the certificates specified in Mail & Newsgroups Account Settings for this mail account are valid and trusted for mail.

Expected results:

The mail message would be sent signed as it was with TB 91.x

Please let me know what additional information I can provide to assist in debugging this issue.

Component: Untriaged → Security

I can send S/MIME signed unencrypted messages without any problems using a fresh install of Thunderbird 102.
Can you please verify your Account Settings under End-To-End Encryption?
Then under S/MIME click the button Manage S/MIME Certificates and View the properties of your certificate.
Make sure it's valid for signing and that the certificate chain van be verified.

Do you only have problems with unencrypted signed messages, or also with S/MIME encrypted messages, either signed of unsigned?

(In reply to Onno Ekker [:nONoNonO UTC+1] from comment #2)

I cannot send S/MIME signed unencrypted messages without any problems using a fresh install of Thunderbird 102.
Can you please verify your Account Settings under End-To-End Encryption?

The certificates listed for S/MIME Personal certificate for digital signing and encryption are both correct. I've attempted removing the certs and re-importing them without luck.

These certificates are valid and are in use with Thunderbird 91.x and iOS Mail.

Then under S/MIME click the button Manage S/MIME Certificates and View the properties of your certificate.

Key Usages Purposes: Digital Signature, Non-Repudiation, Key Encipherment, Data Encipherment

Extended Key Usages Purposes: Client Authentication, E-mail Protection

Make sure it's valid for signing and that the certificate chain van be verified.

The full chain is present and there are not reported verification failures. Note that the UI doesn't indicate when the chain has been verified.
The RootCA is "IdenTrust Commercial Root CA 1" which is included in the builtin set of authorities.

Do you only have problems with unencrypted signed messages, or also with S/MIME encrypted messages, either signed of unsigned?

I can not send an unencrypted signed message using S/MIME.

I can not send an encrypted signed message using S/MIME.

I can send encrypted but unsigned messages using S/MIME. I can read the message stored in the "Sent" folder. S/MIME encrypted messages sent to my account can be decrypted and read.

Could you check if the error console reports anything related? (found in menu tools, developer tools)

You might want to clear it, prior to hitting the send toolbar button, and check if the attempt to send produces new information on the console.

(In reply to Kai Engert (:KaiE:) from comment #4)

Could you check if the error console reports anything related? (found in menu tools, developer tools)

From mailnews.send there are two messages:
MessageSend.jsm:130:27 Exception { name: "NS_ERROR_FAILURE", message: "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIMsgComposeSecure.finishCryptoEncapsulation]", result: 2147500037, filename: "resource:///modules/MimeMessage.jsm", lineNumber: 86, columnNumber: 0, data: null, stack: "createMessageFile@resource:///modules/MimeMessage.jsm:86:27\n", location: XPCWrappedNative_NoHelper }

Sending failed; , exitCode=2147500037, originalMsgURI= MessageSend.jsm:335:27
fail resource:///modules/MessageSend.jsm:335
createAndSendMessage resource:///modules/MessageSend.jsm:138

Thanks. Comment 5 shows that we're indeed failing while attempting to complete the generation of the S/MIME message.

We need to find out what's specific in your environment that Thunderbird cannot handle. Maybe it's special attributes of your S/MIME certificates. If you can get me a free S/MIME cert from the exact same CA, then I could try to reproduce and debug myself.

Are your certs on a smartcard?

Flags: needinfo?(jaltman)

Do you have a primary password set in Thunderbird? If yes, ensure you have correctly entered it (do not cancel the password prompt).

Just trying to collect more details:
Do you use a single certificate, which you have selected for both signing and encryption in account settings?

While I don't think it's likely that your public certificate will help me in debugging, it also cannot hurt: Could you please use Thunderbird 91.x to send a signed S/MIME message to testmail@kuix.de ? Thanks

(In reply to Kai Engert (:KaiE:) from comment #6)

Thanks. Comment 5 shows that we're indeed failing while attempting to complete the generation of the S/MIME message.

We need to find out what's specific in your environment that Thunderbird cannot handle. Maybe it's special attributes of your S/MIME certificates. If you can get me a free S/MIME cert from the exact same CA, then I could try to reproduce and debug myself.

The certs are purchased from

https://www.identrust.com/certificates/secure-email-smime

pricing starts at $17.

Are your certs on a smartcard?

No. The certs are stored in the Thunderbird NSS Internal PKCS #11 Module - Software Security Device.
The status of the device is "Logged in".

Flags: needinfo?(jaltman)

(In reply to Kai Engert (:KaiE:) from comment #7)

Do you have a primary password set in Thunderbird? If yes, ensure you have correctly entered it (do not cancel the password prompt).

Yes. There is a primary password which has been entered correctly. I would be unable to read or send mail without doing so.
I can send unsigned and encrypted e-mail which also requires access to the private key.

(In reply to Jeffrey Altman from comment #9)

The certs are purchased from
https://www.identrust.com/certificates/secure-email-smime
pricing starts at $17.

Behavior might be different based on the specific certificate purchased and its attributes.
Are you using the one for $17 or a more expensive one?
If you prefer, feel free to share this by private email to kaie@kuix.de

(In reply to Kai Engert (:KaiE:) from comment #8)

Just trying to collect more details:
Do you use a single certificate, which you have selected for both signing and encryption in account settings?

I have multiple e-mail accounts each with multiple identities. Not all identities have S/MIME certificates associated.
For the account identities which do have S/MIME certificates associated the same certificate is always selected for both signing and encryption.

While I don't think it's likely that your public certificate will help me in debugging, it also cannot hurt: Could you please use Thunderbird 91.x to send a signed S/MIME message to testmail@kuix.de ? Thanks

Sent

Apparently that CA distinguishes between customers from the US and non-US. Your certificate is for US, I'm outside the US. We can only hope that this won't make a difference?

Another interesting aspect of my configuration is that although S/MIME is my preference these days I also have PGP keys configured for the identities.

(In reply to Kai Engert (:KaiE:) from comment #13)

Apparently that CA distinguishes between customers from the US and non-US. Your certificate is for US, I'm outside the US. We can only hope that this won't make a difference?

If it doesn't work, contact me privately and I will arrange to setup a test e-mail account and purchase a certificate for it.

ok, I have a cert from IdenTrust and I can reproduce. I'll debug.
(It really seems to be specific to that cert, a free cert from Actalis worked.)

(In reply to Kai Engert (:KaiE:) from comment #16)

ok, I have a cert from IdenTrust and I can reproduce. I'll debug.
(It really seems to be specific to that cert, a free cert from Actalis worked.)

Excellent. I am very curious as to what changed from TB 91.x.

I have the same issue, my cert was working in 91 and it does not work in 102.

My certificate is from DigiCert.

I was able to sign emails again after adding a mid CA that was not needed in 91, but the signature is wrong it can not be verified anymore

If I remove the mid CA then I'm not able again to sign emails, notice that the imported CA goes to the "Software Security Device" and not to the "Builtin Object Token"

More details in: https://support.mozilla.org/en-US/questions/1381217#answer-1516026

I came to the some conclusion. Our certificate validation code cannot find the intermediate CA named "TrustID CA A13".

When sending a signed email, we run through some common code that checks all involved certificates for validity. For that, our S/MIME code uses an older engine of our NSS library.

That validity check fails, because it fails to find the issuer certificate.

I didn't expect that when interactively looking at that earlier, because Thunderbird's certificate viewer can find the intermediate and display it. I'm guessing this is because the cert viewer/manager uses a newer engine for certificate code, and it might have automatically downloaded the intermediate cert.

However, our S/MIME code is unfortunately rather old, and it isn't using the modern engine, and it can only use the certificates that have been permanently imported into the NSS database.

In the past, when a CA delivered a certificate to their users, it included the full set of required intermediate certificates. Apparently this didn't happen here. Or, it didn't happen when the certificate was exported from the original store into which you had imported (I assume you had to export it from somewhere, prior to importing it into Thunderbird).

I manually downloaded the intermediate CA cert from https://crt.sh/?id=2459043195 (download link https://crt.sh/?d=2459043195 ). Then I imported it into the Thunderbird cert manager, authorities tab (I kept all checkbox unchecked).

After that import, sending a signed message works.

I wonder why this worked with 91.

Status: UNCONFIRMED → NEW
Ever confirmed: true

The pfx file I had exported from Windows does contain the intermediate.
I'll check if there is a difference on importing, maybe 91 imported it, but 102 maybe doesn't export it?
Just a theory, will investigate further.

(In reply to Kai Engert (:KaiE:) from comment #20)

However, our S/MIME code is unfortunately rather old, and it isn't using the modern engine, and it can only use the certificates that have been permanently imported into the NSS database.

In the past, when a CA delivered a certificate to their users, it included the full set of required intermediate certificates. Apparently this didn't happen here. Or, it didn't happen when the certificate was exported from the original store into which you had imported (I assume you had to export it from somewhere, prior to importing it into Thunderbird).

In my case, TB 102 was installed as an upgrade from TB 91. Nothing was exported and re-imported by me. Was there an export and import performed by the upgrade process?

I manually downloaded the intermediate CA cert from https://crt.sh/?id=2459043195 (download link https://crt.sh/?d=2459043195 ). Then I imported it into the Thunderbird cert manager, authorities tab (I kept all checkbox unchecked).

That is the CA A13 cert. I need to find the CA A12 cert. I will try later.

After that import, sending a signed message works.

I wonder why this worked with 91.

Me too ...

My test results are very confusing.

With 102 and a fresh profile, importing that identrust.pfx file, the CA A13 gets imported, too, and S/MIME signing works immediately.

Same with 91 and a fresh profile.

Then I went back to the initial profile that I had used for testing. That was a profile which I've been using for a longer time, and which had already contained other S/MIME personal certificate.
With that old profile, using 102, importing identrust.pfx, I could reproduce the failure reported here.
I don't know what had happened. I don't know if CA A13 was excluded when importing, or if CA A13 was somehow incorrectly imported.

I tried to reproduce the failure again, by deleting both personal cert and CA A13, restarting, then importing again. But in my repeated attempts, it always works. I can no longer reproduce the original failure. (Well, I could by explicitly deleting the CA A13 cert, but that wouldn't be a correct reproducting of this bug.)

So, in short:

Apparently sometimes an intermediate CA cert isn't imported, or isn't correctly imported, when importing a personal S/MIME certifiate, resulting in the intermediate not being found, and the personal cert not being usable.

I removed the intermittent CA and added again without the checkboxes, and yes I can send signed emails, but still they can not be verified

A bit more info... after I manually imported the mid CA "DigiCert SHA2 Assured ID CA" that is by the way in the chain of my certificate, I remove my certificate and added again, hoping that with the mid CA already imported it will work.

By surprise after that I was not able to sign mails any more, then I realize that importing my certificate removes the manually imported mid CA "DigiCert SHA2 Assured ID CA". manually add the mid CA again let me again send (wrongly signed messages).

I hope that this helps to track and fix the bug.

((enjoy))
cr

I haven’t tried finding a regression range, but maybe this is also regressed by the different way of handling the fqdn/alternate name in bug 1691122.

(In reply to Onno Ekker [:nONoNonO UTC+1] from comment #26)

I haven’t tried finding a regression range, but maybe this is also regressed by the different way of handling the fqdn/alternate name in bug 1691122.

very unlikely. If this would be related to the removed fallback to common name, it would never work. But it works randomly.

I have a hard time reproducing this reliably. In 90% of attempts it just works. I was able to reproduce it again twice. But when I thought I could simply repeat my steps again (after restoring the previous state of the profile), then it worked again on the next attempt.

Either this is completely random, or a special order of importing/viewing/selecting/using/restart is required.

Well, for me always fails, no matter in which order I try.

I've tested upgrade from 91, that worked for me, too.

Which operating system do you use? I'm testing on Linux.

Hi, same behaviour here with SwissSign Personal Silver certs on W2012r2 OS. After the upgrade from 91 to 102, it refuses signed emails. Also, after manually importing the intermediate CA cert (SwissSign Personal Silver CA 2014-G22) to the Authorities, the problem is solved.

(In reply to kr from comment #30)

Hi, same behaviour here with SwissSign Personal Silver certs on W2012r2 OS. After the upgrade from 91 to 102, it refuses signed emails. Also, after manually importing the intermediate CA cert (SwissSign Personal Silver CA 2014-G22) to the Authorities, the problem is solved.

Hi cr, Can you please check if you or any other recipients can verify your signature, as for me adding the intermediate CA cert let me send the signed email but the signature is broken and could not be verified.

(In reply to Kai Engert (:KaiE:) from comment #29)

I've tested upgrade from 91, that worked for me, too.

Which operating system do you use? I'm testing on Linux.

I have MacOS 12.4

Hi Carlos, I can't see any obvious problem with the signature on the receiver side after fixing the sender-side signing.

I am not having this problem, after upgrading TB from 91 to 102. However I am using an S/MIME certificate issued by my company (Actalis).

Jeffrey, you may want to apply for a Free S/MIME at the following URL to see if it changes anything:
https://extrassl.actalis.it/portal/uapub/freemail?reqid=&lang=en

IMO, it would help a lot if TB shown the whole certificate chain as it used to do in the past. This basic feature got broken over time, but TB developers just ignored it as if it was something superfluous. I do believe that TB developers should fix that soon, as the actual (broken) behaviour is really dumb and hampers diagnosing problems like this one. See https://bugzilla.mozilla.org/show_bug.cgi?id=1716998

I have also had this problem after upgrading from 91.10.0 to 102.0. I have been unable to find what causes it though, one day it works then another day it doesn't with no changes to the system or any configuration. I am running on Slackware Linux 15.0 using the binary download from thunderbird.net. I also tested with the Slackware provided package but get the same "Unable to sign message." error intermittently

(In reply to Jeffrey Altman from comment #22)

(In reply to Kai Engert (:KaiE:) from comment #20)

However, our S/MIME code is unfortunately rather old, and it isn't using the modern engine, and it can only use the certificates that have been permanently imported into the NSS database.

In the past, when a CA delivered a certificate to their users, it included the full set of required intermediate certificates. Apparently this didn't happen here. Or, it didn't happen when the certificate was exported from the original store into which you had imported (I assume you had to export it from somewhere, prior to importing it into Thunderbird).

That is the CA A13 cert. I need to find the CA A12 cert. I will try later.

After that import, sending a signed message works.

I wonder why this worked with 91.

Me too ...

For completeness, Thunderbird 102 is running on Windows 11 [Version 10.0.22621.169].

I downloaded the four certificates and private keys from IdentTrust and imported them without removing the prior certificates.
The CA A12 intermediate certificate was installed. I sent one e-mail signed but not encrypted to another e-mail account I control and verified that the signature was ok.

My desktop locked. After unlocking it I attempted to send another e-mail and I once again received the "Unable to sign message. ..." message. I opened the Certificate Manager and the Trust CA A12 intermediate certificate was once again missing from the "Authorities" list.

I imported the certificate again. Sent an e-mail and confirmed that the signature was valid.

Returned to Thunderbird 102 and guess what? The "Trust CA A12" certificate is once again missing.

I imported the certificates once again. Verified that the "Trust CA A12" certificate was present and shutdown Thunderbird 102. Restarted it, entered my password, and the "Trust CA A12" certificate was present. Sent a third S/MIME signature test e-mail. Verified the signature.

Returned to TB 102 and waited five minutes. The cert was still there. Used the "Certificate Manager" "View ..." button to view the "Trust CA A12" certificate, all looked good. Closed the "Certificate Manager", opened it again and the "Trust CA A12" cert was gone.

Some code does not like that intermediate certificate and removes it from the "Software Security Device".

(In reply to Jeffrey Altman from comment #36)

Some code does not like that intermediate certificate and removes it from the "Software Security Device".

Re-imported the certificate. Began to write a long e-mail. TB102 attempted to save the draft and "Unable to sign message. ..." dialog appears. Open Certificate Manager and once again the "Trust CA A12" certificate is gone.

Is anyone familiar with a PKCS#11 provider to for Firefox/Thunderbird to permit use of the Windows Personal Certificate Store?

PKCS#11 and Windows' Certificate Store are different "stacks" to interact with certificates and key containers. Normally, when you are using the former, you are not using the latter, and vice versa.

(In reply to Adriano Santoni from comment #39)

PKCS#11 and Windows' Certificate Store are different "stacks" to interact with certificates and key containers. Normally, when you are using the former, you are not using the latter, and vice versa.

Firefox and Thunderbird do not use the certificates stored in the Windows Personal Certificate Store unless a PKCS#11 Provider is loaded to make those certificates and use of associated keys available. More than a decade ago I wrote such a PKCS#11 Provider for Firefox in order to permit Kerberized Certificate Authority issued short-lived X.509 certificates to be used for web site authentication. The PKCS#11 Provider I had at the time stopped working more than six years ago but was specific to the KCA issued certs.

If such a PKCS#11 provider was available, then the certs which are also present in the Windows Personal Certificate Store could be used from there as a workaround to the intermediate CA certificate disappears after a few minutes.

Blocks: tb102found

The enabled addons I have installed are Enigmail, Remove Duplicate Messages and Simple Mail Redirection.

I saw the automatic disappearance of the intermediate myself.
I will use a debug environment to watch for the general cert deleting code to be reached.
Hopefully that will succeed, and will tell me what other code triggers the automatic deletion.

I realized that while testing, one needs to be careful what emails you click.
Because, as soon as you click/view any email that was signed using a certificate that was issued by the same intermediate CA, simply clicking it will cause the intermediate to be available again (cached).

This means, as another temporary workaround, if you experience this bug, instead of manually importing the intermedia, it might be sufficient to simply click one of the signed emails that you have sent, found in your sent folder.

In other words, if you get the error, go to your sent folder, find a signed email you have sent, click it, then go back to your composer window and try to send again.

Hopefully I'll have news soon.

I found that the Mozilla core platform code, which is shared between Mozilla and Thunderbird, has introduced an automatic cleanup of intermediate CA certificates.

Firefox uses some more modern code for certificate verification, and it has access to a list of pre-loaded intermediates. This is likely the reason that viewing a certificate using certificate manager can display the full chain (additional tabs with intermediates and root), because this is core Firefox code.

The cleanup code runs regularly. It checks if the permanent storage contains intermediate CAs which are already contained in the preload list that Firefox uses. If it finds such certs, it deletes them from the permanent storage.

This is very likely causing this regression bug. Because Thunderbird uses older certificate verification code, which doesn't use the list of preloaded intermediates, it can no longer find the intermediate, because it was deleted.

The cleanup behavior can apparently be turned off using a preference.

If would like to help test, please open Thunderbird settings, open the config editor. In the search box, paste the following text:
security.intermediate_preloading_healer.enabled

Below the text you have entered, an additional line should appear, with the same security.intermediate_preloading_healer.enabled text, plus the text "true" in a second column.

Double click the word "true", and it should be automatically changed to false.

After this, ensure you have the intermediate CA imported one last time (for example by clicking a signed email that uses that intermediate).

Please report back if that fixes this bug for you.

If it does, as an automatic fix, we can try to set this setting for Thunderbird by default to false in the next update.

(In reply to Kai Engert (:KaiE:) from comment #42)

I realized that while testing, one needs to be careful what emails you click.
Because, as soon as you click/view any email that was signed using a certificate that was issued by the same intermediate CA, simply clicking it will cause the intermediate to be available again (cached).

This means, as another temporary workaround, if you experience this bug, instead of manually importing the intermedia, it might be sufficient to simply click one of the signed emails that you have sent, found in your sent folder.

I can confirm this behavior on my system as well. Leave a tab open to a signed message and click on the "S/MIME" button. Then sending of signed messages succeed.

(In reply to Kai Engert (:KaiE:) from comment #43)

If would like to help test, please open Thunderbird settings, open the config editor. In the search box, paste the following text:
security.intermediate_preloading_healer.enabled

Set to false and the intermediate CA cert is now visible without manual re-adding.

Sending an S/MIME signature worked after waiting multiple minutes.

Assignee: nobody → kaie
Status: NEW → ASSIGNED
Whiteboard: workaround: Use Settings, Config Editor, set security.intermediate_preloading_healer.enabled to false

Kai,
Thank you for finding the workaround. This is most helpful.
Jeffrey

Keywords: regression
Keywords: checkin-needed-tb

The code that has caused this Thunderbird S/MIME regression was added to Firefox in bug 1630434 [edited], but it has been turned off by default most of the time.

Bug 1769638 enabled the code by default in the 102 release cycle.

status-thunderbird103: --- → affected
status-thunderbird_esr102: --- → affected
Regressed by: 1769638

Comment on attachment 9284343 [details]
Bug 1777336 - Disable automatic intermediate CA cleanup, because it breaks S/MIME. r=rjl

[Approval Request Comment]
Regression caused by (bug #): 1769638
User impact if declined: broken S/MIME secure email feature
Testing completed (on c-c, etc.):
Risk to taking this patch (and alternatives if risky): very low, restores previous behavior

Attachment #9284343 - Flags: approval-comm-esr102?
Attachment #9284343 - Flags: approval-comm-beta?

Pushed by geoff@darktrojan.net:
https://hg.mozilla.org/comm-central/rev/93ff11032402
Disable automatic intermediate CA cleanup, because it breaks S/MIME. r=rjl

Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Keywords: checkin-needed-tb
Resolution: --- → FIXED
Target Milestone: --- → 104 Branch
Severity: -- → S2
tracking-thunderbird_esr102: --- → +
Priority: -- → P1
Summary: S/MIME Sending of signed messages fails after upgrade to TB102 → S/MIME Sending of signed messages fails after upgrade to TB102 due to intermediate cert handling

Comment on attachment 9284343 [details]
Bug 1777336 - Disable automatic intermediate CA cleanup, because it breaks S/MIME. r=rjl

[Triage Comment]
Approved for beta

Attachment #9284343 - Flags: approval-comm-beta? → approval-comm-beta+
Whiteboard: workaround: Use Settings, Config Editor, set security.intermediate_preloading_healer.enabled to false → [TM:102.0.3] workaround: Use Settings, Config Editor, set security.intermediate_preloading_healer.enabled to false

(In reply to Kai Engert (:KaiE:) from comment #20)

However, our S/MIME code is unfortunately rather old, and it isn't using the modern engine, and it can only use the certificates that have been permanently imported into the NSS database.

What are the plans to use the newest versions of Mozilla for s/mime? I see this patch as just that, a patch of something that needs updating before Mozilla pull the old code entirely.

(In reply to Matt from comment #54)

(In reply to Kai Engert (:KaiE:) from comment #20)

However, our S/MIME code is unfortunately rather old, and it isn't using the modern engine, and it can only use the certificates that have been permanently imported into the NSS database.

What are the plans to use the newest versions of Mozilla for s/mime? I see this patch as just that, a patch of something that needs updating before Mozilla pull the old code entirely.

Good point - would be great to use the latest code everywhere...?

At least the workaround seems to work for now.

Comment on attachment 9284343 [details]
Bug 1777336 - Disable automatic intermediate CA cleanup, because it breaks S/MIME. r=rjl

[Triage Comment]
Approved for esr102

Attachment #9284343 - Flags: approval-comm-esr102? → approval-comm-esr102+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: