215997 - RSAES-OAEP decryption support in S/MIME

Status: RESOLVED FIXED

(NSS :: Libraries, enhancement, P1)

Description

[Julien Pierre]

RFC3560 specifies the use of RSAES-OAEP in S/MIME .
It is available from http://www.ietf.org/rfc/rfc3560.txt .
We should support it when we add support for RSAES-OAEP to NSS .

Comment 1

[Nicholas Sushkin]

More recently, AES (RFC 3565) has been added as a SHOULD implement to S/MIME 3.1 (RFC 3851).

http://www.apps.ietf.org/rfc/rfc3851.html#sec-2.7
http://www.apps.ietf.org/rfc/rfc3565.html

2.7 ContentEncryptionAlgorithmIdentifier

Sending and receiving agents MUST support encryption and decryption with DES EDE3 CBC, hereinafter called "tripleDES" [CMSALG]. Receiving agents SHOULD support encryption and decryption using the RC2 [CMSALG] or a compatible algorithm at a key size of 40 bits, hereinafter called "RC2/40". Sending and receiving agents SHOULD support encryption and decryption with AES [CMSAES] at a key size of 128, 192, and 256 bits.

...

   [CMSAES]      Schaad, J., "Use of the Advanced Encryption Standard
                 (AES) Encryption Algorithm in Cryptographic Message
                 Syntax (CMS)", RFC 3565, July 2003.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

AES support in libSMIME should be a separate enhancement request.
Nicholas, please file a new enhancement request, product NSS, 
component libraries.  Give the same text you gave in comment 1 above
(plus any more you deem necessary).  Please CC me on that bug.

Comment 3

[Kai Engert [:KaiE:]]

Thunderbird needs this improvement for bug 1826086.

I've been told that NIST has deprecated the RSA PKCS#1 v1.5 padding for encryption.
Support for RSA-OAEP is necessary for compliance and compatibility with other email clients.

Comment 4

[Kai Engert [:KaiE:]]

Using the test data that was contributed in bug 1826086, I had an initial look at our level of support so far.

When parsing an incoming CMS message, we need the ability to parse the parameters for the RSAES-OAEP encryption, as described in RFC 8017 section A.2.1
https://www.rfc-editor.org/rfc/rfc8017#appendix-A.2.1
(I currently don't see existing templates in NSS for parsing these parameters.)

We'll also need encoding for CMS messages we send.

The contributed patches in bug 676118 might serve as inspiration.

Comment 5

[Kai Engert [:KaiE:]]

Attachment: 215997-decode-partial-v1.patch

Initial code to hook up decryption. Misses parameter decoding.

Comment 6

[Kai Engert [:KaiE:]]

(In reply to Kai Engert (:KaiE:) from comment #4)

When parsing an incoming CMS message, we need the ability to parse the parameters for the RSAES-OAEP encryption, as described in RFC 8017 section A.2.1
https://www.rfc-editor.org/rfc/rfc8017#appendix-A.2.1

See also:
https://datatracker.ietf.org/doc/html/rfc3560#section-3

Comment 7

[David Cooper]

About 4 years ago I created a patch to decrypt messages that were encrypted using OAEP. However, I never bothered to submit it since bug 676100 and bug 676118 were never reviewed. I haven't looked at the OAEP patch that I created in 4 years, but if there is real interest I can bring the patch up to date and submit it. However, I'd prefer not to spend any time on it if it would just suffer the same fate as the ECDH patches.

Comment 8

[David Cooper]

Attachment: Decrypt RSA OAEP encrypted messages

I've attached the patch that I created 4 years ago, with a few minor updates. I verified that it compiles, the NSS tests run successfully, and I can decrypt the test messages provided in bug 1826086. Feel free to make any modifications to the patch or to review and adopt it, as appropriate.

Comment 9

[Anna Weine]

(In reply to David Cooper from comment #8)

Created attachment 9355157 [details] [diff] [review]
Decrypt RSA OAEP encrypted messages

I've attached the patch that I created 4 years ago, with a few minor updates. I verified that it compiles, the NSS tests run successfully, and I can decrypt the test messages provided in bug 1826086. Feel free to make any modifications to the patch or to review and adopt it, as appropriate.

Hi,
Could you submit the patch through phabricator? https://phabricator.services.mozilla.com/
Thanks!

Comment 10

[David Cooper]

Attachment: Bug 215997 - Decrypt messages that were encrypted using RSA-OAEP.

Comment 11

[David Cooper]

Attachment: Bug 215997 - Decrypt RSA-OAEP encrypted messages.

Comment 12

[David Cooper]

Attachment: Bug 215997 - Fix smime.sh

Depends on D189584

Comment 13

[Kai Engert [:KaiE:]]

Today I saw the first complaint that Thunderbird cannot decrypt a message that uses rsaOAEP, see 1870629.

Comment 14

[Anna Weine]

Is this patch https://phabricator.services.mozilla.com/D189689 supposed to be empty?

This has the same problem - https://phabricator.services.mozilla.com/D189690

Comment 15

[Josef Ender]

(In reply to Kai Engert (:KaiE:) from comment #13)

Today I saw the first complaint that Thunderbird cannot decrypt a message that uses rsaOAEP, see 1870629.

Yes, from me. Unfortunately also multiple client complain they cannot decrypt messages from seppmail. Seppmail is widespred in Switzerland and Germany.

Comment 16

[chokito]

I have the same problem. Once SEPPmail is involved, Thunderbird cannot decrypt the email. If the email is selected in the inbox, the text "Thunderbird cannot decrypt the message" appears in the message window and no headers are displayed. When the email is saved externally and then opened, headers appear in the message window, but the message window itself remains blank. With previous emails where SEPPmail was also involved, this always worked without any problems. To test it yourself, send a signed email to <[e-mail]> asking you to reply in encrypted form.

Comment 17

[Kai Engert [:KaiE:]]

Bitrot again. I'll abandon all phab revision. I was able to merge David's patch again, and it still passes tests.
I'll try to review, ask rrelyea for comments, and try to push it over the finish line.

Comment 18

[Kai Engert [:KaiE:]]

Attachment: Bug 215997 - Decrypt RSA OAEP encrypted messages. r=kaie

Comment 19

[Kai Engert [:KaiE:]]

Attachment: WIP: Bug 215997 - Test RSA OAEP decryption. Patch by David Cooper.

Comment 20

[Kai Engert [:KaiE:]]

The patches provided implement decryption.
They don't yet provide encryption.

I'd like to get decryption landed quickly, so it can be included in the next Thunderbird release.

Because the timing of the availability of encryption will likely differ from decryption,
I'd like to split encryption using OAEP into a separate NSS bug.

Comment 21

[Kai Engert [:KaiE:]]

I've landed David's patches, with minor tweaks and fixes.
https://hg.mozilla.org/projects/nss/rev/c27d30f6d576da78fefcf30099e2c6c1afbbe713
https://hg.mozilla.org/projects/nss/rev/273b546db041752cb6d028c2a939b4e9ee604622

I didn't fully update phabricator, it isn't working well for this scenario here, were the person handling the phabrictor revision is the reviewer, not the original author.

Comment 22

[Kai Engert [:KaiE:]]

Attachment: Bug 215997 - Follow-up to add common function SEC_GetMgfTypeByOidTag. r=rrelyea

Comment 23

[BugBot [:suhaib / :marco/ :calixte]]

A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)

Comment 24

[Kai Engert [:KaiE:]]

The follow-up patch was commited today, but was too late for the 3.100 release.
I'll move it to a separate bug (for 3.101 tracking) and file another follow-up to adjust versions.

Comment 25

[Kai Engert [:KaiE:]]

(In reply to Kai Engert (:KaiE:) from comment #24)

The follow-up patch was commited today, but was too late for the 3.100 release.
I'll move it to a separate bug (for 3.101 tracking) and file another follow-up to adjust versions.

See bug 1895367

I cannot move the phabricator revision of the follow-up over to the other bug.
I was commited as
https://hg.mozilla.org/projects/nss/rev/de11cc24384a09a76785f8e5c97f83f7b3e3b2fa
with the version fixed in
https://hg.mozilla.org/projects/nss/rev/dd30e09bd4d3c2bf9279ac6fa4631c34d9d19403

Comment 26

[Anna Weine]

Attachment: Bug 215997 - Clang-formatting of SEC_GetMgfTypeByOidTag update

Comment 27

[John Schanck [:jschanck] (out of office Aug 11-18)]

https://hg.mozilla.org/projects/nss/rev/e009d9ba2cb33b19092339bd592b1b3f9e5c5287