Relevant code is exposed through RSA_Sign and RSA_Encrypt. Here's the call path for Sign: https://searchfox.org/mozilla-central/source/security/nss/lib/softoken/pkcs11c.c#2622 https://searchfox.org/mozilla-central/source/security/nss/lib/softoken/pkcs11c.c#2417 https://searchfox.org/mozilla-central/source/security/nss/lib/freebl/rsapkcs.c#1307 https://searchfox.org/mozilla-central/source/security/nss/lib/freebl/rsapkcs.c#241 (line 239 is not compiled in release builds) Encrypt (as in the added test) is similar. rsa_FormatBlock's case for `RSA_BlockRaw` checks the length, so actually CKM_RSA_X_509 is not effected.
Bug 1539788 Comment 16 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Relevant code is exposed through RSA_Sign and RSA_Encrypt. Here's the call path for Sign: https://searchfox.org/mozilla-central/source/security/nss/lib/softoken/pkcs11c.c#2622 https://searchfox.org/mozilla-central/source/security/nss/lib/softoken/pkcs11c.c#2417 https://searchfox.org/mozilla-central/source/security/nss/lib/freebl/rsapkcs.c#1307 https://searchfox.org/mozilla-central/source/security/nss/lib/freebl/rsapkcs.c#241 (line 239 is not compiled in release builds) Encrypt (as in the added test) is similar. rsa_FormatBlock's case for `RSA_BlockRaw` checks the length, so actually CKM_RSA_X_509 is not affected.