309580 - NSS attempting RC2 encryption on PKCS11 module that can only wrap/unwrap

Status: RESOLVED INACTIVE

(NSS :: Libraries, defect)

Description

[Nelson Bolyard (seldom reads bugmail)]

A vendor of a forthcoming crypto accelerator device has been testing 
his PKCS11 module with servers that use NSS 3.10.1.   He reports several
disconcerting things about NSS's behavior.   I am reporting them here.
I consider this bug unconfirmed until more tests are done.

The Vendor claims the following:

- his PKCS11 module can do RC2 wrap and unwrap, but not encrypt or decrypt.

- his PKCS11 module advertises CKM_RC2_CBC, but the flags returned from 
C_GetMechanismInfo show that it can only be used for wrap and unwrap, 
not for encrypt and decrypt.  However, ...

- his PKCS11 module receives C_GetMechanismInfo calls for the mechanisms 
CKM_RSA_PKCS and CKM_DES3_ECB but not other mechanisms.  

- his module does all the SSL/TLS key derivation mechanisms, and contains
the private key for the cert.  (So, it is likely that all key derivation 
from the PMS onward is done in his module.)

- when he tests with an RC2 ciphersuite, NSS tries to do the RC2 
encryption and decryption with his module, even though his module 
cannot do those algorithms.  This leads to failure to interoperate using
RC2 cipher suites when his module is enabled.  


His immediate request is for a workaround or fix that will enable a server
to serve all the ciphersuites, including RC2 ones, while his module is 
configured in secmod.db and while it is claiming CKM_RC2_CBC.  
He believes that changing his module to not show support for CKM_RC2_CBC 
would make RC2 work again in NSS.  I don't know that he's tested that.
But he does not want to disable his module's capability just to work around 
this issue, which he perceives to be an NSS bug.


My guesses are that either:
a) NSS isn't checking that his module can do CKM_RC2_CBC encryption, or 
b) the check is returning a false positive because it supports some RC2_CBC
functionality.  In other words, perhaps NSS's test for "can this slot 
support this mechanism" isn't fine-grained enough to notice the distinction
between wrapping and encryption capabilities.  

I'm hoping that the PKCS11 debug logging abilities of NSS 3.10.x will 
suffice to provide clear evidence of what's really going on at the PKCS11
interface to NSS.  We're looking into that.

Comment 1

[Robert Relyea]

The issue described isn't new for 3.10, it's been in place since before NSS 3.3
(when we first were export certified to include PKCS #11 modules, long before we
were open source). The problem is we test the following algorithms to make sure
the operate as advertized (that is someone hasn't replaced one of these
algorithms with something 'stronger'). We need to do this because we need to use
strong versions of these ciphers, even to implement weak versions of SSL cipher
suites.

Strong use of strong ciphers were then prohibitted by policy.

Probably the easiest way to handle this is if a module fails the encryption test
to turn off that cipher rather than fail the whole module. This shouldn't be a
problem for this token, as you will need to encrypt/decrypt in order to to an
RC2 SSL cipher or RC2 S/MIME.

I can attach a patch pretty quickly, once I get my pk11wrap area clear.

bob