603604 - The Way NSS uses a PKCS# 11 library may be incorrect

Status: RESOLVED INCOMPLETE

(NSS :: Libraries, defect, P5)

Description

[Augustin P. SARR]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 (.NET CLR 3.5.30729)

Whith a PKCS#11 implementation supporting the CKM_RSA_PKCS mechanism for encryption, but not for signature, SSL connections may fail with error SEC_ERROR_INVALID_ALGORITHM
The reason is that NSS only list mechanisms, whitout trying to get mechanism infos.

A PKCS#11 mechanism may be implemented for encryption, but not for signature
(this is allowed by the PKCS#11 standard specification -- version 2.20).
When using a PKCS#11 library, NSS lists the mechanisms, and may try to use one of them for a cryptographic operation, prior to calling C_GetMechanismInfo to know whether or not the mechanism is implemented for the operation.
C_GetMechanismList is not enough; C_GetMechanismInfo should also be called to know, whether or not a particular (public key) mechanism is only implemented for encryption or both encryption and signature.


Reproducible: Always

Steps to Reproduce:
1. Load a PKCS#11 library, which supports the CKM_RSA_PKCS mechanism only for encryption
2. Connect to server, using SLL, and negociate the key handshake to have CKM_RSA_PKCS signature mechanism
3. NSS will try to use the CKM_RSA_PKCS mechanism for signature, and to connection will end with error SEC_ERROR_INVALID_ALGORITHM

Actual Results:  
SLL connection failure with error  SEC_ERROR_INVALID_ALGORITHM


Expected Results:  
This should not occur.
Prior to trying to use a mechanism for an operation, NSS should call G_GetMechanismInfo to know whether or not the mechanism is implemented for the operation