294812 - Most PK11 crypto operation failures set the SEC_ERROR_IO error code.

Status: RESOLVED INACTIVE

(NSS :: Libraries, defect, P5)

Description

[Wan-Teh Chang]

In the past two weeks two NSS users reported that an
NSS crypto function failed with the SEC_ERROR_IO error
code.

Looking at our source code, I found that this is
because in lib/softoken/pkcs11c.c, we return CKR_DEVICE_ERROR
when (*context->update)() fails, ignoring the error code
set by (*context->update)(), which is some function in
lib/freebl, and then lib/pk11wrap maps CKR_DEVICE_ERROR
to SEC_ERROR_IO.

I've since changed our error code documentation at
http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html#1039257
to say:

SEC_ERROR_IO  An I/O error occurred during authentication; or
              an error occurred during crypto operation (other
              than signature verification).

Note: If (*context->verify)() fails, lib/softoken/pkcs11c.c
returns CKR_SIGNATURE_INVALID, which lib/pk11wrap maps to
SEC_ERROR_BAD_SIGNATURE, hence the "(other than signature verification)"
in the SSL Reference.

Is this new description of SEC_ERROR_IO clear?  Do we
need to do more?

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

IMO, yes, we need to greatly improve our error reporting.  
There are FAR too many places where we report some generic error 
instead of a useful one, even in cases where a useful one is defined.
IMO, SEC_ERROR_IO should be reserved for failures that *cannot* be 
better diagnosed.  
I think we should consider any occurence of SEC_ERROR_IO at the 
end of a cryptographic operation that did not involve hardware to be
a BUG in NSS.  
Another bug that is used far too often (and in cases where it is simply
wrong) is SEC_ERROR_NO_MEMORY.  

Comment 2

[Wan-Teh Chang]

I plan to fix this bug as follows.  Please review
this strategy.

1. Compile a list of all NSS error codes set by the
functions in lib/freebl.

2. Create a table that maps those NSS error codes to
PKCS #11's function return values (CKR_xx).  Section
11.1 "Function return values" in PKCS #11 v2.20 is
useful for this step.

3. Implement a function in lib/softoken that maps
those NSS error codes to PKCS #11 function return
values and have lib/softoken call this function
when (*context->update)() fails.

4. Finally, review the PK11_MapError function in
lib/pk11wrap, which should be the inverse of the
lib/softoken error code mapping function.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months, so the assignee is being reset.