449087 - PK11_PubWrapSymKey ignores mechanism type when no slot implements it, and uses another

Status: NEW

(NSS :: Libraries, defect, P3)

Description

[Nelson Bolyard (seldom reads bugmail)]

Reported in dev-tech-crypto mailing list by Dave Allen in thread entitled
"RSA OAEP encryption support in NSS" on 2008-08-02.

When PK11_PubWrapSymKey is called with a mechanism type argument that is 
not implemented in any available slot/token, rather than reporting an error,
it ignores the specified mechanism type and uses CKM_RSA_PKCS.  

The problem is in the use (and design) of the private function pk11_ForceSlot.  
That function can have any of the following outcomes:

1) the key is already in a slot that does the requested mechanism.
	SUCCESS: returns NULL
2) the key is not already in a slot that does the requested mechanism,
   and another slot can be found that does it.  It attempts to copy the
   symkey to the slot that can do the mechanism, and that succeeds.
   It creates a new PK11SymKey struct for the new copy of the key.
   This all succeeds:
	SUCCESS: returns address of new symkey
3) Same as 2 above, but fails to copy key to new slot, or fails to allocate
   new symkey struct.
	FAILURE: returns NULL
4) the key is not already in a slot that does the requested mechanism,
   and no slot exists that can do it.
	FAILURE: returns NULL

As you can see from that description, a NULL return is entirely ambiguous.
It means either
a) the key is fine and no slot change is needed (SUCCESS), or
b) the operation CANNOT succeed because the key is not in a slot that can
   do the requested mechanism (FAILURE).

IMO, pk11_ForceSlot should be changed to return a success/failure indicator
that is independent of whether a new symkey was allocated or not.  It
should also be able to output a new symkey.

The next step is to look at all the callers of pk11_ForceSlot and see just
how big a change it would take to fix this.

Comment 1

[Robert Relyea]

So the problem here is likely a problem in hand_wrap, not in pk11_ForceSlot().

Even though we continue past forceSlot(), we still should still call the resulting slot with the original mechanism. That should fail and we fall back to doing a handunwrap(). I suspect that hand unwrap trying to map the wrapping mechanism to an encryption mechanism, and it's not properly handling, nor recognizing that it can't handle the AEOP mechanism.


bob