356623 - ssl needs finer grained error codes for server key derivation failures

Status: NEW

(NSS :: Libraries, enhancement, P3)

Description

[Julien Pierre]

I configured selfserv to run with Solaris Crypto Framework . I used the cipher suite SSL3 RSA WITH AES 256 CBC SHA .
The Solaris crypto framwork supports AES, but currently only for 128 bits.
As a result, C_DeriveKey failed with 0x13 - CKR_ATTRIBUTE_VALUE_INVALID .
PK11_Derive failed and returned a NULL key.
Then, libssl sets the following error :

Current function is ssl3_DeriveConnectionKeysPKCS11
 2766           ssl_MapLowLevelError(SSL_ERROR_SESSION_KEY_GEN_FAILURE);

This is not the correct error to use. There was no key generation, but rather a key derivation.

(dbx) where
current thread: t@3
  [1] PORT_SetError(value = -12207), line 181 in "secport.c"
  [2] ssl_MapLowLevelError(hiLevelError = -12207), line 68 in "sslerr.c"
=>[3] ssl3_DeriveConnectionKeysPKCS11(ss = 0x81a7ca0), line 2766 in "ssl3con.c"
  [4] ssl3_InitPendingCipherSpec(ss = 0x81a7ca0, pms = 0x8197e90), line 1541 in "ssl3con.c"
  [5] ssl3_HandleRSAClientKeyExchange(ss = 0x81a7ca0, b = 0x819a48c "", length = 130U, serverKey = 0x818bfd0), line 6527 in "ssl3con.c"
  [6] ssl3_HandleClientKeyExchange(ss = 0x81a7ca0, b = 0x819a48c "", length = 130U), line 6620 in "ssl3con.c"
  [7] ssl3_HandleHandshakeMessage(ss = 0x81a7ca0, b = 0x819a48c "", length = 130U), line 7722 in "ssl3con.c"
  [8] ssl3_HandleHandshake(ss = 0x81a7ca0, origBuf = 0x81a7efc), line 7793 in "ssl3con.c"
  [9] ssl3_HandleRecord(ss = 0x81a7ca0, cText = 0xfe60b12c, databuf = 0x81a7efc), line 8056 in "ssl3con.c"
  [10] ssl3_GatherCompleteHandshake(ss = 0x81a7ca0, flags = 0), line 206 in "ssl3gthr.c"
  [11] ssl_GatherRecord1stHandshake(ss = 0x81a7ca0), line 1258 in "sslcon.c"
  [12] ssl_Do1stHandshake(ss = 0x81a7ca0), line 149 in "sslsecur.c"
  [13] ssl_SecureRecv(ss = 0x81a7ca0, buf = 0xfe60b51c "", len = 10239, flags = 0), line 1032 in "sslsecur.c"
  [14] ssl_SecureRead(ss = 0x81a7ca0, buf = 0xfe60b51c "", len = 10239), line 1051 in "sslsecur.c"
  [15] ssl_Read(fd = 0x81799b0, buf = 0xfe60b51c, len = 10239), line 1434 in "sslsock.c"
  [16] PR_Read(fd = 0x81799b0, buf = 0xfe60b51c, amount = 10239), line 141 in "priometh.c"
  [17] handle_connection(tcp_sock = 0x81799b0, model_sock = 0x8083740, requestCert = 0), line 960 in "selfserv.c"
  [18] jobLoop(a = (nil), b = (nil), c = 0), line 518 in "selfserv.c"
  [19] thread_wrapper(arg = 0x817e0d4), line 486 in "selfserv.c"
  [20] _pt_root(arg = 0x817e320), line 220 in "ptthread.c"
  [21] _thr_setup(0xfe780000), at 0xfebff9ae
  [22] _lwp_start(), at 0xfebffc90
(dbx)

This is the stack of the derivation failure :

=>[1] pk11_DeriveWithTemplate(baseKey = 0x8172ae0, derive = 886U, param = 0xfe60ae1c, target = 4226U, operation = 260U, keySize = 32, userAttr = (nil), numAttrs = 0, isPerm = 0), line 1460 in "pk11skey.c"
  [2] PK11_Derive(baseKey = 0x8172ae0, derive = 886U, param = 0xfe60ae1c, target = 4226U, operation = 260U, keySize = 32), line 1316 in "pk11skey.c"
  [3] ssl3_DeriveConnectionKeysPKCS11(ss = 0x81a7ca0), line 2764 in "ssl3con.c"
  [4] ssl3_InitPendingCipherSpec(ss = 0x81a7ca0, pms = 0x8197e90), line 1541 in "ssl3con.c"
  [5] ssl3_HandleRSAClientKeyExchange(ss = 0x81a7ca0, b = 0x819a48c "", length = 130U, serverKey = 0x818bfd0), line 6527 in "ssl3con.c"
  [6] ssl3_HandleClientKeyExchange(ss = 0x81a7ca0, b = 0x819a48c "", length = 130U), line 6620 in "ssl3con.c"
  [7] ssl3_HandleHandshakeMessage(ss = 0x81a7ca0, b = 0x819a48c "", length = 130U), line 7722 in "ssl3con.c"
  [8] ssl3_HandleHandshake(ss = 0x81a7ca0, origBuf = 0x81a7efc), line 7793 in "ssl3con.c"
  [9] ssl3_HandleRecord(ss = 0x81a7ca0, cText = 0xfe60b12c, databuf = 0x81a7efc), line 8056 in "ssl3con.c"
  [10] ssl3_GatherCompleteHandshake(ss = 0x81a7ca0, flags = 0), line 206 in "ssl3gthr.c"
  [11] ssl_GatherRecord1stHandshake(ss = 0x81a7ca0), line 1258 in "sslcon.c"
  [12] ssl_Do1stHandshake(ss = 0x81a7ca0), line 149 in "sslsecur.c"
  [13] ssl_SecureRecv(ss = 0x81a7ca0, buf = 0xfe60b51c "", len = 10239, flags = 0), line 1032 in "sslsecur.c"
  [14] ssl_SecureRead(ss = 0x81a7ca0, buf = 0xfe60b51c "", len = 10239), line 1051 in "sslsecur.c"
  [15] ssl_Read(fd = 0x81799b0, buf = 0xfe60b51c, len = 10239), line 1434 in "sslsock.c"
  [16] PR_Read(fd = 0x81799b0, buf = 0xfe60b51c, amount = 10239), line 141 in "priometh.c"
  [17] handle_connection(tcp_sock = 0x81799b0, model_sock = 0x8083740, requestCert = 0), line 960 in "selfserv.c"
  [18] jobLoop(a = (nil), b = (nil), c = 0), line 518 in "selfserv.c"
  [19] thread_wrapper(arg = 0x817e0d4), line 486 in "selfserv.c"
  [20] _pt_root(arg = 0x817e320), line 220 in "ptthread.c"
  [21] _thr_setup(0xfe780000), at 0xfebff9ae
  [22] _lwp_start(), at 0xfebffc90

Comment 1

[Julien Pierre]

I believe there is a separate patch for Crypto Framework that may allow AES256, but by default only AES128 is supported.
This bug is similar to bug 356897 that I just filed. There was a key derivation failure. NSS might actually be able to recover by switching to another slot. I'm not sure if this is desirable, or if this is simply something the server administrator should avoid doing - ie. don't configure SSL with AES256 if the token that contains the server private key doesn't support deriving keys with AES256 .

Most server administrators will not test every single SSL cipher suite available, and if the default they have is not failing, they won't know about the problem.

It would be better if we could either disable the SSL cipher suite early in this type of situation, or be able to switch slots.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

As noted in bug 356897, NSS's libSSL should detect that we cannot do AES256
and not negotiate a cipher suite that we cannot do.  The failure to be able
to perform a certain cipher after negotiating it should never occur, except
perhaps in unusual cases of PKCS#11 tokens being removed while in use.

Comment 3

[Julien Pierre]

Yes, tokens that get removed would be a problem. Hopefully the token that contains the server private key does not get removed, otherwise SSL is not going to work, anyway ;).

In the case of a virtual token like the metaslot however, I don't know if we can assume that the supported mechanism list stays the same during the time the token is "plugged in". I hope we can.

Comment 4

[Julien Pierre]

Actually the supported mechanism list is a property of the slot, not token. But in the current PKCS#11 specs, slots can be removed too.

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

Unsetting target milestone in unresolved bugs whose targets have passed.