51559 - SSL lib returns SEC_ERROR_EXTENSION_NOT_FOUND error

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P3)

Description

[Nelson Bolyard (seldom reads bugmail)]

Several people have reported that the NSS SSL Server code 
sometimes returns SEC_ERROR_EXTENSION_NOT_FOUND to a call that's
doing a handshake (e.g. SSL_ForceHandshake or PR_Read). 
This error has also been reported appearing in iWS server logs.
At least one user reports that this happens when the server 
requests client authentication and the client sends a cert to the 
server.  

I believe the problem is in the function ssl3_HandleCertificate()
in ssl3con.c.  At the end of the that function, errCode is set to
the proper error code, and it calls ssl_MapLowLevelError() which 
replaces the already-set PR error code IFF it's currently set to 
any of several rather meaningless error code values.  If the error
code has already been set to SEC_ERROR_EXTENSION_NOT_FOUND, then 
ssl_MapLowLevelError() does not change it.  

There are several possible solutions here, and they're not all
mutually exclusive.  

a) be sure the error code is cleared (set to zero) at the beginning
   of this function.
b) add SEC_ERROR_EXTENSION_NOT_FOUND to the list of errors that are 
   overridden by ssl_MapLowLevelError() 
c) change the logic of ssl3_HandleCertificate() to set some errors
   by calling PORT_SetError directly.

It would be good to fix this in the next release of NSS, since 
multiple people have experienced (or are experiencing) this error.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Two adiditional comments:
1. The PR error code _IS_ always set to zero before processing any 
   SSL handshake message, and
2. This problem has also been reported about the error message
   SSL_ERROR_SESSION_NOT_FOUND under the same circumstances.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

I believe this is fixed by my recent checkins to sslslnce.c and sslerr.c
I will reopen this if not.

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

Changed target fix version to 3.0.1