51409 - SSL_DataPending nearly always returns zero

Status: NEW

(NSS :: Libraries, defect, P3)

Description

[Nelson Bolyard (seldom reads bugmail)]

This was originally bugsplat bug 338596, dated 1998-12-31.

The PR_Available call on an SSL socket nearly always returns zero,
even if there are one or more SSL records waiting to be read out of 
the underlying socket.  This makes PR_Available useless with SSL on
non-blocking sockets.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

At one time, years ago, I considered changing PR_Available for SSL sockets 
so that if no data was already available (that is, if SSL_DataPending 
returns zero) it would check the underlying socket to see if data was 
available on it, and if so, would read from that socket, in hopes that it
would retrieve one or more SSL records containing application data.  

There are a number of potential complications with that scheme.  It may be
that the record read in is a handshake record.  Continuing the handshake at
that point could potentially require :

a) cryptographic computations (e.g. a signature for client auth, or a 
decryption of a received encrypted pre-master secret.  Those, in turn,
could require the presentation of a dialog to obtain a password.  The 
application likely will not expect to handle a password callback in 
response to calling PR_Available.  or

b) sending out handshake records on a socket, which might have insufficient
buffer space to send them (e.g. EWOULDBLOCK on the send call).  The 
application might not realize it needs to poll/select on write after a 
call to PR_Available (!).  

Either of these scenarious might make the PR_Available call take quite a 
while, but applications expect that PR_available will be more-or-less
instantaneous, and will not initiate I/O.  

Some of my coworkers thought it was a bad idea for PR_Available to initiate
I/O, and no-one was pressing for this bug to be fixed, so it was "futured"
and languished there.

If someone now believes that this is a higher priority than in the past, 
Please speak up.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

I just realized that we never made SSL's PRIOMethods table for PR_Available 
point to SSL_DataPending().  So, the truth is that SSL_DataPending is the 
function that nearly always returns zero (except right after 
SSL_ForceHandshake returns), and PR_Available on an SSL socket tends to 
return -1 with a NOT_IMPLEMENTED error.  :-(

So, I'm changing this bug to name SSL_DataPending again.  Guess I should 
file another bug about the fact that PR_Available doesn't call this function.