242984 - workaround for incomplete NSPR I/O layer in lib/ssl

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P2)

Description

[Julien Pierre]

We have an application that has its own base NSPR I/O layer for sockets.
Unfortunately, its layer is incomplete, and many callbacks were set to NULL. The
applications functioned with NSPR 4.1 and NSS 3.3, but when moving to NSS 3.9
and NSPR 4.4, core dumps are seen upon SSL socket close due to the changes made
to disable the nagle delay, and the application NSPR layer's NULL
setsocketoption callback .

We are making a fix to the application so that it has a complete NSPR I/O layer
(at least, stub functions). However, because the update of the application and
the NSPR/NSS components does not always happen simultaneously, I would like to
make a workaround in libssl to check whether the setsocketoption callback is
non-NULL before calling it, for NSS 3.9.2 .

Comment 1

[Julien Pierre]

Attachment: check for missing setsocketoption method

Comment 2

[Wan-Teh Chang]

Comment on attachment 147947 [details] [diff] [review]
check for missing setsocketoption method

sr=wtc.

May want to make the types right as follows.

1. Declare 'rv' with the type SECStatus.
2. Add a (SECStatus) typecast before
osfd->methods->setsocketoption(osfd, &opt),
which returns PRStatus.

Comment 3

[Wan-Teh Chang]

Comment on attachment 147947 [details] [diff] [review]
check for missing setsocketoption method

Julien, you should call PORT_SetError(PR_NOT_IMPLEMENTED_ERROR)
if osfd->methods->setsocketoption is NULL.  This is more
important than getting the types (int/PRStatus/SECStatus)
right.	Please submit a new patch.  Thanks.

Comment 4

[Wan-Teh Chang]

Attachment: check for missing setsocketoption method, v2

Set the PR_NOT_IMPLEMENTED_ERROR error code if
the setsocketoption method is missing.

Feel free to edit this patch for taste.

Comment 5

[Julien Pierre]

Comment on attachment 148144 [details] [diff] [review]
check for missing setsocketoption method, v2

Looks good. I personally prefer to have fewer points of return (see the
previous patch), but this works too.

Comment 6

[Julien Pierre]

Attachment: third patch. stylistic changes, as well as typecasts

Comment 7

[Nelson Bolyard (seldom reads bugmail)]

Comment on attachment 148156 [details] [diff] [review]
third patch. stylistic changes, as well as typecasts

r=MisterSSL

Comment 8

[Julien Pierre]

Fixed for 3.9.2 and tip.

Checking in sslsock.c;
/cvsroot/mozilla/security/nss/lib/ssl/sslsock.c,v  <--  sslsock.c
new revision: 1.31.38.1; previous revision: 1.31

Checking in sslsock.c;
/cvsroot/mozilla/security/nss/lib/ssl/sslsock.c,v  <--  sslsock.c
new revision: 1.34; previous revision: 1.33

Comment 9

[Nelson Bolyard (seldom reads bugmail)]

Comment on attachment 148144 [details] [diff] [review]
check for missing setsocketoption method, v2

removing review request for obsolete patch