389568 - SSL stress tests fail for shared DB on Solaris.

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P2)

Description

[Robert Relyea]

Turning on SSL stress with shared DBs on Solaris causes the client auth SSL stress tests fail.

Historical notes: Other platforms (like linux) failed the stress tests until a thread safe version of sqlite was compiled.  Currently solaris is compiling sqlite with threed safe turned on.

Comment 1

[Robert Relyea]

OK, this turns out not to  be a race in sqlite. It's a 'run out of file descriptors' problem.

sqlite cannot share database handles between threads. To handle this sdb.c opens a database as needed on each operation. If all enough threads find themselves in an sdb operation it's possible to run out of file descriptors (solaris has max file descriptors set to 256, Linux seems to top out at 1000 some). 

Possible solutions:
1. reduce the threads on stress test.
2. increase the file descriptors count on solaris.
3. detect the "out of descriptor" conditions and wait for the file descriptors
to free up in sdb.c.
4. change the way db descriptors are used.
5. fix sqlite3.
6. I'm open to suggestions because 1-5 all sound non-optimal to me..

At this point 3 looks like the best short term solution.

bob

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

Bob, good find.  Within NSS, how does this error manifest itself? 
What NSS operations fail, and what NSS error codes do they return?

(embarassing admission) I don't know how to increase the per-process limit on
file descriptors in Solaris, but if it's easy, I'd suggest that as the short 
term fix. 

Comment 3

[Julien Pierre]

Nelson,

Re: comment 2, how to do that depends on your shell . Typically, it is something like unlimit, or ulimit. You can usually check the current values with limit . There is also a maximum per-user that can be configured separately. Root is typically unlimited (well, 4gb file descs) on Solaris.

Comment 4

[Robert Relyea]

Nelson,

It manifests itself in a couple of ways, typically a failure to sign (err is an invalid key for operation). The error PKCS #11 is returning is CKR_NETSCAPE_BAD_KEY_DB. The primary error is lost to us immediately in SQLITE (which returns simply SQLITE_CANTOPEN).

Julien,

I'm running Solaris 10 x86 (under vmware, but the latter shouldn't matter for this). ulimit -a shows a file descriptor limit of 256, setting ulimit -n 512 doesn't seem to do anything. Same with a root shell.;(.

bob

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

In reply to comment 4: Bob, I meant programmatically.  
We don't want browser users to need to change shell scripts.

Comment 6

[Julien Pierre]

Bob,

That's odd. I installed s10 x86 myself and this is what I got. The box is using NIS, but I don't think that matters for the root user. I am using tcsh as the shell for both myself and root .

If you want to raise the per-user limits, I believe you need to use the Solaris   Management Console (SMC). Run as root and type smc . Make sure your xhost and DISPLAY are set properly. Yes, it is a GUI progrma. Java even. Sigh.

[jp96085@monstre]/home/jp96085 1 % limit
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       10240 kbytes
coredumpsize    unlimited
vmemoryuse      unlimited
descriptors     65536
[jp96085@monstre]/home/jp96085 2 % unlimit
[jp96085@monstre]/home/jp96085 3 % limit
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       unlimited
coredumpsize    unlimited
vmemoryuse      unlimited
descriptors     65536
[jp96085@monstre]/home/jp96085 4 % su -
Password:
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
# limit
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       unlimited
coredumpsize    unlimited
vmemoryuse      unlimited
descriptors     65536
# unlimit
# limit
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       unlimited
coredumpsize    unlimited
vmemoryuse      unlimited
descriptors     unlimited
#

There is a programmatic way to do what the shell does, however to raise the per-user limit I believe you have to be root, which many people will not be. If there is a significant increase in file descriptor requirements, that will be a problem for many customers, especially server users. It sounds like it would affect client auth servers the most. We should find a way to get around this problem.

Comment 7

[Robert Relyea]

Aha! it was my shell.
I was trying to run ulimit in my tcsh!
 ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        10240
coredump(blocks)     unlimited
nofiles(descriptors) 256
vmemory(kbytes)      unlimited
solaris1(220) limit
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       10240 kbytes
coredumpsize    unlimited
vmemoryuse      unlimited
descriptors     256
solaris1(221) unlimit
solaris1(222) limit
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       unlimited
coredumpsize    unlimited
vmemoryuse      unlimited
descriptors     65536
solaris1(223) ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        unlimited
coredump(blocks)     unlimited
nofiles(descriptors) 65536
vmemory(kbytes)      unlimited
solaris1(224)

Comment 8

[Robert Relyea]

So it doesn't look like it's selfserv that's running into the limits, it's strsclnt. changing my shell ulimit caused ssl.sh to complete successfully.

Comment 9

[Nelson Bolyard (seldom reads bugmail)]

So, is this bug invalid, then?

Comment 10

[Robert Relyea]

Well the bug is that the tests fail on tinderbox, which means we need to either bump the limits on tinderbox or have the script bump the limits

Comment 11

[Robert Relyea]

Attachment: patch to make solaris x86 successfully run shared db tests.

This patch contains 2 fixes:

1) in all.sh 2&>1 was fixed to be 2>&1. The former parsed out as 2 & (launch in background) > 1 (redirect to file '1') instead of the intended latter (redirect FD2 into FD1) This would sometimes cause the key removal to run before key generation, leaving an extra key in the FIPS db which caused the FIPS test to fail later.

2) in ssl.sh increase the FD limit.

bob

Comment 12

[Nelson Bolyard (seldom reads bugmail)]

Comment on attachment 273987 [details] [diff] [review]
patch to make solaris x86 successfully run shared db tests.

good catch, Bob