426245 - Assertion failure went undetected by tinderbox

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P1)

Description

[Alexei Volkov]

Killing of selfserv is producing a leak that went undetected since March 26 after patch to bug 397832 enabled assertion check on the number of leaked objects. 

Objects leaked on all platforms.
 
ssl.sh: #845: TLS Require client auth (client auth)(cert TestUser51 - not revoked) produced a returncode of 0, expected is 0 - PASSED
Server Args: -r_-r
tstclnt -p 8443 -h dositups.red.iplanet.com -f -d ../client \
        -w nss -n TestUser52   < /export/tinderbox/SunOS_5.10/mozilla/security/nss/tests/ssl/sslreq.dat
4[817d180]: Error at level 0: Unable to build chain
4[817d180]: Error at level 1: pkix_Build_InitiateBuildChain failed
4[817d180]: Error at level 2: pkix_BuildForwardDepthFirstSearch failed
4[817d180]: Error at level 3: pkix_DefaultCRLChecker_Check_Store failed
4[817d180]: Error at level 4: pkix_DefaultCRLChecker_Check_Store failed
4[817d180]: Error at level 5: Certificate is revoked by CRL for key compromise
selfserv: HDX PR_Read returned error -8180:
Peer's Certificate has been revoked.
tstclnt: write to SSL socket failed: SSL peer rejected your certificate as revoked.
kill -0 2995 >/dev/null 2>/dev/null
selfserv with PID 2995 found at Mon Mar 31 10:03:29 PST 2008
ssl.sh: #846: TLS Require client auth (client auth)(cert TestUser52 - revoked) produced a returncode of 254, expected is 254 - PASSED
trying to kill selfserv with PID 2995 at Mon Mar 31 10:03:29 PST 2008
kill -USR1 2995
1[8087790]: Class BigInt leaked 8 objects of size 8 bytes, total = 64 bytes
1[8087790]: Class Cert leaked 8 objects of size 136 bytes, total = 1088 bytes
1[8087790]: Class List leaked 40 objects of size 20 bytes, total = 800 bytes
1[8087790]: Class PublicKey leaked 8 objects of size 4 bytes, total = 32 bytes
1[8087790]: Class X500Name leaked 16 objects of size 24 bytes, total = 384 bytes
1[8087790]: Assertion failure: numLeakedObjects == 0, at pkix_pl_lifecycle.c:294
Assertion failure: numLeakedObjects == 0, at pkix_pl_lifecycle.c:294
selfserv: 0 cache hits; 41 cache misses, 0 cache not reusable
          0 stateless resumes, 0 ticket parse failures
selfserv with PID 2995 killed at Mon Mar 31 10:03:29 PST 2008
ssl.sh: TLS Require client auth (client auth) ----
selfserv starting at Mon Mar 31 10:03:29 PST 2008
selfserv -D -p 8443 -d ../server -n dositups.red.iplanet.com  \

Here is what happened on the system, where core was detected:

ssl.sh: #2890: TLS Require client auth (client auth)(cert TestUser52 - revoked) produced a returncode of 254, expected is 254 - PASSED
trying to kill selfserv with PID 412 at Thu Mar 27 05:48:59 PDT 2008
kill -USR1 412
Assertion failure: numLeakedObjects == 0, at pkix_pl_lifecycle.c:294
selfserv: 0 cache hits; 41 cache misses, 0 cache not reusable
          0 stateless resumes, 0 ticket parse failures
selfserv with PID 412 killed at Thu Mar 27 05:48:59 PDT 2008
ssl.sh: #2891: kill_selfserv core detection step - Core file is detected.
ssl.sh: TLS Require client auth (client auth) ----

It looks like all systems(except one) have ulimit for code set to 0. Slavo, please check out our test system and set variable to "unlimited" value.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Slavo, please treat this as your TOP priority bug, immediately.

Comment 2

[Slavomir Katuscak]

Attachment: Patch.

Added setting ulimit -c unlimited if ulimit exists and executable.
Added FAILED string when core file detected.

Comment 3

[Slavomir Katuscak]

For now I set ulimit to unlimited on all Solaris and Linux trunk Tinderboxes, but I prefer to have this setting in script.

Comment 4

[Slavomir Katuscak]

Attachment: Patch v2.

Ulimit detection didn't work on Linux correctly (ulimit is part of shell, which command doesn't detect it). I rather skip ulimit only for Windows platform. Then there is no limitation for OS.

Comment 5

[Alexei Volkov]

Comment on attachment 312922 [details] [diff] [review]
Patch v2.

r+ provided the fix is tested on all unix platforms.

Comment 6

[Slavomir Katuscak]

Checking in init.sh;
/cvsroot/mozilla/security/nss/tests/common/init.sh,v  <--  init.sh
new revision: 1.63; previous revision: 1.62
done