Closed Bug 123091 Opened 23 years ago Closed 23 years ago

SSL server doing client auth in second handshake leaks memory in NSS 3.4

Categories

(NSS :: Libraries, defect, P1)

Product:
NSS
Component:
Libraries
Platform:
Sun
Solaris
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: julien.pierre, Assigned: wtc)

References

Details

This is occurring with web server 6 and NSS 3.4. It does not occur with web server 6 and NSS 33.2 . The test is with an SSL client that presents a client cert signed by a CA which is trusted by the server. The server has SSL but with client auth allowed rather than expected. There is no ACL on the server so the client cert is not further checked, only by NSS against the trusted CA certs. dbx checkleaks shows this : (dbx) showleaks Checking for memory leaks... Actual leaks report (actual leaks: 0 total size: 0 bytes) Possible leaks report (possible leaks: 276 total size: 543780 bytes) Total Num of Leaked Allocation call stack Size Blocks Block Address ====== ====== ========== ======================================= 216720 105 - PR_Malloc < PL_ArenaAllocate < PORT_ArenaAlloc < EncodeDBCertKey < ReadDBCertEntry < FindCertByKey < nsslowcert_FindCertByKey < nsslowcert_TraversePermCertsForSubject 165120 80 - PR_Malloc < PL_ArenaAllocate < PORT_ArenaAlloc < PORT_ArenaZAlloc < SEC_ASN1DecoderStart < SEC_ASN1Decode < SEC_ASN1DecodeItem < nsslowcert_DecodeDERCertificate 117648 57 - PR_Malloc < PL_ArenaAllocate < PORT_ArenaAlloc < PORT_ArenaZAlloc < SEC_ASN1DecoderStart < SEC_ASN1Decode < SEC_ASN1DecodeItem < nsspkcs5_AlgidToParam 24768 12 - PR_Malloc < PL_ArenaAllocate < PORT_ArenaAlloc < ReadDBCertEntry < FindCertByKey < nsslowcert_FindCertByKey < nsslowcert_TraversePermCertsForSubject < pk11_searchCertsAndTrust 14448 7 - PR_Malloc < PL_ArenaAllocate < PORT_ArenaAlloc < EncodeDBSubjectKey < ReadDBSubjectEntry < nsslowcert_NumPermCertsForSubject < pk11_searchCertsAndTrust < pk11_searchTokenList 4128 2 - PR_Malloc < PL_ArenaAllocate < PORT_ArenaAlloc < PORT_ArenaZAlloc < nsspkcs5_NewParam < nsspkcs5_AlgidToParam < seckey_decrypt_private_key < seckey_decode_encrypted_private_key 88 1 0x43e0c0 calloc < PR_Calloc < PR_NewLock < nssList_Create < nssList_Clone < nssList_CreateIterator < nssPKIObject_Initialize < __CERT_NewTempCertificate 88 1 0x45de98 calloc < PR_Calloc < PR_NewLock < nssPKIObject_Initialize < __CERT_NewTempCertificate < ssl3_HandleCertificate < ssl3_HandleHandshakeMessage < ssl3_HandleHandshake 88 1 0x4a7fd0 calloc < PR_Calloc < PR_NewLock < PORT_NewArena < SECKEY_CopyPrivateKey < ssl_DupSocket < SSL_ImportFD < SSLSocketConfiguration::enable 88 1 0x595fe8 calloc < PR_Calloc < PR_NewLock < nssList_CreateIterator < nssPKIObject_Initialize < __CERT_NewTempCertificate < ssl3_HandleCertificate < ssl3_HandleHandshakeMessage 88 1 0x487600 calloc < PR_Calloc < PR_NewLock < nssList_Create < nssPKIObject_Initialize < __CERT_NewTempCertificate < ssl3_HandleCertificate < ssl3_HandleHandshakeMessage 88 1 0x466a60 calloc < PR_Calloc < PR_NewLock < nssArena_Create < NSSArena_Create < __CERT_NewTempCertificate < ssl3_HandleCertificate < ssl3_HandleHandshakeMessage 88 1 0x4c4698 calloc < PR_Calloc < PR_NewLock < PORT_NewArena < CERT_DecodeDERCertificate < nssDecodedPKIXCertificate_Create < STAN_GetCERTCertificate < __CERT_NewTempCertificate 88 1 0x54adf8 calloc < PR_Calloc < PR_NewLock < PORT_NewArena < CERT_DupCertList < ssl_DupSocket < SSL_ImportFD < SSLSocketConfiguration::enable 60 1 0x3eaee8 calloc < PR_Calloc < nss_ZAlloc < nssArena_Create < NSSArena_Create < __CERT_NewTempCertificate < ssl3_HandleCertificate < ssl3_HandleHandshakeMessage 52 1 0x575bf0 calloc < PR_Calloc < nss_ZAlloc < nssDecodedPKIXCertificate_Create < STAN_GetCERTCertificate < __CERT_NewTempCertificate < ssl3_HandleCertificate < ssl3_HandleHandshakeMessage 44 1 0x57ed80 calloc < PR_Calloc < PORT_ZAlloc < PORT_NewArena < CERT_DupCertList < ssl_DupSocket < SSL_ImportFD < SSLSocketConfiguration::enable 44 1 0x59cb58 calloc < PR_Calloc < PORT_ZAlloc < PORT_NewArena < CERT_DecodeDERCertificate < nssDecodedPKIXCertificate_Create < STAN_GetCERTCertificate < __CERT_NewTempCertificate 44 1 0x5b1298 calloc < PR_Calloc < PORT_ZAlloc < PORT_NewArena < SECKEY_CopyPrivateKey < ssl_DupSocket < SSL_ImportFD < SSLSocketConfiguration::enable
Priority: -- → P1
Summary: Memory leak in server-side SSL application with client auth in NSS 3.4 → Memory leak in server-side SSL application with client cert in NSS 3.4
Blocks: 123094
Just FYI, when I ran this test, I configured the web server with a single worker thread, in order to get the most accurate results possible from the checkleaks output. However, the web server still has other non-worker background threads running, which occasionally allocate memory. If a block was allocated by these threads between the start and end of my run, it will show up erroneously as a leak. I know for sure that's the case when I run, say, a couple thousand requests and see a 528 bytes total possible "leak", so I count that as not leaking. This is another reason why I try to have a high request count before taking the output of checkleaks.
I am not quite sure I'm testing this correctly . I'm using test aclS04d that I added to the web server GAT. This leak does not appear to be reproducible in the server with the current tip of NSS 3.4 . Before I go ahead and close this as resolved (it may well have been resolved, given all the fixes done by Ian and Nelson), I'm wondering what's supposed to happen in this case, namely : client is configured with a client cert server is configured to allow client auth, but not requiring it Is that supposed to cause the cert to be presented over SSL or not ? This was the case where I was seeing a leak in the server last week. I was not - and still am not - seeing a leak in the server when neither the client or the server have client auth enabled. I know, I could check the code, or ssltap output, but it's rather late and I'm going home.
Target Milestone: --- → 3.4
I have found the answer I was looking for by looking at the code - there is no cert exchange going on at all if client auth is not required on the SSL socket and there is no ACL, even if the client has a cert available. The client only presents the cert if requested by the server, through an NSS callback function. Therefore, the initial description of this bug must have been incorrect. Either I had an ACL on the web server for client auth, or client auth was enforced on the SSL socket (required). The later would be a dupe of 123479. So I'm going to use this bug for a leak on the second handshake in the server, and will rerun tests to confirm that.
Summary: Memory leak in server-side SSL application with client cert in NSS 3.4 → SSL server doing client auth in second handshake leaks memory in NSS 3.4
I ran another test over the week-end, with NES 6 and double-handshake client auth. Neither the client or server leaked memory. I got 1648310 sessions on the client side, and 94711 errors on the server side. The errors were [22/Feb/2002:15:53:11] failure (25911): for host 10.169.25.24 trying to GET /, C lient-Auth reports: Timeout while waiting for client certificate. I believe this may be a bug in the web server "get-client-cert" function, which doesn't give the client enough time. I'm investigating that with the web server tea. I think the original bug in this defect, which is a memory leak, is fixed, so I'm closing this bug.
Status: NEW → RESOLVED
Closed: 23 years ago
OS: SunOS → Solaris
Resolution: --- → FIXED
Marking verified.
You need to log in before you can comment on or make changes to this bug.