169573 - certificate database gets corrupt after adding and deleting an object too many times

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P1)

Description

[Julien Pierre]

I'm opening a separate bug for this problem which was found as part of the
investigqtion of 167615.

Here the test case :
1. I started with an empty cert database.
2. I imported the 85 KB thawte CRL with crlutil -d . -I -i thawte2.crl -B
3. I deleted the CRL with crlutil -d . -E
4. went back to step 2, in a loop

At each iteration after the initial addition, the cert database grew by 4 KB.
Then, ultimately, it reached a peak size of 417,792 bytes. At that point, even
if I tried to import the CRL into it, it wouldn't show anymore with crlutil -L.

Therefore, there is a pre-existing corruption problem in DBM.

I repeated the test with the Netscape CRL, which is below 64 KB. The same
problem occurred, at a different peak size for cert7.db (229,376 bytes).

When I use a 470 or a 2524 bytes CRL, I don't get the problem. The cert7.db
stays at 16,384 bytes the entire time.

I don't have any CRL in between 2524 and 29313 bytes (Netscape CRL) so I
couldn't test any other case to find out what exact threshold is the problem.

Comment 1

[Wan-Teh Chang]

Assigned the bug to Bob.

Comment 2

[Julien Pierre]

Attachment: batch file to reproduce the problem

Comment 3

[Julien Pierre]

Attachment: CRL to reproduce the problem

Comment 4

[Julien Pierre]

FYI, I tried this with NSS 3.3 (I had to backport the -E option to it for the
test). The problem has not occurred yet in sevreal minutes, but the cert
database is constantly growing. It has already reached over 1.5 MB. However, the
CRL is being successfully deleted and added, and still being listed.

Comment 5

[Julien Pierre]

With NSS 3.3, the cert7.db peaked at 5,341,184 bytes. The script is still
running successfully. So part of the problem is a regression that was introduced
since that release.

Still, I don't think the database should grow that large for holding a single 85
KB object ...

Comment 6

[Julien Pierre]

Hmm, never mind, it kept growing, and it reached a peak of  10,584,064 .
And the same problem showed up too, with different error msgs :

crlutil: unable to import CRL: New CRL is not later than the current one.
crlutil: fail to look up CRLs (Unable to find the certificate or key necessary f
or authentication.)

Comment 7

[Robert Relyea]

Attachment: patch to surface errors in lookups up.

The basic idea of this patch is that error types are sent up using
PORT_SetError(). In our search routine, if we simply don't find what we are
looking for, we set the error NOT_FOUND. Upper level search codes, when no CRL
is found check for this error code and treats it as a no error condition. This
allows the CRL data to fail for other reasons.

This patch does fail cert validation if the C_FindInit* calls return an error,
or the object is found, but an error is found in subsequent processing.

Our standard regression tests pass with this code.

This patch deals with the Cert lookup and Cert Validations paths, so it is
deemed high risk. It should be tested against Julian's tests, and ran in a
couple of existing products before it goes into RTM code. I recommend moving
this bug to NSS 3.7.

Comment 8

[Robert Relyea]

Comment on attachment 101200 [details] [diff] [review]
patch to surface errors in lookups up.

This patch, and corresponding comment, was for bug 164501

Comment 9

[Robert Relyea]

Patch checked in.