217392 - softoken should allow storing more than one CRL per issuer

Status: NEW

(NSS :: Libraries, enhancement, P5)

Description

[Julien Pierre]

Currently, we only allow storing a single CRL for each issuer subject.

When a new CRL is imported, we check if it has a newer lastUpdate value, and if
it does, we replace the old CRL with the new one.

This works well for server applications, which always verify certificates as of
the login date. Such certificates by definition must not be expired before their
revocation status is checked. Since the CRL contains a list of all unexpired
revoked certificates, all is well.

Where this breaks down is for client applications, which verify dates in the
past, such as S/MIME e-mail clients.

If I go back to reading a 1999 signed e-mail message, and use the current CRL in
2003, I will in effect have no revocation checking performed, because the
current CRL will not contain the serial number of the certificate used in that
message, even though that certificate was indeed revoked.

To verify the message with some degree of revocation accuracy would require
using a CRL from 1999, which has a chance of listing the old signing certificate .

Because we always replace old CRLs with new CRLs however, there is no good way
to use the old CRL. I would have to keep the DER of the CRL on the disk myself,
delete the current CRL from my NSS database, and then install the 1999 CRL, in
order to do revocation. Clearly this is impractical.

In order to make this type of revocation possible, we must be able to store more
than one CRL for a given issuer, and the application (and cert verification
code, crl cache) must be able to fetch more than one stored CRL.

We also need to redefine our APIs and perhaps the PKCS#11 extension itself to be
able to return more than one CRL object.

This brings the additional question of how many CRLs we should keep in the token.

Several possibilities :
1. an infinite number : wekeep all CRLs, unless explicitly deleted. Let the
application decide.

If we choose this way, and users update their CRL frequently (eg. using
mozilla's auto-update feature, set to daily) then the cert9 database will grow
to be very large very quickly.

Also, CRL processing will become slower over time, since all the CRLs for the
issuer will need to be read from disk and decoded to build the CRL cache

Not a good option at all.

2. keep all CRLs that have new or modified entries

Eg. we would not add a CRL to the DB if all the entries in that new CRL were
duplicate of entries in other CRLs we already had.

We would also eliminate all CRLs that dropped entries, but did not add any new
entries . That's because a cert can never be made valid again after being
revoked ("un-revoked"). The only thing that may change is the revocation
date/reason code, and that happens through a new entry in a later CRL.

This would only be slightly better than option 1. If a user updates his CRL
daily, each CRL might have a couple new entries each day, so technically they
would all be added.

3. use a more complicated algorithm to eliminate unnecessary CRLs

With this method, we wouldn't only decide whether or not to store a CRL being
imported, we would also optimize the existing CRLs stored, and delete them
selectively. This optimization algorithm would have to be run each time we add a
CRL to the database.

It might work this way :
- sort CRLs from oldest to newest based on lastupdate, and process them in that
order
- for each CRL, keep a counter, initially set to the number of entries in that CRL
- for each CRL, inser tthe entries to a hash table indexed by serial number (SN)
of all the CRL entries
- for each entry in the hash table, keep an origin pointer to the full CRL it
comes from
- for duplicate SNs, replace the entry in the hash table, and decrement the
counter of the CRL that contained the entry that was removed from the hash table
- keep only CRLs whose counter is greater than 0 and delete all others

The execution of this algorithm would be facilitated by the CRL cache which uses
a hash table of CRL entries already, although I don't believe it currently keeps
track of the origin CRL for each entry.

Comment 1

[Julien Pierre]

This should be dealt with in order to add support for other types of CRLs (Distribution Points). Otherwise, the only way for these CRLs to be obtained will be over the network, but they will never be stored locally. This is generally not the way to go for CRLs.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

This is unlikely to be resolved for 3.12, IMO.

Comment 3

[Julien Pierre]

Bob, does the new DB for 3.12 allow provision for this in the future ?

Comment 4

[Nelson Bolyard (seldom reads bugmail)]

Julien, Please take this bug.  I think now is a good time to start work on it.

Comment 5

[Julien Pierre]

Taking bug. But I'm not the DB expert. I will need some assistance from Bob to make progress.

Comment 6

[Robert Relyea]

For the old database (dbm), it will require a database format change (Subject is the database key that a CRL can be stored under and in dbm database keys are unique).

For the new database (sql), it should simply be a matter of turning off the code that prevents more than once CRL to be stored under a single subject.

bob