174643 - CERT_CreateCertificate truncates serial number to 32 bits

Status: NEW

(NSS :: Libraries, defect, P3)

Description

[Nelson Bolyard (seldom reads bugmail)]

This is a bug on all platforms where 
    sizeof(unsigned long) > sizeof(uint32)

CERT_CreateCertificate takes a unsigned long serialNumber argument.
It passes that argument to DER_SetUInteger, which takes a uin32 argument.
On 64-bit platforms, or platforms with longs bigger than 32 bits,
this causes all but the least signficant 32-bits of serialNumber to be 
discarded.  

Possible solutions include:

a) document (in the .c and .h file) that CERT_CreateCertificate only uses 
the 32 least significant bits of serialNumber, and don't try to fix it.  
Document certutil's limitation also.  or 

b) change DER_SetUInteger to take an unsigned long, and properly DER 
encode it for platforms, regardless of the size of unsigned long.
Do the same thing for DER_SetInteger (using long instead of in32).
This might break binary compatibility (but I doubt it would).  or

c) write two new functions, DER_SetULong and DER_SetLong that take 
longs and encoded them properly regardless of their size, or 

d) write two new functions, DER_SetUint64 and DER_SetInt64 that take
PRUint64 and PRInt64 arguments and encode them properly.  Then change
certutil to use PRUint64 for serial numbers.

To fix a separate bug, I am changing certutil to use ulongs for serial
numbers, to match the type expected by CERT_CreateCertificate.  If option
d is chosen, that change would be superseded by a change to use PRUint64.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

The workaround for this is pretty trivial.

Call CERT_CreateCertificate with some bogus serial number, e.g. 1.
Then when it succeeds, replace the serial number in the CertCertificateStr.

Something like 

    SECItem mySerial;

    mySerial.type = siUnsigned
    mySerial.data = <your serial number as a byte string, not DER encoded,
                     with leading zero byte if needed to make it positive>;
    mySerial.len  = <number of bytes in above string>;

    cert = CERT_CreateCertificate(1, ...);
    if (cert) {
        SECStatus rv;
        rv = SECITEM_CopyItem(cert->arena, &cert->serialNumber, &mySerial);

This will NOT leak the previous serial number.