265675 - checkcert has an assertion failure with the NISCC SMIME test certificates

Status: NEW

(NSS :: Tools, defect, P3)

Description

[Jason Reid]

While attempting to examine the certificates in the NISCC SMIME suites,
I reproducibly received an assertion failure with checkcert. 
Certutil loads both Client.crt and CA.crt with no difficulties.

mahatma[svbld]:/niscc/NISCC_SMIME_testcases> setenv NSSPATH
/share/builds/mccrel3/security/security39/builds/20041013.1/wozzeck_Solaris8/mozilla/dist/SunOS5.10_i86pc_DBG.OBJ/bin/
mahatma[svbld]:/niscc/NISCC_SMIME_testcases> setenv LD_LIBRARY_PATH $NSSPATH/../lib
mahatma[svbld]:/niscc/NISCC_SMIME_testcases> $NSSPATH/checkcert -aAv Client.crt
CA.crt 
Decoded ok as X509 SIGNED data.
Decoded ok as an X509 certificate.
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
Assertion failure: oidhash != NULL, at secoid.c:1649
Abort (core dumped)
mahatma[svbld]:/niscc/NISCC_SMIME_testcases> file core
core:           ELF 32-bit LSB core file 80386 Version 1, from 'checkcert'
mahatma[svbld]:/niscc/NISCC_SMIME_testcases>
/usr/dist/share/forte_dev_i386,v6.2/SUNWspro/bin/dbx $NSSPATH/checkcert core
Reading checkcert
core file header read successfully
Reading ld.so.1
Reading libssl3.so
Reading libsmime3.so
Reading libnss3.so
Reading libplc4.so
Reading libplds4.so
Reading libnspr4.so
Reading libthread.so.1
Reading libnsl.so.1
Reading libsocket.so.1
Reading librt.so.1
Reading libdl.so.1
Reading libc.so.1
Reading libsoftokn3.so
Reading libpthread.so.1
Reading libaio.so.1
Reading libmd5.so.1
Reading libmp.so.2
detected a multithreaded program
program terminated by signal ABRT (Abort)
0xd0ff2e05: _lwp_kill+0x0015:   jae    _lwp_kill+0x23 <0xd0ff2e13>
Current function is PR_Assert
  530       abort();
(/usr/dist/share/forte_dev_i386,v6.2/SUNWspro/bin/dbx) where                 
  [1] _lwp_kill(0x1, 0x6), at 0xd0ff2e05
  [2] _thr_kill(0x1, 0x6), at 0xd0fefe1d
  [3] raise(0x6), at 0xd0f9ccaf
  [4] abort(), at 0xd0f7f1cd
=>[5] PR_Assert(s = 0xd12d2c18 "oidhash != NULL", file = 0xd12d2c28 "secoid.c",
ln = 1649), line 530 in "prlog.c"
  [6] SECOID_FindOID(oid = 0x808e638), line 1649 in "secoid.c"
  [7] SECU_PrintObjectID(out = 0xd101d100, oid = 0x808e638, m = 0x807f800
"Signature Algorithm", level = 2), line 1239 in "secutil.c"
  [8] SECU_PrintAlgorithmID(out = 0xd101d100, a = 0x808e638, m = 0x807f800
"Signature Algorithm", level = 2), line 1265 in "secutil.c"
  [9] SECU_PrintCertificate(out = 0xd101d100, der = 0x808d4a8, m = 0x807fab4
"Data", level = 1), line 1882 in "secutil.c"
  [10] SECU_PrintSignedData(out = 0xd101d100, der = 0x804774c, m = 0x807f56c
"Certificate", level = 0, inner = 0x8055af8 = &SECU_PrintCertificate(FILE *out,
struct SECItemStr *der, char *m, int level)), line 2543 in "secutil.c"
  [11] main(argc = 4, argv = 0x80477a4), line 473 in "checkcert.c"
(/usr/dist/share/forte_dev_i386,v6.2/SUNWspro/bin/dbx) 

Here is a brief analysis Julien Pierre did for me when I asked about this
crash:
> From what I see in the core file I just got when I tried, the crash has 
>to do with the dynamic OID table. I know there are serious bugs in this 
>area, but cannot remember what they are. You may want to search bugzilla 
>for related bugs before filing a new one.

Comment 1

[Wan-Teh Chang]

Jason, checkcert is not being maintained.  I found
that it is not calling any NSS initialization function
(try searching for "Init" in checkcert.c).  This is
most likely why the assertion "oidhash != NULL" failed.

Comment 2

[Julien Pierre]

Wan-Teh,

Thanks. I wasn't aware that checkcert wasn't maintained. Given that this is the
case, should we remove it from our tree ?

Comment 3

[Wan-Teh Chang]

It may not be that much work to bring
checkcert back to working order.  Adding
a NSS initialization call may be all it
takes.

Comment 4

[Julien Pierre]

Wan-Teh,

I tried just adding an NSS_NoDB_Init at the beginning .

The program asserted later on :
PORT_Assert(rsapubkey->keyType == rsaKey); /* XXX RSA */

It doesn't seem that the keyType ever gets set.

I also see that it's using SEC_ASN1DecodeItem rather than SEC_QuickDERDecodeItem.

If I remove the assertion, the program runs to completion. But that doesn't mean
everything is correct . There may be be more work to fully revive it . Is it
worth it ?

Comment 5

[Julien Pierre]

Attachment: my changes to checkcert

Comment 6

[Robert Relyea]

Why is this function hand decoding the key? What is wrong with "Extract key"?

The reason key type isn't being set is because it's set by that standard NSS
calls which create keys, none of which are called to create this one.

bob