232479 - NIST PKITS tests: more self-issued cert test failures

Status: RESOLVED DUPLICATE of bug 232737

(NSS :: Libraries, defect)

Description

[Bishakha Banerjee]

Besides the ones mentioned in bug 231030, the following tests with Self-issued
certs (4.5.1, 4.5.5 and 4.5.7) also fail.

The tests are:
  VFY_ACTION="Valid Basic Self-Issued Old With New Test1"
  certImport -n BasicSelfIssuedNewKeyCACert -i \
      $certs/BasicSelfIssuedNewKeyCACert.crt
  crlImport $crls/BasicSelfIssuedNewKeyCACRL.crl
  pkits $certs/ValidBasicSelfIssuedOldWithNewTest1EE.crt \
      $certs/BasicSelfIssuedNewKeyOldWithNewCACert.crt \
      $certs/BasicSelfIssuedNewKeyCACert.crt


  VFY_ACTION="Invalid Basic Self-Issued New With Old Test5"
  certImport -n BasicSelfIssuedOldKeyCACert -i \
      $certs/BasicSelfIssuedOldKeyCACert.crt
  crlImport $crls/BasicSelfIssuedOldKeyCACRL.crl
  pkitsn $certs/InvalidBasicSelfIssuedNewWithOldTest5EE.crt \
      $certs/BasicSelfIssuedOldKeyNewWithOldCACert.crt \
      $certs/BasicSelfIssuedOldKeyCACert.crt
  delete BasicSelfIssuedOldKeyCACert


  VFY_ACTION="Invalid Basic Self-Issued CRL Signing Key Test7"
  certImport -n BasicSelfIssuedCRLSigningKeyCACert -i \
      $certs/BasicSelfIssuedCRLSigningKeyCACert.crt
  crlImport $crls/BasicSelfIssuedOldKeyCACRL.crl
  pkitsn $certs/InvalidBasicSelfIssuedCRLSigningKeyTest7EE.crt \
      $certs/BasicSelfIssuedCRLSigningKeyCRLCert.crt \
      $certs/BasicSelfIssuedCRLSigningKeyCACert.crt


The output log states for the following tests:
./pkits.sh: Valid Basic Self-Issued Old With New Test1 --------------------------
vfychain -d PKITSdb -u 4
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/ValidBasicSelfIssuedOldWithNewTest1EE.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedNewKeyOldWithNewCACert.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedNewKeyCACert.crt
Chain is bad, -8159 = New CRL has an invalid format.
PROBLEM WITH THE CERT CHAIN:
CERT 0. CN=Valid Basic Self-Issued Old With New EE Certificate Test1,O=Test
Certificates,C=US :
  ERROR -8159: New CRL has an invalid format.
CERT 1. CN=Basic Self-Issued New Key CA,O=Test Certificates,C=US [Certificate
Authority]:
  ERROR -8159: New CRL has an invalid format.
./pkits.sh ERROR: Valid Basic Self-Issued Old With New Test1 failed 2
ERROR: Valid Basic Self-Issued Old With New Test1 failed 2

This test (4.5.1) should have validated successfully


./pkits.sh: Invalid Basic Self-Issued New With Old Test5 --------------------------
vfychain -d PKITSdb -u 4
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/InvalidBasicSelfIssuedNewWithOldTest5EE.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedOldKeyNewWithOldCACert.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedOldKeyCACert.crt
Chain is good!
./pkits.sh ERROR: Invalid Basic Self-Issued New With Old Test5 failed 0
ERROR: Invalid Basic Self-Issued New With Old Test5 failed 0

This test (4.5.5) should have not validated as the EE's cert has been revoked.


./pkits.sh: Invalid Basic Self-Issued CRL Signing Key Test7
--------------------------
vfychain -d PKITSdb -u 4
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/InvalidBasicSelfIssuedCRLSigningKeyTest7EE.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedCRLSigningKeyCRLCert.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedCRLSigningKeyCACert.crt
Chain is good!
./pkits.sh ERROR: Invalid Basic Self-Issued CRL Signing Key Test7 failed 0
ERROR: Invalid Basic Self-Issued CRL Signing Key Test7 failed 0

This test (4.5.7) should not have validated successfully as EE's cert is revoked.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

The above test script excerpt shows that test 4 involves a cert import and 
a crl import before the vfychain command, but the certutil command (that 
does the cert import) and the crlutil command (that does the CRL import)
are not shown in the test script output log excerpt.  I suspect that one
or more of those commands failed, but the output log does not show it.

The output log should show every command exactly as it is run, and the results
including any output written to stdout or stderr.  Any failure (program returns
a non-zero value) should be noted in the log also.

Those issues must be resolved before any further resolution of this bug is 
psossible.  

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

Reassigningto Bishakha to make the script changes described above.
If these test cases continue to fail after those issues are fixed,
and the reason is not obvious, please let me know.

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

*** Bug 232572 has been marked as a duplicate of this bug. ***

Comment 4

[Bishakha Banerjee]

Attachment: test script

okay, I see what you are getting at. I have made the modifications to the
script to 
a) every time the script runs certutil, crlutil, or vfychain, it echos
   the *exact* command being run into the output log,
b) If any of those programs fail (return non-zero), that is reported in
   the log, and 
c) any and all output of those programs is included in the log.

about cert import and crl import that are not getting written to the output
log. They are, just in a previous test. (I mentioned before, some adjacent
tests use the same certs and CRLs, in which case I do not delete and re-import
them). When I cut and pasted from my script, I cut and pasted the CRL and cert
import commands from the previous test, but did not cut and paste the output in
the log from the previous tests, I'm sorry about that.
Anyway, now I am attaching a sample test script that tests only these three
tests in the 4.5 section of NIST PKITS test suite.

./test.sh: certutil -d PKITSdb -A -t ",," -n BasicSelfIssuedNewKeyCACert -i
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedNewKeyCACert.crt

./test.sh: crlutil -d PKITSdb -I -i
/share/builds/sbstools/nsstools/tmp/PKITS_data/crls/BasicSelfIssuedNewKeyCACRL.crl

Elapsed :  0: 0.  2
./test.sh: Valid Basic Self-Issued Old With New Test1
--------------------------
vfychain -d PKITSdb -u 4
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/ValidBasicSelfIssuedOldWithNewTest1EE.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedNewKeyOldWithNewCACert.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedNewKeyCACert.crt

Chain is bad, -8159 = New CRL has an invalid format.
PROBLEM WITH THE CERT CHAIN:
CERT 0. CN=Valid Basic Self-Issued Old With New EE Certificate Test1,O=Test
Certificates,C=US :
  ERROR -8159: New CRL has an invalid format.
CERT 1. CN=Basic Self-Issued New Key CA,O=Test Certificates,C=US [Certificate
Authority]:
  ERROR -8159: New CRL has an invalid format.
./test.sh ERROR: Valid Basic Self-Issued Old With New Test1 failed 2
./test.sh: crlutil -d PKITSdb -D -n BasicSelfIssuedNewKeyCACert
./test.sh: certutil -d PKITSdb -D -n BasicSelfIssuedNewKeyCACert
./test.sh: certutil -d PKITSdb -A -t ",," -n BasicSelfIssuedOldKeyCACert -i
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedOldKeyCACert.crt

./test.sh: crlutil -d PKITSdb -I -i
/share/builds/sbstools/nsstools/tmp/PKITS_data/crls/BasicSelfIssuedOldKeyCACRL.crl

crlutil: unable to import CRL: The CRL for the certificate's issuer has an
invalid signature.
Elapsed :  0: 0.  2
./test.sh: Invalid Basic Self-Issued New With Old Test5
--------------------------
vfychain -d PKITSdb -u 4
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/InvalidBasicSelfIssuedNewWithOldTest5EE.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedOldKeyNewWithOldCACert.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedOldKeyCACert.crt

Chain is good!
./test.sh ERROR: Invalid Basic Self-Issued New With Old Test5 failed 0
./test.sh: crlutil -d PKITSdb -D -n BasicSelfIssuedOldKeyCACert
crlutil: could not find BasicSelfIssuedOldKeyCACert's CRL: No matching CRL was
found.
crlutil: could not find the issuer BasicSelfIssuedOldKeyCACert's CRL: No
matching CRL was found.
./test.sh: certutil -d PKITSdb -D -n BasicSelfIssuedOldKeyCACert
./test.sh: certutil -d PKITSdb -A -t ",," -n BasicSelfIssuedCRLSigningKeyCACert
-i
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedCRLSigningKeyCACert.crt

./test.sh: crlutil -d PKITSdb -I -i
/share/builds/sbstools/nsstools/tmp/PKITS_data/crls/BasicSelfIssuedOldKeyCACRL.crl

crlutil: unable to import CRL: Peer's Certificate issuer is not recognized.
Elapsed :  0: 0.  0
./test.sh: Invalid Basic Self-Issued CRL Signing Key Test7
--------------------------
vfychain -d PKITSdb -u 4
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/InvalidBasicSelfIssuedCRLSigningKeyTest7EE.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedCRLSigningKeyCRLCert.crt
/share/builds/sbstools/nsstools/tmp/PKITS_data/certs/BasicSelfIssuedCRLSigningKeyCACert.crt

Chain is good!
./test.sh ERROR: Invalid Basic Self-Issued CRL Signing Key Test7 failed 0
./test.sh: crlutil -d PKITSdb -D -n BasicSelfIssuedCRLSigningKeyCACert
crlutil: could not find BasicSelfIssuedCRLSigningKeyCACert's CRL: No matching
CRL was found.
crlutil: could not find the issuer BasicSelfIssuedCRLSigningKeyCACert's CRL: No
matching CRL was found.
./test.sh: certutil -d PKITSdb -D -n BasicSelfIssuedCRLSigningKeyCACert

So, you can see that test 4.5.5 and 4.5.7 do not load the CRLs correctly (they
are invalid CRLs), consequently vfychain reports that the chain is good. Is
this the right response from vfychain?

For our purposes, we can of course look at our log and see why the test results
are incorrect (we expect an invalid chain to be reported, this is a negative
test case. However, vfychain says the chain is valid, because the CRL did not
get loaded and we can easily check that from our output log.)

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

I opened a new bug that describes this problem a little better.

*** This bug has been marked as a duplicate of 232737 ***