464907 - Vfychain exited with return value 0 even if test failed.

Status: REOPENED

(NSS :: Test, defect, P2)

Description

[Slavomir Katuscak]

Occured on: jesma51 MINGW32_NT-5.2 64bit DBG on 2008/11/13 18:10:47 (Tinderbox)

Vfychain test with inhibit any policy extension with level 5 was expected to fail (negative test). Even if output looks like vfychain test really failed, exit value was most probably 0.

Log:
---
vfychain -d AllDB -pp -vv   -o OID.1.0  EE67CA67.der RootCA.der CA61RootCA.der CA62CA61.der CA63CA62.der CA64CA63.der CA65CA64.der CA66CA65.der CA67CA66.der  -t RootCA.der
Chain is bad, -8172 = Peer's certificate issuer has been marked as not trusted by the user.
PROBLEM WITH THE CERT CHAIN:
CERT 9. CN=RootCA ROOT CA,O=RootCA,C=US [Certificate Authority]:
  ERROR -8172: Peer's certificate issuer has been marked as not trusted by the user.
    CN=RootCA ROOT CA,O=RootCA,C=US
chains.sh: #2636: AnyPolicyWithLevel: Verifying certificate(s)  EE67CA67.der RootCA.der CA61RootCA.der CA62CA61.der CA63CA62.der CA64CA63.der CA65CA64.der CA66CA65.der CA67CA66.der with flags -d AllDB   -o OID.1.0  -t RootCA.der - FAILED
---

Status PASSED/FAILED is reported by functions, html_passed()/html_failed().

Algorithm in chains.sh to choose which to call is:
if [ "${EXP_RESULT}" = "pass" -a ${RESULT} -eq 0 ]; then
    html_passed "${SCENARIO}${TESTNAME}"
elif [ "${EXP_RESULT}" = "fail" -a ${RESULT} -ne 0 ]; then
    html_passed "${SCENARIO}${TESTNAME}"
else
    html_failed "${SCENARIO}${TESTNAME}"
fi

EXP_RESULT was set to fail for this test, so it seems that exit value was 0, otherwise it will pass [ "${EXP_RESULT}" = "fail" -a ${RESULT} -ne 0 ] check. (It was ran 4 times in 4 cycles, other cycles had correct results, only once it failed).

Comment 1

[Slavomir Katuscak]

Attachment: Patch to log return values. (checked in)

Patch to log return value + expected value. Can help with troubleshooting next time when this problem occurs.

Comment 2

[Slavomir Katuscak]

Checking in chains.sh;
/cvsroot/mozilla/security/nss/tests/chains/chains.sh,v  <--  chains.sh
new revision: 1.8; previous revision: 1.7
done

Comment 4

[Slavomir Katuscak]

Occured once again:

chains.sh: Verifying certificate(s)  EE67CA67.der RootCA.der CA61RootCA.der CA62CA61.der CA63CA62.der CA64CA63.der CA65CA64.der CA66CA65.der CA67CA66.der with flags  -d AllDB    -o OID.1.0  -t RootCA.der
vfychain -d AllDB -pp -vv     -o OID.1.0  EE67CA67.der RootCA.der CA61RootCA.der CA62CA61.der CA63CA62.der CA64CA63.der CA65CA64.der CA66CA65.der CA67CA66.der  -t RootCA.der
Chain is bad, -8032 = Cert chain fails policy validation
PROBLEM WITH THE CERT CHAIN:
CERT 8. CN=RootCA ROOT CA,O=RootCA,C=US [Certificate Authority]:
  ERROR -8032: Cert chain fails policy validation
Returned value is 0, expected result is fail
chains.sh: #7484: AnyPolicyWithLevel: Verifying certificate(s)  EE67CA67.der RootCA.der CA61RootCA.der CA62CA61.der CA63CA62.der CA64CA63.der CA65CA64.der CA66CA65.der CA67CA66.der with flags  -d AllDB    -o OID.1.0  -t RootCA.der - FAILED

From log is clear that vfychain returned value 0.

Comment 5

[Slavomir Katuscak]

This problem is seen on nss2000s machine much more often then on other machines.

Comment 6

[Slavomir Katuscak]

Machine: goride MINGW32_NT-5.2 32bit DBG on 2009/09/14 00:46:18

vfychain -d AllDB -pp -vv     -o OID.2.0  EE1CA13.der RootCA.der CA1RootCA.der CA12CA1.der CA13CA12.der  -t RootCA.der
Chain is bad, -8032 = Cert chain fails policy validation
PROBLEM WITH THE CERT CHAIN:
CERT 4. CN=RootCA ROOT CA,O=RootCA,C=US [Certificate Authority]:
  ERROR -8032: Cert chain fails policy validation
Returned value is 0, expected result is fail
chains.sh: #7549: AnyPolicyWithLevel: Verifying certificate(s)  EE1CA13.der RootCA.der CA1RootCA.der CA12CA1.der CA13CA12.der with flags  -d AllDB    -o OID.2.0  -t RootCA.der - FAILED

Comment 8

[Slavomir Katuscak]

This problem seems to be Windows specific, I haven't seen it on any other OS yet.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee is inactive on Bugzilla, and this bug has priority 'P2'.
:beurdouche, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 10

[Marco Castelluccio [:marco]]

We have modified the bot to only consider P1 as high priority, so I'm cancelling the needinfo here.