Closed Bug 310491 Opened 19 years ago Closed 19 years ago

JSS tests don't report failures

Categories

(JSS Graveyard :: Tests, defect)

Product:
JSS Graveyard
Version:
4.1.1
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: christophe.ravel.bugs, Assigned: Sandeep.Konchady)

Details

Attachments

(2 files, 3 obsolete files)

JSS tests with deprecation removed but without KeyWrapping and SymKeyGen tests
25.57 KB, patch
glenbeasley
: superreview+
Details | Diff | Splinter Review
Missed some System.exit(0) and reverted SigTest.java to original format
4.51 KB, patch
glenbeasley
: superreview+
Details | Diff | Splinter Review 
Some tests can fail but the result is still pass.
Example:

============= Generate known cert pair for testing
***FilePasswordCallback returns netscape
Exiting main()
============= convert PKCS11 cert to PKCS12 format
pk12util: PKCS12 EXPORT SUCCESSFUL
============= Start JSSE server tests
============= Start JSS client tests
***FilePasswordCallback returns netscape
Unable to connect: (-5961) TCP connection reset by peer.
org.mozilla.jss.ssl.SSLSocketException: Unable to connect: (-5961) TCP
connection reset by peer.
        at org.mozilla.jss.ssl.SSLSocket.socketConnect(Native Method)
        at org.mozilla.jss.ssl.SSLSocket.<init>(SSLSocket.java:258)
        at org.mozilla.jss.ssl.SSLSocket.<init>(SSLSocket.java:229)
        at org.mozilla.jss.tests.JSS_SSLClient.doIt(JSS_SSLClient.java:212)
        at org.mozilla.jss.tests.JSS_SSLClient.main(JSS_SSLClient.java:419)
JSSTEST_CASE 9 (JSSE server / JSS client): PASS
Assignee: wtchang → glen.beasley
Assignee: glen.beasley → Sandeep.Konchady
Status: NEW → ASSIGNED
Target Milestone: --- → 4.1.1
Added System.exit(1) to JSSE_SSLClient, JSS_SSLClient and JSS_SSLServer to
report failure.  Also added logic to check if the number of pass is equal to
number of tests.  If these are not equal, then report "FAILURE" else report
"SUCCESS".
Attachment #198273 - Flags: superreview?(glen.beasley)
Comment on attachment 198273 [details] [diff] [review]
Modified JSS/JSSE Client, JSS Server and all.pl to report failure

can you check all the tests 
ran by all.pl and ensure 
that each have System.exit 
with success and fails, also all.pl should print FAILURE at the end, plus 
report the status code.
Attachment #198273 - Flags: superreview?(glen.beasley) → superreview-
Checked and added System.exit(1) to all applicable files.
Attachment #198273 - Attachment is obsolete: true
Attachment #198526 - Flags: superreview?(glen.beasley)
Comment on attachment 198526 [details] [diff] [review]
Checked and added System.exit(1) to all applicable files

r+. note many of the files that you added  "system.exit(1)"
to are not actually ran by all.pl.  You should go back and
review those files to see if it is worth adding the test to all.pl.
If you do add the test make sure you remove
any deprecated code from it first.
Attachment #198526 - Flags: superreview?(glen.beasley) → superreview+
After adding System.exit(1) to all the relevant test source, the following
changes were also done

[1] Added test result status reporting to all.pl
[2] Added more tests to all.pl
[3] Removed deprecated code from test source
[4] Aligned test sources to match 80 column

This is how the test result looks like now:

JSSTEST_CASE 22 (Check JSS jar version): PASS
JSSTEST_SUITE: 22 / 22
JSSTEST_RATE: 100 %
Test Status: SUCCESS
Attachment #198526 - Attachment is obsolete: true
Attachment #199493 - Flags: superreview?(glen.beasley)
Comment on attachment 199493 [details] [diff] [review]
Added a few more tests to all.pl and also removed deprecated code from test source

your changes to KeyStoreTest.java
no longer use the "Mozilla-JSS" provider
which is the orginal 
intent of the test program. 

you did not remove the deprecated code from the sample programs:
hint: add -deprecation to javac lin in build_java.pl and you will see the
deprecation warnings. example warnings:

/usr/java/bin/javac -deprecation -g -sourcepath . -d ../../dist/classes_DBG 
./org/mozilla/jss/tests/SigTest.java ./org/mozilla/jss/tests/SymKeyGen.java
./org/mozilla/jss/tests/SymKeyGen.java:57: warning: [deprecation]
org.mozilla.jss.crypto.KeyGenerator in org.mozilla.jss.crypto has been
deprecated
	KeyGenerator kg = token.getKeyGenerator(KeyGenAlgorithm.DES);
	^
./org/mozilla/jss/tests/SymKeyGen.java:57: warning: [deprecation]
getKeyGenerator(org.mozilla.jss.crypto.KeyGenAlgorithm) in
org.mozilla.jss.crypto.CryptoToken has been deprecated
	KeyGenerator kg = token.getKeyGenerator(KeyGenAlgorithm.DES);
			       ^
./org/mozilla/jss/tests/SymKeyGen.java:77: warning: [deprecation]
org.mozilla.jss.crypto.Cipher in org.mozilla.jss.crypto has been deprecated
	Cipher cipher =
token.getCipherContext(EncryptionAlgorithm.DES_CBC_PAD);	^
./org/mozilla/jss/tests/SymKeyGen.java:77: warning: [deprecation]
getCipherContext(org.mozilla.jss.crypto.EncryptionAlgorithm) in
org.mozilla.jss.crypto.CryptoToken has been deprecated
	Cipher cipher =
token.getCipherContext(EncryptionAlgorithm.DES_CBC_PAD);		       
     ^
./org/mozilla/jss/tests/SymKeyGen.java:97: warning: [deprecation]
getKeyGenerator(org.mozilla.jss.crypto.KeyGenAlgorithm) in
org.mozilla.jss.crypto.CryptoToken has been deprecated
	kg = token.getKeyGenerator(KeyGenAlgorithm.DES3);
Attachment #199493 - Flags: superreview?(glen.beasley) → superreview-
JSS tests with deprecation removed but without KeyWrapping and SymKeyGen tests.  Also added is modifications to TestKeyGen.java.  The actual fix for this is outlined in bug 306347.  I will leave that bug open to address solutions recomended by Nelson.
Attachment #199493 - Attachment is obsolete: true
Attachment #202560 - Flags: superreview?(glen.beasley)
Comment on attachment 202560 [details] [diff] [review]
JSS tests with deprecation removed but without KeyWrapping and SymKeyGen tests

Per our conversation. 
Remove your changes to KeyStoreTest.java, and 
SigTest.java (put in comment saying code is deprecated look at JCASigTest for samples.)

remove tabs from SelfTest.java
Attachment #202560 - Flags: superreview?(glen.beasley) → superreview+
Checking in CloseDBs.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/CloseDBs.java,v  <--  CloseDBs.java
new revision: 1.7; previous revision: 1.6
done
Checking in GenerateTestCert.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/GenerateTestCert.java,v  <--  GenerateTestCert.java
new revision: 1.3; previous revision: 1.2
done
Checking in JSS_SSLClient.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/JSS_SSLClient.java,v  <--  JSS_SSLClient.java
new revision: 1.6; previous revision: 1.5
done
Checking in JSS_SSLServer.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/JSS_SSLServer.java,v  <--  JSS_SSLServer.java
new revision: 1.6; previous revision: 1.5
done
Checking in JSS_SelfServServer.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/JSS_SelfServServer.java,v  <--  JSS_SelfServServer.java
new revision: 1.2; previous revision: 1.1
done
Checking in ListCerts.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/ListCerts.java,v  <--  ListCerts.java
new revision: 1.7; previous revision: 1.6
done
Checking in PK10Gen.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/PK10Gen.java,v  <--  PK10Gen.java
new revision: 1.6; previous revision: 1.5
done
Checking in SDR.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/SDR.java,v  <--  SDR.java
new revision: 1.2; previous revision: 1.1
done
Checking in SelfTest.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/SelfTest.java,v  <--  SelfTest.java
new revision: 1.8; previous revision: 1.7
done
Checking in SigTest.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/SigTest.java,v  <--  SigTest.java
new revision: 1.6; previous revision: 1.5
done
Checking in TestKeyGen.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/TestKeyGen.java,v  <--  TestKeyGen.java
new revision: 1.8; previous revision: 1.7
done
Checking in VerifyCert.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/VerifyCert.java,v  <--  VerifyCert.java
new revision: 1.3; previous revision: 1.2
done
Checking in all.pl;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/all.pl,v  <--  all.pl
new revision: 1.26; previous revision: 1.25
done
Status: ASSIGNED → RESOLVED
Closed: 19 years ago
Resolution: --- → FIXED
Some more issues with System.exit(1) and SigTest.java
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Missed some System.exit(0) and reverted SigTest.java to original format as the JCA implementation can now be found in JCASigTest.java
Attachment #204077 - Flags: superreview?(glen.beasley)
Target Milestone: 4.1.1 → 4.2
Attachment #204077 - Flags: superreview?(glen.beasley) → superreview+
Checking in GenerateTestCert.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/GenerateTestCert.java,v  <--  GenerateTestCert.java
new revision: 1.4; previous revision: 1.3
done
Checking in JSSE_SSLClient.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/JSSE_SSLClient.java,v  <--  JSSE_SSLClient.java
new revision: 1.6; previous revision: 1.5
done
Checking in JSS_SelfServServer.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/JSS_SelfServServer.java,v  <--  JSS_SelfServServer.java
new revision: 1.3; previous revision: 1.2
done
Checking in ListCerts.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/ListCerts.java,v  <--  ListCerts.java
new revision: 1.8; previous revision: 1.7
done
Checking in PK10Gen.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/PK10Gen.java,v  <--  PK10Gen.java
new revision: 1.7; previous revision: 1.6
done
Checking in SDR.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/SDR.java,v  <--  SDR.java
new revision: 1.3; previous revision: 1.2
done
Checking in SigTest.java;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/SigTest.java,v  <--  SigTest.java
new revision: 1.7; previous revision: 1.6
done
Status: REOPENED → RESOLVED
Closed: 19 years ago → 19 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: