297573 - Modify JSS tests such that they access certs from platform specific directories

Status: RESOLVED FIXED

(JSS Graveyard :: Tests, defect)

Description

[Sandeep Konchady]

Currently JSS tests cannot be run in parallel.  This is because the tests access
certificates and keystore from a common location.  So to allow JSS tests to run
in parallel, as in the case of NSS, modify the tests such that the path to certs
and keystore can be provided as a commandline argument.  Also place these certs
and keystore in platform and option specific directory (EG: SunOS-5.9_DBG.OBJ).

Comment 1

[Christophe Ravel]

Attachment: Make $testdir a specific directory for each run

testdir should be different for each run (OS, OPT/DBG, 32/64) to allow to run
multiple JSS tests in parallel.
File modified: mozilla/security/jss/org/mozilla/jss/tests/all.pl

Rules proposed:

if release type of arguments:
    testdir = last part of nss_rel_dir (typically SunOS5.8_OPT.OBJ)
if dist type of arguments:
    testdir = last part of dist_dir (made absolute directory)

Fall back for manual testing special cases:
if testdir does not end with ".OBJ"
    testdir = hostname_pid

Comment 2

[Sandeep Konchady]

Comment on attachment 186262 [details] [diff] [review]
Make $testdir a specific directory for each run

I tested this on WinXP using dist option, and it works fine.

Comment 3

[Christophe Ravel]

Comment on attachment 186262 [details] [diff] [review]
Make $testdir a specific directory for each run

patch diff_297573_1 committed.

Checking in all.pl;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/all.pl,v  <--  all.pl
new revision: 1.18; previous revision: 1.17
done

Comment 4

[Christophe Ravel]

Attachment: Display the result of each test case and global pass rate

I would like to propose the following changes to the all.pl script to display
the actual result for each test.
The current behavior is:
- nothing if passed
- die with an error message if failed

This doesn't help to parse the test results.

My proposition:
- count the number of test cases run
- count the number of test cases passed
- for each test case:
    JSSTEST_CASE <test_number> (<test_title>): [PASS|FAIL]
- at the end:
    JSSTEST_SUITE: <testpass> / <testrun>
    JSSTEST_RATE: <pass_rate> %

Comment 5

[Christophe Ravel]

Comment on attachment 186510 [details] [diff] [review]
Display the result of each test case and global pass rate

Patch diff_297573_2 committed.

Checking in all.pl;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/all.pl,v  <--  all.pl
new revision: 1.19; previous revision: 1.18
done

Comment 6

[Wan-Teh Chang]

Comment on attachment 186262 [details] [diff] [review]
Make $testdir a specific directory for each run

Please avoid using tabs in NSS and JSS code.  If
you use tabs, find out what the right tab stop
(2, 4, or 8) is for that file.

Comment 7

[Sandeep Konchady]

Attachment: Modified JSS/JSSE Client-Server tests to use platform specific cert directory

Modified JSS/JSSE client server tests to use platform specific cert
directories.  Main modification is related to the tests accepting testdirectory
via commandline.  Also included is streamlining of JSSE_SSLClient to eliminate
while loop when performing startHandshake() as it is a synchronous call.  All
files were 4 space tab delimited, and modifications follow the same order.

Comment 8

[Christophe Ravel]

Comment on attachment 186564 [details] [diff] [review]
Modified JSS/JSSE Client-Server tests to use platform specific cert directory

Hi Sandeep,

Is there any reason why the program shouldn't accept absolute path for the cert
database directory ?
Your code seems to prevent it ("./" at the begining of the path).

/**
+ * Set the location of keystore.pfx
+ * @param String fKeystoreLoc
+ */
+ public void setKeystoreLoc(String fKeystoreLoc) {
+ keystoreLoc = "./" + fKeystoreLoc + "/" + keystoreLoc;
+ }
+

Comment 9

[glen beasley]

Comment on attachment 186564 [details] [diff] [review]
Modified JSS/JSSE Client-Server tests to use platform specific cert directory

The logic is fine, but please fix the identation.

Comment 10

[Christophe Ravel]

Attachment: Force Perl to do unbuffered output to avoid having Java and Perl output out of sync.

$|
    If set to nonzero, forces a flush after every write or print on the
currently selected output channel. Default is 0. Note that STDOUT will
typically be line buffered if output is to the terminal and block buffered
otherwise. Setting this variable is useful primarily when you are outputting to
a pipe, such as when you are running a perl script under rsh and want to see
the output as it's happening. (Mnemonic: when you want your pipes to be piping
hot.)

Comment 11

[Christophe Ravel]

Comment on attachment 186873 [details] [diff] [review]
Force Perl to do unbuffered output to avoid having Java and Perl output out of sync.

patch diff_297573_3 committed.

Checking in all.pl;
/cvsroot/mozilla/security/jss/org/mozilla/jss/tests/all.pl,v  <--  all.pl
new revision: 1.20; previous revision: 1.19
done
b

Comment 12

[Sandeep Konchady]

All JSS tests now accept cert directory as commandline input.  Parallel testing
on multiple platforms using a NFS location is now possible.  For test results
please refer to http://tinderbox.mozilla.org/showbuilds.cgi?tree=NSS.

Comment 13

[Wan-Teh Chang]

Sandeep, please set the target milestone (4.1?).  Thanks.

Comment 14

[Christophe Ravel]

Comment on attachment 186564 [details] [diff] [review]
Modified JSS/JSSE Client-Server tests to use platform specific cert directory

Removing the old review request.