Closed
Bug 232894
Opened 20 years ago
Closed 16 years ago
Automate the running of NIST PKITS tests in NSS nightly QA
Categories
(NSS :: Test, defect, P2)
Tracking
(Not tracked)
RESOLVED
FIXED
3.12
People
(Reporter: bishakhabanerjee, Assigned: slavomir.katuscak+mozilla)
References
(Blocks 1 open bug)
Details
Attachments
(1 file, 3 obsolete files)
2.16 KB,
patch
|
nelson
:
review+
|
Details | Diff | Splinter Review |
Run the set of NIST PKITS tests every night, with the rest of the NSS tests. Patch follows.
Reporter | ||
Comment 1•20 years ago
|
||
Reporter | ||
Comment 2•20 years ago
|
||
Comment 3•20 years ago
|
||
Comment on attachment 140431 [details] [diff] [review] patch to header to set dir for cert and CRL data for NIST tests I rather doubt this patch would work on Win2k.
Comment 4•20 years ago
|
||
Bishakha, Since we do not distribute the PKITS test data with NSS, we cannot make all.sh invoke pkits.sh.
Reporter | ||
Comment 5•20 years ago
|
||
Patch that addresses Nelson's latest comments from email: "1. We do not want to run wget and pull down a fresh copy of the zip file, and unzip the PKITS data with every tinderbox run. We want to allow the user to specify an environment variable stating where the files are. If the files are there, then we don't want to pull down a fresh copy. If the files aren't there, then we pull down a copy. " It would not be a problem to download the data and unzip every time for Tinderboxes (the time it takes to download and unzip is 3-4 minutes), but it would store a copy of the data files in every Tinderbox run, and take up space that we are borrowing from Anthony on the server. Hence, I went ahead and implemented Nelson's suggestion. Here are the relevant lines of code: ############################## pkits_init ############################## # local shell function to initialize this script ######################################################################## pkits_init() { SCRIPTNAME=pkits.sh if [ -z "${CLEANUP}" ] ; then CLEANUP="${SCRIPTNAME}" fi if [ -z "${INIT_SOURCED}" -o "${INIT_SOURCED}" != "TRUE" ]; then cd ../common . ./init.sh fi PKITSDIR=${HOSTDIR}/pkits mkdir -p ${PKITSDIR} if [ ! -d "${PKITS_DATA}" ]; then PKITS_DATA=${PKITSDIR}/PKITS_data mkdir -p ${PKITS_DATA} wget http://csrc.nist.gov/pki/testing/PKITS_data.zip \ -O ${PKITS_DATA}/PKITS_data.zip cd ${PKITS_DATA} unzip -o PKITS_data fi certs=${PKITS_DATA}/certs crls=${PKITS_DATA}/crls if [ ! -d "${certs}" ]; then echo "download test data from " echo "http://csrc.nist.gov/pki/testing/PKITS_data.zip " echo "and save to PKITS_DATA" exit 1 fi cd ${PKITSDIR} PKITSdb=${PKITSDIR}/PKITSdb PKITS_LOG=${PKITSDIR}/pkits.log #getting its own logfile if [ ! -d "${PKITSdb}" ]; then mkdir -p ${PKITSdb} else echo "$SCRIPTNAME: WARNING - ${PKITSdb} exists" fi echo "HOSTDIR" $HOSTDIR echo "PKITSDIR" $PKITSDIR echo "PKITSdb" $PKITSdb echo "PKITS_DATA" $PKITS_DATA echo "certs" $certs echo "crls" $crls echo nss > ${PKITSdb}/pw certutil -N -d ${PKITSdb} -f ${PKITSdb}/pw certutil -A -n TrustAnchorRootCertificate -t "C,C,C" -i \ $certs/TrustAnchorRootCertificate.crt -d $PKITSdb crlutil -I -i $crls/TrustAnchorRootCRL.crl -d ${PKITSdb} } "2. Rather than not running the PKITS tests, it would be preferable to run the pkits tests, but not allow PKITS failures to become tinderbox QA failures until we've resolved more of the CRL related issues." This may entail some amount of work. Before doing this (and I'm not averse to doing this), I'd like to know the timeframe for the CRL and UTF8 issues to get resolved. Is it going to be months, or a few weeks? In any case, this would need to be fixed in a different script, "header". Are the changes to pkits.sh okay to check in?
Comment 6•20 years ago
|
||
Bishakha, please make patches using cvs diff -u, and attach them as patches
to the bug rather than putting them in the comment.
Your newest code above contains these
> if [ ! -d "${PKITS_DATA}" ]; then
> PKITS_DATA=${PKITSDIR}/PKITS_data
> mkdir -p ${PKITS_DATA}
I think that test is insufficient. Imagine that on the first run,
it creates the directory, but the wget fails. Then on the second run,
it will not attempt to redo the wget.
You need to test for more than the presence of the directory. You also
need to test for the presence of a file in it. I'd suggest you test
for the presence of the last file to come out of the ZIP file.
Reporter | ||
Comment 7•20 years ago
|
||
Hi Nelson, We just discussed this, and I am attaching a patch of what I mentioned made with a cvs diff -u. Here, I am first checking to see if the top level dir, PKITS_DATA, is set in the environment (this check is mostly for our test and Tinderbox systems (and I am assuming that if taht is set to a networked dir, the underlying dirs would all be there). If I do not find the directory, I go ahead and download the data using wget. For instance, if you were using a system from home, or someone else wishing to download all the NSS tests to run them, and who would not have this env variable set, could also run this suite. Now, if your machine does not have wget, I do a furthur check later on to check for the cert and crl base directories, and exit if I do not find them. About testing for the last file in the zipped data file, as opposed to testing for the top-level directory, as I just mentioned to you, the number of default tries to connect in "wget" is 25. I can always up the number of tries, but I am fairly comfortable (even in the case of patchy network connections) that 25 tries would be more than fine to download one zipped datafile. Besides, for our automated test and Tinderbox cases, we wouldn't even go that route, since we have data on a network drive set as an environment variable.
Attachment #140431 -
Attachment is obsolete: true
Reporter | ||
Updated•20 years ago
|
Attachment #142375 -
Flags: review?(MisterSSL)
Comment 8•20 years ago
|
||
Comment on attachment 142375 [details] [diff] [review] patch v3 other problems with this patch. 1. If the user sets PKITS_DATA in his environment, but the directory does not exist, this patch will ignore the user's value of PKITS_DATA and create its own. 2. If the directory exists, but the contents are incomplate, this script will simply fail in each subsequent run, without attempting to fill it again. I believe that what you want to do is this (in pseudo code here): if (PKITS_DATA not defined in environment) { define PKITS_DATA to be ${PKITSDIR}/PKITS_data } if ($PKITS_DATA does not exist as a directroy) { create it; } if ($PKITS_DATA does not contain the certs and crls directories) { wget the zip file unzip it } if ($PKITS_DATA still does not contain the certs and crls directories) { complain and exit }
Attachment #142375 -
Flags: review?(MisterSSL) → review-
Updated•19 years ago
|
Assignee: bishakhabanerjee → jason.m.reid
QA Contact: bishakhabanerjee → jason.m.reid
Updated•18 years ago
|
Assignee: jason.m.reid → nobody
QA Contact: jason.m.reid → test
Updated•18 years ago
|
Assignee: nobody → richard.freedman
OS: Windows 2000 → All
Priority: -- → P2
Hardware: PC → All
Target Milestone: --- → 3.12
Comment 9•18 years ago
|
||
tests/libpkix/common/libpkix_init_nist.sh checks for two environment variables. If NIST_FILES_DIR is defined, tests using the NIST files will be run, and will expect to find the files in that directory. If it is NULL or not defined, those tests will be skipped. If NIST_FILES_DIR is defined, there is an additional check for whether PDVAL is defined. If both variables are defined, then a couple of additional tests are run. Those tests use URIs found in AIA extensions to try to find additional certs from an LDAP server, for path validation. Again, those tests are skipped if either NIST_FILES_DIR or PDVAL is undefined. When LIBPKIX's HttpCertStore is ready, more tests will be added under PDVAL, with URIs pointing to HTTP sources as well as LDAP sources. I am ready to check in a third set of tests, to be executed or skipped according to whether OCSP is defined. These tests will access an OCSP server (whose URI comes from an AIA extension). But the necessary files include some certs whose creation (because of the AIAs) is beyond the present capability of certutil, and the necessary server has to be running on pride.red.iplanet.com. Obviously this is a stopgap solution, but is worth including to capture the test for LIBPKIX's OCSP checking. Who will have responsibility for coordinating the environment values of NIST_FILES_DIR, PDVAL, and OCSP for the libpkix tests with the existence of the necessary files and servers?
Updated•17 years ago
|
Assignee: richard.freedman → slavomir.katuscak
Assignee | ||
Comment 10•17 years ago
|
||
I'm going to prepare new patch for this bug, but first I want to write my idea how it should look: 1. if PKITS_DATA variable defined and data OK then goto 7 2. if PKITS_DATA variable defined and data not OK then complain (warning) and continue 3. define (or redefine) PKITS_DATA variable to ${PKITSDIR}/PKITS_data 4. mkdir PKITS_DATA 5. wget zip file and unzip it 6. if data not OK then complain (warning) and exit (skip PKITS tests) 7. PKITS testing My idea of checking if data are OK is to list directory recursively and count lines (`ls -1R | wc -l` -eq 1573). Nelson, please send me feedback, if this is OK with you, or some suggestions what to improve.
Assignee | ||
Updated•16 years ago
|
Comment 11•16 years ago
|
||
Slavo, It sounds to me like your proposal will make this test script dependent on a test program that is NOT (or may not be) a part of the standard set of test programs available on every supported platform. I am referring to wget. It is undesirable to increase the number of tools (test programs not built as part of NSS) required on every platform in order to test NSS. The creation of a pkits data directory should be viewed as part of the test machine setup, just as installing the compilers, shells, etc. are also part of the machine setup. Your script should merely determine if the data is available (at the directory named in some shell variable) and run PKITS if it is present, and do not run PKITS if the data is absent.
Assignee | ||
Comment 12•16 years ago
|
||
Simplified to 2 steps: 1. Check if PKITS_DATA defined, if not then print message and skip PKITS testing. 2. Check if directory with PKITS_DATA exists, if not then print message and skip PKITS testing. No error messages in case that PKITS_DATA is not defined or directory doesn't exists, only message in logfile.
Attachment #140430 -
Attachment is obsolete: true
Attachment #142375 -
Attachment is obsolete: true
Attachment #310201 -
Flags: review?(nelson)
Updated•16 years ago
|
Attachment #142375 -
Attachment description: patch → patch v3
Updated•16 years ago
|
Attachment #310201 -
Attachment description: Patch. → Patch v4
Comment 13•16 years ago
|
||
Comment on attachment 310201 [details] [diff] [review] Patch v4 Please make these very minor changes, then you can commit. >-tests="cipher perf libpkix cert dbtests tools fips sdr crmf smime ssl ocsp merge" >+tests="cipher perf libpkix cert dbtests tools fips sdr crmf smime ssl pkits ocsp merge" Make pkits last in the list. >+ if [ ! -d "${PKITS_DATA}" ]; then >+ echo "${SCRIPTNAME}: PKITS data directory ${PKITS_DATA} doesn't exists, skipping." change "exists" to "exist".
Attachment #310201 -
Flags: review?(nelson) → review+
Assignee | ||
Comment 14•16 years ago
|
||
Checking in all.sh; /cvsroot/mozilla/security/nss/tests/all.sh,v <-- all.sh new revision: 1.47; previous revision: 1.46 done Checking in pkits/pkits.sh; /cvsroot/mozilla/security/nss/tests/pkits/pkits.sh,v <-- pkits.sh new revision: 1.21; previous revision: 1.20 done
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•