libpkix test scripts need to be enhanced

RESOLVED FIXED in 3.12

Status

P1
normal
RESOLVED FIXED
13 years ago
12 years ago

People

(Reporter: julien.pierre, Assigned: richard.freedman)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: PKIX)

(Reporter)

Description

13 years ago
Currently, libpkix has dozens of QA test that run and are invoked by all.sh, but they are summarized in the following categories in results.hmtl :

LIBPKIX Tests
Test Case	Result
RUNNING tests in pkix_pl_test 	Passed
RUNNING tests in pkix_test 	Passed
RUNNING performance tests in sample_apps 	Passed
ALL LIBPKIX TESTS COMPLETED SUCCESSFULLY 	Passed

There is no way to immediately tell which test fails. The tests scripts should be enhanced to show more details.
(Reporter)

Updated

13 years ago
Summary: libpkix scripts need to be enhanced → libpkix test scripts need to be enhanced
(Reporter)

Updated

13 years ago
Priority: -- → P2
Target Milestone: --- → 3.12
(Reporter)

Comment 1

13 years ago
We are getting a lot of test failures on the PKIX branch now. But all we see in the report is something like "27 LIBPKIX tests failed". This is not good enough. I'm raising the priority of this bug to P1. We cannot ship NSS 3.12 without this being fixed. Richard, can you please work on this ? This has priority over the HTTP cert store.

In addition, I think everything that is in libpkix.log should be piped to output.log as well.
Assignee: jason.m.reid → richard.freedman
Priority: P2 → P1
(Assignee)

Comment 2

13 years ago
The failures have been determined to be failures in the CRL checking, but more investigation is still needed to fix the problem. Meanwhile, libpkix test scripts have been enhanced to show in results.html details of tests, including how many pass (out of how many) in each category.

The changes are all in mozilla/security/nss/tests/libpkix:
Checking in libpkix.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/Attic/libpkix.sh,v  <--  libpkix.sh
new revision: 1.1.2.9; previous revision: 1.1.2.8
done
Checking in common/libpkix_init.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/common/Attic/libpkix_init.sh,v  <--  libpkix_init.sh
new revision: 1.1.2.13; previous revision: 1.1.2.12
done
Checking in pkix_pl_tests/runPLTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_pl_tests/Attic/runPLTests.sh,v  <--  runPLTests.sh
new revision: 1.1.2.9; previous revision: 1.1.2.8
done
Checking in pkix_pl_tests/module/runPLTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_pl_tests/module/Attic/runPLTests.sh,v  <--  runPLTests.sh
new revision: 1.1.2.17; previous revision: 1.1.2.16
done
Checking in pkix_pl_tests/pki/runPLTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_pl_tests/pki/Attic/runPLTests.sh,v  <--  runPLTests.sh
new revision: 1.1.2.15; previous revision: 1.1.2.14
done
Checking in pkix_pl_tests/system/runPLTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_pl_tests/system/Attic/runPLTests.sh,v  <--  runPLTests.sh
new revision: 1.1.2.9; previous revision: 1.1.2.8
done
Checking in pkix_tests/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.6; previous revision: 1.1.2.5
done
Checking in pkix_tests/certsel/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/certsel/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.10; previous revision: 1.1.2.9
done
Checking in pkix_tests/checker/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/checker/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.7; previous revision: 1.1.2.6
done
Checking in pkix_tests/crlsel/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/crlsel/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.8; previous revision: 1.1.2.7
done
Checking in pkix_tests/params/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/params/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.12; previous revision: 1.1.2.11
done
Checking in pkix_tests/results/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/results/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.10; previous revision: 1.1.2.9
done
Checking in pkix_tests/store/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/store/Attic/runTests.sh,v  <--  runTests.shnew revision: 1.1.2.9; previous revision: 1.1.2.8
done
Checking in pkix_tests/top/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/top/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.24; previous revision: 1.1.2.23
done
Checking in pkix_tests/util/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/util/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.9; previous revision: 1.1.2.8
done
Checking in sample_apps/runPerf.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/sample_apps/Attic/runPerf.sh,v  <--  runPerf.sh
new revision: 1.1.2.13; previous revision: 1.1.2.12
done
QA Contact: jason.m.reid → test
(Assignee)

Comment 3

13 years ago
Additional changes:

mozilla/security/nss/tests/libpkix/libpkix.sh
revision 1.1.2.10
date: 2006/03/29 21:59:37;  author: richard.freedman%sun.com;  state: Exp;  lines: +4 -2
Use an extra line to echo a blank line; echo does not interpret backslash-n
properly on all platforms.
****************************
mozilla/security/nss/tests/libpkix/common/libpkix_init.sh
revision 1.1.2.16
date: 2006/04/06 20:14:51;  author: richard.freedman%sun.com;  state: Exp;  lines: +2 -2
Once again include all output in log
----------------------------
revision 1.1.2.15
date: 2006/03/29 22:03:17;  author: richard.freedman%sun.com;  state: Exp;  lines: +8 -6
Use a shortened form of output for the HTML lines. With tracing on, use
shortened output in output log as well, for non-error lines.
----------------------------
revision 1.1.2.14
date: 2006/03/27 23:46:49;  author: richard.freedman%sun.com;  state: Exp;  lines: +3 -0
Fix calculations; output more detail information for pkix tests
****************************
mozilla/security/nss/tests/libpkix/pkix_tests/runTests.sh
revision 1.1.2.7
date: 2006/03/29 22:04:34;  author: richard.freedman%sun.com;  state: Exp;  lines: +1 -4
Some shell variables were not enclosed in braces. This may have been the
cause of some incorrect calculations.
****************************
mozilla/security/nss/tests/libpkix/pkix_tests/top/runTests.sh
revision 1.1.2.25
date: 2006/03/27 23:46:52;  author: richard.freedman%sun.com;  state: Exp;  lines: +7 -1
Fix calculations; output more detail information for pkix tests
Status: NEW → ASSIGNED
(Assignee)

Comment 4

13 years ago
The long pathnames are typically the result of prepending ${curdir} to a relative path. I have removed ${curdir}/ from a number of command lines, and the tests
run happily with their relative paths. The resulting output table is no longer the full width of a window.

Another set of changes:

Checking in libpkix/pkix_pl_tests/module/runPLTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_pl_tests/module/Attic/runPLTests.sh,v  <--  runPLTests.sh
new revision: 1.1.2.18; previous revision: 1.1.2.17
done
Checking in libpkix/pkix_pl_tests/pki/runPLTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_pl_tests/pki/Attic/runPLTests.sh,v  <--  runPLTests.sh
new revision: 1.1.2.16; previous revision: 1.1.2.15
done
Checking in libpkix/pkix_tests/certsel/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/certsel/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.11; previous revision: 1.1.2.10
done
Checking in libpkix/pkix_tests/crlsel/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/crlsel/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.9; previous revision: 1.1.2.8
done
Checking in libpkix/pkix_tests/params/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/params/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.13; previous revision: 1.1.2.12
done
Checking in libpkix/pkix_tests/results/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/results/Attic/runTests.sh,v  <--  runTests.sh
new revision: 1.1.2.11; previous revision: 1.1.2.10
done
Checking in libpkix/pkix_tests/top/runTests.sh;
/cvsroot/mozilla/security/nss/tests/libpkix/pkix_tests/top/Attic/runTests.sh,v <--  runTests.sh
new revision: 1.1.2.28; previous revision: 1.1.2.27
done

Updated

12 years ago
Whiteboard: PKIX

Comment 5

12 years ago
integrated into the truck
Status: ASSIGNED → RESOLVED
Last Resolved: 12 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.