347112 - all.sh html test names are not unique

Status: RESOLVED WORKSFORME

(NSS :: Test, defect, P2)

Description

[Nelson Bolyard (seldom reads bugmail)]

If one looks at the html output of all.sh, one finds that there are 
many sets of duplicated test names.  If one copies a test name out of that 
output, and pastes it into a bug report, to report a failure of a test,
it may be that there are numerous different tests that all have that same
name, and so the developer cannot determine from test name which exact test 
failed.

So, the test names that are reported in the all.sh html output need to have
names that are (a) unique [no two tests with the same name in the same run],
and (b) the same name for the same test from one run to the next , and
(c) findable by simple search in a shell script or data file.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Even worse than not being unique, some of the names are just wrong.

For example, we have a test where the server requests, but does not require
client authentication.  We want to test the behavior when the client does 
NOT authenticate to the server.  The test client has a cert and COULD 
authenticate itself to the server when the server requests it.  To prevent
that, we intentionally run the test client with the wrong password.  The 
name of that test is:  "Request don't require client auth (bad password)"

That name appears MANY times in the output log, which is bad because the 
reader cannot tell what factors differentiate the various tests that share
that name.  

But in fact, some of the tests that share that name do not use a "bad 
password" AT ALL.  For example, there is a test where the server requests
but does not require client auth, and the client DOES attempt to authenticate
with a cert, one that happens to be revoked.  There's no "bad password" 
involved in that test, but that name still appears in the log. 

Some tests are run more than once, under slightly different test conditions,
e.g. client bypass vs server bypass.  The name string reported in the 
results.html file and in the output log should include some text that 
identifies the different test condition (e.g. say "client bypass" or "server bypass" somewhere in the string).  

Fixing this bug may affect lots of lines of shell script.  But it should be
straight forward.  I think the project should be 1-2 days of work at most.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

This bug is now over a year old.  I want to see some traction on it soon.

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

Unsetting target milestone in unresolved bugs whose targets have passed.

Comment 4

[Marcus Burghardt]

I ran a full test right now and analysed the html report briefly.
Looks that the problems pointed in previous comments are solved in the current version.

Nelson, could you confirm if we can close this bug, please?

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:beurdouche, since the bug has high priority, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee is inactive on Bugzilla, and this bug has priority 'P2'.
:beurdouche, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 7

[Marco Castelluccio [:marco]]

We have modified the bot to only consider P1 as high priority, so I'm cancelling the needinfo here.

The reporter is not answering, so we can close this bug as WORKSFORME.