333766 - QA tests need to verify correct PATH and tools first

Status: NEW

(NSS :: Test, enhancement, P2)

Description

[Nelson Bolyard (seldom reads bugmail)]

In bug 331111 comment 5, Julien wrote:

> Could we have a test that checks for the proper tools installed on the 
> system required for QA , such as find ?

In bug 331111 comment 6, I wrote:

> Yeah, a pre-NSS-test QA test script whose only purpose is to verify that
> the tools and PATH are correct would be good.  

This request is specifically for Windows systems, but could also be done
for other platforms, too.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Further explanation:

There are several commands used in our QA test scripts that expect to invoke
Unix style programs with those command names, but for which there exist other
programs with the same names (on Windows) that do different things.  Examples
include (but are not limited to):  

find (which on Windows is a program like fgrep)
sort (which takes very different command line arguments on windows)

If a test system has its path misconfigured, so that (for example) the find
command does the wrong thing and fails to find any files (when it should find
some), then the QA test scripts will misbehave.  Sadly, those scripts do not
always detect the misbehavior, resulting in false green test results.

So, the point of this bug/RFE is request that this problem (wrong tools in 
path) be detected up front, (say, in init.sh), so that the entire test will
fail, and be reported as a failure, if the path is setup incorrectly.  

Comment 2

[Julien Pierre]

I think it might be possible to obtain a list of all the tools executed during all.sh by running truss on Unix .

Comment 3

[Slavomir Katuscak]

Decreasing priority to P3 (based on priorities set on meeting with Nelson in
September).

Comment 4

[Nelson Bolyard (seldom reads bugmail)]

Readjusting priority back to P2 to be consistent with the priority definitions
used in NSS.  
A fix for this bug/RFE is desired by one of the companies that sponsor NSS.  
By the present definition of NSS priorities, that makes it at least a P2.  

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

Unsetting target milestone in unresolved bugs whose targets have passed.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee is inactive on Bugzilla, and this bug has priority 'P2'.
:beurdouche, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 7

[Marco Castelluccio [:marco]]

We have modified the bot to only consider P1 as high priority, so I'm cancelling the needinfo here.