all.sh NSS QA script should run on OS/2

REOPENED
Unassigned

Status

P3
normal
REOPENED
14 years ago
11 years ago

People

(Reporter: julien.pierre, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

14 years ago
I have been trying to get the QA to run on OS/2 over the week-end.

I ran into the following obstacles so far :

1) setting LIBPATH (BEGINLIBPATH, ENDLIBPATH, and LIBPATHSTRICT)

The NSS init.sh attempts to set the path to load dynamic libraries from . On
every other operating system, this can be set through an environment variable.
On OS/2, there are pseudo-environment variables, BEGINLIBPATH and ENDLIBPATH.
But these are in fact handled by the OS/2 shell program and turned into system
calls to DosSetExtLIBPATH .

We also want to set the LIBPATHSTRICT pseudo-environment variable (which is also
handled by the OS/2 shell and translated to DosSetExtLIBPATH) in order to make
sure that NSPR and NSS DLLs from other processes (eg. a running mozilla browser)
don't get accidentally loaded if the DLLs cannot be loaded in the PATH, because
the script needs to make sure it tests the DLLs in the test tree, not some other
DLLs.

It appears that the sh.exe program that the mozilla build uses on OS/2 does not
know about these pseudo-environment variables and API calls. Thus, there is no
way to set the library path from a shell script.

Unless we can find a version of sh.exe that knows about these pseudo-environment
variables, we'll have to use a .CMD script and the OS/2 CMD.EXE interpreter to
be able to do that. This script could be used to wrap all.sh .

2) starting of selfserv

ssl.sh attempts to start selfserv in the background by appending "&" at the end
. I found that this is broken and causes the shell to hang in many cases . This
is most likely a bug in the shell program. We'll have to add some OS/2-specific
logic. I found that calling "CMD /C START selfserv.exe <arguments>" works better.

3) querying of running selfserv process.

is_selfserv_running always thinks the selfserv process is not detectable. This
is because kill -0 doesn't work. This may be another bug in the shell program,
or there may just not be a way to check for a running process with a signal in
OS/2. I think the former is true. So, we need to find another way to query for
selfserv.

Unfortunately, the current OS/2 mozilla build tools don't include a version of
"ps", so the "ps -ef | grep $PID" test can't be used either. OS/2 includes a
tool called PSTAT which does the job when passed the /P:pid argument, except
that it only handles PIDs in hexadecimal .

Is there a way to convert the pid from decimal to hexadecimal in the shell
script, so it can be passed to PSTAT ?

If not, would anybody object if I modified selfserv to write the pid in the pid
file in hexadecimal for OS/2 only ? I think this feature of selfserv is used
only for all.sh .
(Reporter)

Updated

14 years ago
Priority: -- → P3
Target Milestone: --- → 3.10
I think that NSS is already difficult enough to maintain and develop. 
Let's not add to the maintenance burden for all NSS developers, just for 
an OS that, at this point, is only a hobby OS for a very few developers. 

Is there a cygwin for OS/2?  If not, maybe that would be a good project.
Making OS/2 able to be a peer of the other tier-1 NSS platforms (by virtue
of it having all the requisite shells and command tools) seems like a more 
worthy goal than adding many kludges to NSS to support OS/2.  Much more
software than merely NSS would be enabled by something like cygwin for OS/2.

I'd prefer that effort was spent finding the necesssary tools to make OS/2 
able to work with the existing scripts (e.g. ps and a bourne-like shell) 
than to make NSS scripts and test programs more ugly specifically for OS/2.  
That's my $.02

Comment 2

14 years ago
I would have to agree with Nelson here.

Noone is interested in anything on OS/2 except NSS as it relates to the browser.
(Reporter)

Comment 3

14 years ago
Nelson,

There isn't a cygwin for OS/2.

Many people have ported a lot of unix tools to OS/2 over the years, which is how
Mozilla is able to build. But there is no single source for those tools. In
general, each tool port comes from a different individual, and was done when a
particular project needed the tool. Some of the ports have many bugs, or
sometimes some of the features just can't be implemented in a fully compatible
way because OS/2 underneath is too different (eg: the case of not using
environment variables for library path, signal processing, etc.). We usually
accomodate those platform differences in our code and scripts. The test scripts
under nss/tests contain many statements like if [ "$OS_ARCH}" = WINNT ] . I see
no reason not to add similar sections for OS/2 where required. We have done that
in the past. The PS case even has traces in our current scripts of differences.

I agree that we should seek the best possible OS/2 tools to run the QA. But that
alone won't make the QA run on OS/2; some OS/2 specific changes are required,
just like they were for WIN32.

Michael,

I'm interested in running the NSS OS/2 QA, and perhaps even an NSS OS/2
tinderbox. Running the QA may very well find OS/2-specific problems in NSS that
affect the mozilla client as well.
The sh.exe I was using was part of Perl 5.8.0, from 1997, and it is clearly
broken. There is probably something better out there. Mike, do you know what's a
better shell program to use here ?
(Reporter)

Comment 4

14 years ago
Created attachment 179556 [details] [diff] [review]
patch for tstclnt (checked in)

PR_Poll doesn't work with stdin on OS/2, just like on Windows. Change #ifdefs
appropriately. The setmode call isn't needed on OS/2, as the PR_StandardInput
descriptor is going directly to the OS/2 console driver, without any processing
of \r or \n .
(Reporter)

Comment 5

14 years ago
I have succeeded in running of all.sh on OS/2.

I found that the startup of selfserv in the shell (issue 2) wasn't really a
problem after all.

I had to convert the sslcov.txt, sslauth.txt, and sslstress.txt to Unix text
format rather than DOS, because the sh.exe port only handles unix format. A lot
of garbage was being passed to tstclnt and selfserv as arguments with the DOS
format.

To solve problem 3), I used a free tool called "go". It has a -cp <decimalpid>
"check process" option, that does the equivalent of kill -0 . And -k
<decimalpid> to kill the process. I believe we can add this tool to the required
list of tools for mozilla.

I haven't found a port of the ps and kill tools that have unix syntax.

I don't have a good solution to problem 1) (setting library path).
In the past, in NSS, we have had to write CMD scripts that run under the OS/2
command interpreter to set the path. For example, we do this to invoke SHLIBSIGN
to sign the softokn3.dll library.

Unless we can find a better OS/2 port of sh that can set the library path, I
think we'll need a wrapper script as well here.

Now that the QA runs, I have found that some of the tests don't pass.

The following always fails :
smime.sh: Verify Certs-Only by CA . FAILED

The following two fail randomly :

smime.sh: Decode Encrypted-Data . FAILED
smime.sh: Compare Decoded and Original Data . FAILED

Here is an except of output.log where all three tests failed :

cmsutil -D -i co.der -d ../bobdir
cmsutil: failed to decode message.
cmsutil: problem decoding: security library: improperly formatted DER-encoded
message.
smime.sh: Verify Certs-Only by CA . FAILED
smime.sh: Encrypted-Data Message ---------------------------------
cmsutil -C -i alice.txt -e alicehello.env -d ../alicedir \
        -r "bob@bogus.com" > alice.enc
smime.sh: Create Encrypted-Data . PASSED
cmsutil -D -i alice.enc -d ../bobdir -e alicehello.env -p nss \
        -o alice.data2
cmsutil: failed to decode message.
cmsutil: problem decoding: security library: improperly formatted DER-encoded
message.
smime.sh: Decode Encrypted-Data . FAILED
1,6d0
< Date: Wed, 20 Sep 2000 00:00:01 -0700 (PDT)
< From: alice@bogus.com
< Subject: message Alice --> Bob
< To: bob@bogus.com
< 
< This is a test message from Alice to Bob.
smime.sh: Compare Decoded and Original Data . FAILED

The  following SSL test also fails randomly.

ssl.sh: Stress TLS  RC4 128 with MD5 ----

There is nothing in the log that explains why. It may be a parsing problem with
the scripts, or a bug with the HTML generation.
(Reporter)

Updated

14 years ago
Attachment #179556 - Flags: review?(nelson)

Comment 6

14 years ago
Comment on attachment 179556 [details] [diff] [review]
patch for tstclnt (checked in)

r=wtc.

Just curious, why don't we put stdin into
binary mode on OS/2?  Is it because it's not
necessary on OS/2, or because OS/2 doesn't
have the setmode function?
Attachment #179556 - Flags: superreview+
(Reporter)

Comment 7

14 years ago
Wan-Teh,

Thanks for the review.

In the OS/2 implementation NSPR, we use the DosWrite / DosRead calls for file
access. Those kernel APIs always work in binary mode.

There is a setmode call in the gcc libc DLL for OS/2, but it only applies to the
read() and write() calls, which are also implemented in the gcc libc DLL.

Comment 8

14 years ago
I seem to recall we checked in a patch to do \n to
\r\n conversion if stdout is the console.  Did we
check that in?  If so, did we check in a corresponding
patch for stdin?
(Reporter)

Comment 9

14 years ago
The bug you are thinking of is
https://bugzilla.mozilla.org/show_bug.cgi?id=104529 . It was only about
PR_fprintf if the target fd was a tty .

No patch for the read side was made.

IMO, there is no possible fix that correctly abstracts the differences between
OS/2, Windows and Unix in a general way . That is because NSPR doesn't have the
notion of setting a PRFileDesc's text or binary mode in its API.
Attachment #179556 - Flags: review?(nelson) → review+
(Reporter)

Comment 10

14 years ago
I checked in the fix for tstclnt . Thanks for the reviews.

Checking in tstclnt.c;
/cvsroot/mozilla/security/nss/cmd/tstclnt/tstclnt.c,v  <--  tstclnt.c
new revision: 1.40; previous revision: 1.39
done
(Reporter)

Updated

14 years ago
Target Milestone: 3.10 → 3.11
QA Contact: bishakhabanerjee → jason.m.reid
(Reporter)

Updated

13 years ago
Target Milestone: 3.11 → ---
QA Contact: jason.m.reid → test
(Reporter)

Updated

12 years ago
Assignee: julien.pierre.bugs → nobody

Comment 11

12 years ago
Just going through still open OS/2 bugs. I don't really understand what it is about, the only patch went in. Is this still supposed to be open? Anything I can do to resolve it?

Comment 12

12 years ago
This was fixed I believe.
Status: NEW → RESOLVED
Last Resolved: 12 years ago
Resolution: --- → FIXED
(Reporter)

Comment 13

12 years ago
No, the main issues described in comment 0 weren't fixed.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment on attachment 179556 [details] [diff] [review]
patch for tstclnt (checked in)

It appaers to me that thsi patch was checked in long ago.  
So, what is the status of this bug?
Why is it still open?
Attachment #179556 - Attachment description: patch for tstclnt → patch for tstclnt (checked in)
(Reporter)

Comment 15

11 years ago
Nelson,

The status of this bug is that there are plenty of remaining issues that prevent the script from running on OS/2. Most of them are related to problems with the OS/2 port of the shell, such as its inability to set the path for shared libraries, the fact that signals that we use for selfserv probably won't work in the way we want them to.

The library path issue can be worked around with a parent program that uses another shell. The selfserv issues are more problematic. There are some binary vs text issues also.

I think the decision to keep this bug open or not comes down to whether we want to ever fix it or not. I'm not entirely sure, but the answer is probably no.
A lot of the people who were keeping OS/2 alive worked for a company called Innotek. They were doing things like porting gcc to OS/2 among many others.

With Sun's acquisition of Innotek, it seems that all this work will stop, or has already stopped. Just like when Stardivision was acquired by Sun - the OS/2 ports of Staroffice stopped. I find this regrettable.

OS/2 is still alive on this end. I have just spend some time the last couple week ends converting my OS/2 PC to a virtual PC. It actually runs great under Microsoft virtual PC under Vista. At least Microsoft didn't get rid of the OS/2 guest support for Virtual PC when they acquired Connectix, though they killed the (host) OS/2 version of virtual PC.

Before closing this bug as WONTFIX, I would like to hear a little more from the other OS/2 guys cc'ed to this bug who still may have an interest in having security ever QA'ed on OS/2.

Comment 16

11 years ago
As I haven't used the all.sh script to my knowledge until now I guess we can do without in the future. But wrapping it in an all.cmd for OS/2 doesn't seem too much work. I haven't really understood how selfserv fits into this...
(Reporter)

Comment 17

11 years ago
Peter,

Selfserv is an SSL HTTP server using NSS. It is used during the execution of the NSS QA to make sure that the SSL code in NSS works properly. There is also interoperability testing code that checks that the SSL code in NSS works with other clients and servers. The QA is fairly extensive.
You need to log in before you can comment on or make changes to this bug.