I have been trying to get the QA to run on OS/2 over the week-end. I ran into the following obstacles so far : 1) setting LIBPATH (BEGINLIBPATH, ENDLIBPATH, and LIBPATHSTRICT) The NSS init.sh attempts to set the path to load dynamic libraries from . On every other operating system, this can be set through an environment variable. On OS/2, there are pseudo-environment variables, BEGINLIBPATH and ENDLIBPATH. But these are in fact handled by the OS/2 shell program and turned into system calls to DosSetExtLIBPATH . We also want to set the LIBPATHSTRICT pseudo-environment variable (which is also handled by the OS/2 shell and translated to DosSetExtLIBPATH) in order to make sure that NSPR and NSS DLLs from other processes (eg. a running mozilla browser) don't get accidentally loaded if the DLLs cannot be loaded in the PATH, because the script needs to make sure it tests the DLLs in the test tree, not some other DLLs. It appears that the sh.exe program that the mozilla build uses on OS/2 does not know about these pseudo-environment variables and API calls. Thus, there is no way to set the library path from a shell script. Unless we can find a version of sh.exe that knows about these pseudo-environment variables, we'll have to use a .CMD script and the OS/2 CMD.EXE interpreter to be able to do that. This script could be used to wrap all.sh . 2) starting of selfserv ssl.sh attempts to start selfserv in the background by appending "&" at the end . I found that this is broken and causes the shell to hang in many cases . This is most likely a bug in the shell program. We'll have to add some OS/2-specific logic. I found that calling "CMD /C START selfserv.exe <arguments>" works better. 3) querying of running selfserv process. is_selfserv_running always thinks the selfserv process is not detectable. This is because kill -0 doesn't work. This may be another bug in the shell program, or there may just not be a way to check for a running process with a signal in OS/2. I think the former is true. So, we need to find another way to query for selfserv. Unfortunately, the current OS/2 mozilla build tools don't include a version of "ps", so the "ps -ef | grep $PID" test can't be used either. OS/2 includes a tool called PSTAT which does the job when passed the /P:pid argument, except that it only handles PIDs in hexadecimal . Is there a way to convert the pid from decimal to hexadecimal in the shell script, so it can be passed to PSTAT ? If not, would anybody object if I modified selfserv to write the pid in the pid file in hexadecimal for OS/2 only ? I think this feature of selfserv is used only for all.sh .
I think that NSS is already difficult enough to maintain and develop. Let's not add to the maintenance burden for all NSS developers, just for an OS that, at this point, is only a hobby OS for a very few developers. Is there a cygwin for OS/2? If not, maybe that would be a good project. Making OS/2 able to be a peer of the other tier-1 NSS platforms (by virtue of it having all the requisite shells and command tools) seems like a more worthy goal than adding many kludges to NSS to support OS/2. Much more software than merely NSS would be enabled by something like cygwin for OS/2. I'd prefer that effort was spent finding the necesssary tools to make OS/2 able to work with the existing scripts (e.g. ps and a bourne-like shell) than to make NSS scripts and test programs more ugly specifically for OS/2. That's my $.02
I would have to agree with Nelson here. Noone is interested in anything on OS/2 except NSS as it relates to the browser.
Nelson, There isn't a cygwin for OS/2. Many people have ported a lot of unix tools to OS/2 over the years, which is how Mozilla is able to build. But there is no single source for those tools. In general, each tool port comes from a different individual, and was done when a particular project needed the tool. Some of the ports have many bugs, or sometimes some of the features just can't be implemented in a fully compatible way because OS/2 underneath is too different (eg: the case of not using environment variables for library path, signal processing, etc.). We usually accomodate those platform differences in our code and scripts. The test scripts under nss/tests contain many statements like if [ "$OS_ARCH}" = WINNT ] . I see no reason not to add similar sections for OS/2 where required. We have done that in the past. The PS case even has traces in our current scripts of differences. I agree that we should seek the best possible OS/2 tools to run the QA. But that alone won't make the QA run on OS/2; some OS/2 specific changes are required, just like they were for WIN32. Michael, I'm interested in running the NSS OS/2 QA, and perhaps even an NSS OS/2 tinderbox. Running the QA may very well find OS/2-specific problems in NSS that affect the mozilla client as well. The sh.exe I was using was part of Perl 5.8.0, from 1997, and it is clearly broken. There is probably something better out there. Mike, do you know what's a better shell program to use here ?
Created attachment 179556 [details] [diff] [review] patch for tstclnt (checked in) PR_Poll doesn't work with stdin on OS/2, just like on Windows. Change #ifdefs appropriately. The setmode call isn't needed on OS/2, as the PR_StandardInput descriptor is going directly to the OS/2 console driver, without any processing of \r or \n .
I have succeeded in running of all.sh on OS/2. I found that the startup of selfserv in the shell (issue 2) wasn't really a problem after all. I had to convert the sslcov.txt, sslauth.txt, and sslstress.txt to Unix text format rather than DOS, because the sh.exe port only handles unix format. A lot of garbage was being passed to tstclnt and selfserv as arguments with the DOS format. To solve problem 3), I used a free tool called "go". It has a -cp <decimalpid> "check process" option, that does the equivalent of kill -0 . And -k <decimalpid> to kill the process. I believe we can add this tool to the required list of tools for mozilla. I haven't found a port of the ps and kill tools that have unix syntax. I don't have a good solution to problem 1) (setting library path). In the past, in NSS, we have had to write CMD scripts that run under the OS/2 command interpreter to set the path. For example, we do this to invoke SHLIBSIGN to sign the softokn3.dll library. Unless we can find a better OS/2 port of sh that can set the library path, I think we'll need a wrapper script as well here. Now that the QA runs, I have found that some of the tests don't pass. The following always fails : smime.sh: Verify Certs-Only by CA . FAILED The following two fail randomly : smime.sh: Decode Encrypted-Data . FAILED smime.sh: Compare Decoded and Original Data . FAILED Here is an except of output.log where all three tests failed : cmsutil -D -i co.der -d ../bobdir cmsutil: failed to decode message. cmsutil: problem decoding: security library: improperly formatted DER-encoded message. smime.sh: Verify Certs-Only by CA . FAILED smime.sh: Encrypted-Data Message --------------------------------- cmsutil -C -i alice.txt -e alicehello.env -d ../alicedir \ -r "firstname.lastname@example.org" > alice.enc smime.sh: Create Encrypted-Data . PASSED cmsutil -D -i alice.enc -d ../bobdir -e alicehello.env -p nss \ -o alice.data2 cmsutil: failed to decode message. cmsutil: problem decoding: security library: improperly formatted DER-encoded message. smime.sh: Decode Encrypted-Data . FAILED 1,6d0 < Date: Wed, 20 Sep 2000 00:00:01 -0700 (PDT) < From: email@example.com < Subject: message Alice --> Bob < To: firstname.lastname@example.org < < This is a test message from Alice to Bob. smime.sh: Compare Decoded and Original Data . FAILED The following SSL test also fails randomly. ssl.sh: Stress TLS RC4 128 with MD5 ---- There is nothing in the log that explains why. It may be a parsing problem with the scripts, or a bug with the HTML generation.
Comment on attachment 179556 [details] [diff] [review] patch for tstclnt (checked in) r=wtc. Just curious, why don't we put stdin into binary mode on OS/2? Is it because it's not necessary on OS/2, or because OS/2 doesn't have the setmode function?
Attachment #179556 - Flags: superreview+
Wan-Teh, Thanks for the review. In the OS/2 implementation NSPR, we use the DosWrite / DosRead calls for file access. Those kernel APIs always work in binary mode. There is a setmode call in the gcc libc DLL for OS/2, but it only applies to the read() and write() calls, which are also implemented in the gcc libc DLL.
I seem to recall we checked in a patch to do \n to \r\n conversion if stdout is the console. Did we check that in? If so, did we check in a corresponding patch for stdin?
The bug you are thinking of is https://bugzilla.mozilla.org/show_bug.cgi?id=104529 . It was only about PR_fprintf if the target fd was a tty . No patch for the read side was made. IMO, there is no possible fix that correctly abstracts the differences between OS/2, Windows and Unix in a general way . That is because NSPR doesn't have the notion of setting a PRFileDesc's text or binary mode in its API.
Attachment #179556 - Flags: review?(nelson) → review+
I checked in the fix for tstclnt . Thanks for the reviews. Checking in tstclnt.c; /cvsroot/mozilla/security/nss/cmd/tstclnt/tstclnt.c,v <-- tstclnt.c new revision: 1.40; previous revision: 1.39 done
QA Contact: bishakhabanerjee → jason.m.reid
Just going through still open OS/2 bugs. I don't really understand what it is about, the only patch went in. Is this still supposed to be open? Anything I can do to resolve it?
This was fixed I believe.
Status: NEW → RESOLVED
Last Resolved: 12 years ago
Resolution: --- → FIXED
No, the main issues described in comment 0 weren't fixed.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment on attachment 179556 [details] [diff] [review] patch for tstclnt (checked in) It appaers to me that thsi patch was checked in long ago. So, what is the status of this bug? Why is it still open?
Attachment #179556 - Attachment description: patch for tstclnt → patch for tstclnt (checked in)
Nelson, The status of this bug is that there are plenty of remaining issues that prevent the script from running on OS/2. Most of them are related to problems with the OS/2 port of the shell, such as its inability to set the path for shared libraries, the fact that signals that we use for selfserv probably won't work in the way we want them to. The library path issue can be worked around with a parent program that uses another shell. The selfserv issues are more problematic. There are some binary vs text issues also. I think the decision to keep this bug open or not comes down to whether we want to ever fix it or not. I'm not entirely sure, but the answer is probably no. A lot of the people who were keeping OS/2 alive worked for a company called Innotek. They were doing things like porting gcc to OS/2 among many others. With Sun's acquisition of Innotek, it seems that all this work will stop, or has already stopped. Just like when Stardivision was acquired by Sun - the OS/2 ports of Staroffice stopped. I find this regrettable. OS/2 is still alive on this end. I have just spend some time the last couple week ends converting my OS/2 PC to a virtual PC. It actually runs great under Microsoft virtual PC under Vista. At least Microsoft didn't get rid of the OS/2 guest support for Virtual PC when they acquired Connectix, though they killed the (host) OS/2 version of virtual PC. Before closing this bug as WONTFIX, I would like to hear a little more from the other OS/2 guys cc'ed to this bug who still may have an interest in having security ever QA'ed on OS/2.
As I haven't used the all.sh script to my knowledge until now I guess we can do without in the future. But wrapping it in an all.cmd for OS/2 doesn't seem too much work. I haven't really understood how selfserv fits into this...
Peter, Selfserv is an SSL HTTP server using NSS. It is used during the execution of the NSS QA to make sure that the SSL code in NSS works properly. There is also interoperability testing code that checks that the SSL code in NSS works with other clients and servers. The QA is fairly extensive.
You need to log in before you can comment on or make changes to this bug.