257071 - build issue with certain cygwin configurations

Status: RESOLVED WONTFIX

(NSS :: Build, defect)

Description

[Julien Pierre]

I recently found that there is an NSS build problem on Windows if you use
gmake.exe from the moztools package recommended on mozilla.org, but use cygwin
for the rest of the unix tools. Ie. if you use cygwin except have a non-cygwin
version of gmake .

The problem is related to the addition of building absolute pathnames that I
made about a year ago. In this situation, the NSS build system doesn't correctly
detect cygwin, because it depends on the cygwin gmake to detect cygwin.
Curiously, NSPR does not have this same issue.

Here is how the problem manifests itself :

gmake[2]: Entering directory `C:/NSS/tip5/mozilla/security/nss/lib/util'
cl -FoWINNT5.0_DBG.OBJ/quickder.obj -c -Od -Z7 -MD -W3 -nologo -GT -DXP_PC -DDEB
UG -D_DEBUG -UNDEBUG -DDEBUG_Administrator -DWIN32 -D_WINDOWS -D_X86_ -DWINNT -I
../../../../dist/WINNT5.0_DBG.OBJ/include  -I../../../../dist/public/nss -I../..
/../../dist/private/nss -I../../../../dist/public/dbm  /cygdrive/c/NSS/tip5/mozi
lla/security/nss/lib/util/quickder.c
Command line warning D4002 : ignoring unknown option '/cygdrive/c/NSS/tip5/mozil
la/security/nss/lib/util/quickder.c'
Command line error D2003 : missing source filename
gmake[2]: *** [WINNT5.0_DBG.OBJ/quickder.obj] Error 2
gmake[2]: Leaving directory `C:/NSS/tip5/mozilla/security/nss/lib/util'
gmake[1]: *** [libs] Error 2
gmake[1]: Leaving directory `C:/NSS/tip5/mozilla/security/nss/lib'
gmake: *** [libs] Error 2

The problem is in mozilla/security/coreconf/rules.mk

ifeq (,$(filter-out _WIN%,$(NS_USE_GCC)_$(OS_TARGET)))
NEED_ABSOLUTE_PATH := 1
ifeq (,$(findstring ;,$(PATH)))
PWD :=  $(subst \,/,$(shell cygpath -w `pwd`))
else
PWD := $(shell pwd)
endif
endif

For this case, findstring ; is not a good test, because non-cygwin gmake is
used, so the cygpath tool does not get invoked, and thus the untranslated cygwin
path gets passed to the compiler, which causes the failure.

Comment 1

[Wan-Teh Chang]

If you use Cygwin's Bash as the shell for make, you
must use Cygwin's make.

Comment 2

[Julien Pierre]

Wan-Teh,

That wasn't the case here. I was using gmake.exe from moztools, which I believe
is hardcoded to use the shmsdos.exe it comes with. The rest of the tools in my
path were cygwin tools. This happened at first because I had downloaded an
incomplete cygwin package which didn't contain gmake.

The NSS build system couldn't deal with this situation (see the build error).
The NSPR build system somehow could (gmake build_nspr worked).

Comment 3

[Wan-Teh Chang]

Marked the bug WONTFIX.  I don't want to
support a partially-Cygwin build environment.

Comment 4

[Julien Pierre]

The mozilla build instructions frequently leads to this incorrect environment.
This build error comes up repeatedly in the newsgroups. If we don't want to
support this environment in NSS, then we should have the mozilla build
instructions modified to include information about the importance of PATH and
matching gmake to the windows-unix toolkit in use .

Comment 5

[Wan-Teh Chang]

This can be fixed by replacing
    $(PATH)
with
    $(shell echo $$PATH)

$(PATH) evaluates the PATH *make* variable, whose
value is from the PATH *environment* variable but
has been converted by make.  Note: we need to verify
this.  I found this empirically but didn't find it
documented in the GNU Make manual.

$(shell echo $$PATH) evaluates the PATH *environment*
variable in the shell used by make.

The latter is a more accurate way to determine
if we are using Cygwin (pure Cygwin, or Cygwin
with non-Cygwin make).

You can use this simple makefile to see the
difference between the two:

all:
	@echo '$(PATH)'
	@echo
	@echo
	@echo '$(shell echo $$PATH)'

Since pwd is a standalone utility in both MKS
and Cygwin, the above solution is still not
bullet proof, but it'll fix the most common
problem: using non-Cygwin make with Cygwin.

The reason NSPR isn't affected by this problem
is that NSPR uses the following test in its
configure script to determine if Cygwin path
conversion needs to be done:

    if test `echo "${PATH}" | grep -c \;` = 0; then
        CYGWIN_WRAPPER='sh $(topsrcdir)/build/cygwin-wrapper'
    fi

This ${PATH} is evaluated in the shell, not in make.
So my proposed fix above is in the same spirit.

If you can test this patch, I'll reopen the bug
and check it in.  We should also open a bug on
Mozilla build instructions so we don't forget to
update them with a warning about this issue.

Comment 6

[Wan-Teh Chang]

Attachment: Proof of concept (do not check in)

By the way, under Cygwin, we can use cygpath
directly to convert a pathname to an absolute
pathname.  This patch uses that method.  But
I don't want to check it in because it incurs
one cygpath process execution for each file
we compile.

Comment 7

[Wan-Teh Chang]

Attachment: Patch to fix the compilation problem only

I found that using $(shell echo $$PATH) instead
of $(PATH) in coreconf/rules.mk indeed fixes the
*compilation* problem reported in this bug, but
the build still fails when we build the static
library secutil.lib:

lib -NOLOGO -OUT:"WIN954.0_DBG.OBJD/secutil.lib"
WIN954.0_DBG.OBJD\\quickder.obj
 WIN954.0_DBG.OBJD\\secdig.obj WIN954.0_DBG.OBJD\\derdec.obj
WIN954.0_DBG.OBJD\\
derenc.obj WIN954.0_DBG.OBJD\\dersubr.obj WIN954.0_DBG.OBJD\\dertime.obj
WIN954.
0_DBG.OBJD\\nssb64d.obj WIN954.0_DBG.OBJD\\nssb64e.obj
WIN954.0_DBG.OBJD\\nssrwl
k.obj WIN954.0_DBG.OBJD\\nssilock.obj WIN954.0_DBG.OBJD\\nsslocks.obj
WIN954.0_D
BG.OBJD\\portreg.obj WIN954.0_DBG.OBJD\\pqgutil.obj
WIN954.0_DBG.OBJD\\secalgid.
obj WIN954.0_DBG.OBJD\\secasn1d.obj WIN954.0_DBG.OBJD\\secasn1e.obj
WIN954.0_DBG
.OBJD\\secasn1u.obj WIN954.0_DBG.OBJD\\secitem.obj
WIN954.0_DBG.OBJD\\secoid.obj
 WIN954.0_DBG.OBJD\\sectime.obj WIN954.0_DBG.OBJD\\secport.obj
WIN954.0_DBG.OBJD
\\secinit.obj WIN954.0_DBG.OBJD\\utf8.obj
LIB : fatal error LNK1181: cannot open input file
'WIN954.0_DBG.OBJDquickder.obj

So there are other problems using non-Cygwin gmake with
Cygwin's bash.

I can get past the static library build problem by not
changing / to \\ in those pathnames or by changing
/ to \\\\.  I seem to recall Bob Relyea added the
substitution of / by \\ when he implemented the BUILD_TREE
support because an absolute pathname that begins with / is
interpreted by Windows build tools as a command line flag
(Windows build tools treat /x and -x interchangeably).
Apparently, the passing of \ from gmake to the shell has
strange rules that depend on the type of gmake and the
type of shell.

This is becoming a time sink as I feared, so I'm going to
stop here.