301035 - Issues building LDAPCSDK_5_1_6_RTM and PerLDAP on Windows

Status: RESOLVED FIXED

(Directory :: LDAP C SDK, defect)

Description

[mguessan]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.8) Gecko/20050511 Firefox/1.0.4
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.8) Gecko/20050511 Firefox/1.0.4

I managed to build both the LDAP C SDK 5.1.6 and
PerLDAP 1.5b4 on windows... Yet I had some trouble
along the way :
- dist directories inconsistent (NSS/LDAPCSDK)
- the libutil library is needed by client tools but *not*
built automatically
- the include files directories are not those expected
by PerLDAP
- Some issues in PERLDAP code

Reproducible: Always

Steps to Reproduce:
First the environment :
- Windows XP Pro SP2
- Visual C++ 6 + SP5 + Processor Pack
- ActiveState Perl v5.8.6
- Cygwin

I tried to follow the guidelines at :
http://www.mozilla.org/directory/csdk.html

- First the checkout :
set CVSROOT=:pserver:[e-mail]:/cvsroot
cvs login
(password anonymous)

cvs co -r NSPR_4_4_1_RTM mozilla/nsprpub
cvs co -r NSS_3_9_3_RTM mozilla/security/coreconf mozilla/security/nss
cvs co -r DBM_1_61_RTM mozilla/dbm mozilla/security/dbm
cvs co -r SVRCORE_4_0_RTM mozilla/security/svrcore
cvs co -r LDAPCSDK_5_1_6_RTM DirectorySDKSourceC

- then some more build tools :
Download
http://ftp.mozilla.org/pub/mozilla.org/mozilla/source/wintools.zip

set MOZ_TOOLS=D:\moztools
set PATH=%PATH%;%MOZ_TOOLS%\bin
... and launch install.bat

- setup VC++ environment
"C:\Program Files\Microsoft Visual Studio\VC98\Bin\VCVARS32.BAT"

- switch to cygwin bash

- I wanted to build an optimized version, so I added
BUILD_OPT=1 in mozilla/security/coreconf/config.mk
(don't know if this is the right way, but at least this works)

- build NSS
cd mozilla/security/nss
make nss_build_all

- build SVRCORE
cd mozilla/security/svrcore
make

- NSS dist directory is dist/WINNT5.1_OPT.OBJ, the LDAP SDK expects
dist directly, so I moved everything from WINNT5.1_OPT.OBJ to dist

- then the LDAP SDK stuff, again optimized build
cd mozilla/directory/c-sdk
./configure --with-nss --enable-optimize --disable-debug
make


- the libutil library is needed by client tools but *not*
built automatically, sounds like a bug to me :

cd ldap/libraries/libutil
BUILD_OPT=1 in Makefile.client
make -f Makefile.client install

- again the target directory is dist/WINNT5.1_OPT.OBJ,
move everything to dist

- build client tools
cd mozilla/directory/c-sdk
make BUILDCLU=1 HAVE_SVRCORE=1

- the include files directories are not those expected
by PerLDAP, as a quick fix I moved public/ldap/*.h to include

- Now the PerLDAP part :
- the Makefile.PL does not work on windows, I patched
the library names :
diff PerLDAP-1.5.orig/Makefile.PL PerLDAP-1.5/Makefile.PL
42c42
< $libexts = "so|sl|a|lib";
---
> $libexts = "so|sl|a|lib|dll";
100,102c100,102
<    @ldaplib = grep{/^ldap.*$/} @libs;
<    @prldaplib = grep{/^prldap.*$/} @libs;
<    @lberlib = grep{/^lber.*$/} @libs;
---
>    @ldaplib = grep{/^(ns)?ldap.*$/} @libs;
>    @prldaplib = grep{/^(prldap|nsldappr).*$/} @libs;
>    @lberlib = grep{/^(ns)?lber.*$/} @libs;
104c104
<    @ldapslib = grep{/^ssldap.*$/} @libs if $ssl_def;
---
>    @ldapslib = grep{/^(ssldap|nsldapssl).*$/} @libs if $ssl_def;
191,192c191,192
<    $my_extlib = "$lib_ldap\\$ldaplib[0]";
<    $my_extlib .= " $lib_ldap\\$lberlib[0]" if $#lberlib >= 0;
---
>    $my_extlib = "$lib_ldap\\$ldaplib[0].lib";
>    $my_extlib .= " $lib_ldap\\$lberlib[0].lib" if $#lberlib >= 0;
(see attached new file)

- internal_sortcmp_proc LDAPCALL marker and StrCaseCmp signature do
not suit VC++ compiler, patched API.xs :
diff PerLDAP-1.5.orig/API.xs PerLDAP-1.5/API.xs
585d584
< LDAP_CALL
1252,1253c1251
<        int        (*func)() = &StrCaseCmp;
<
---
>        int        (*func)(const char *s, const char *t) = &StrCaseCmp;
1627c1625
<        int        (*func)() = &StrCaseCmp;
---
>        int        (*func)(const char *s, const char *t) = &StrCaseCmp;

- And another little bug in Conn.pm :
diff PerLDAP-1.5.orig/Conn.pm PerLDAP-1.5/Conn.pm
81a82
>       $self->{"entryclass"} = 'Mozilla::LDAP::Entry';
(the non hash constructor did not set the default entryclass)

- set VC++ env (if not already done)
"C:\Program Files\Microsoft Visual Studio\VC98\Bin\VCVARS32.BAT"
- check that the perl in PATH is Activestate and not cygwin :
which perl
- set PerLDAP environment
set LDAPSDKDIR=D:\cvs\mozilla\ldapcsdk5.1.6-WINNT5.1_OPT.OBJ
set LDAPSDKSSL=Y
set LDAPPR=Y
- build (use nmake)
perl Makefile.PL
nmake

- add ldapcsdk libraries (quick but not clean ;-) )
copy %LDAPSDKDIR%\lib\*.dll blib\arch\auto\Mozilla\LDAP\API
- build ppm
tar cvf package.tar blib
gzip --best package.tar
nmake ppd
- install ppm
ppm install PerLDAP.ppd
- enjoy ;-)

Comment 1

[Mark Smith [:mcs]]

Thank you for the detailed bug report.  Integration of the LDAP C SDK build with
the other components (NSS, NSPR, etc.) has always been more fragile than it
should be.

I think we should split the PerLDAP issues into a separate bug report. 

I added Rich and Leif to the CC list.

Comment 2

[mguessan]

Leif committed most of my patch, however I still have an issue 
with the internal_sortcmp_proc function.

Extraction from my initial patch :
diff PerLDAP-1.5.orig/API.xs PerLDAP-1.5/API.xs
585d584
< LDAP_CALL

- the full internal_sortcmp_proc signature is :
static
int
LDAP_CALL
internal_sortcmp_proc(const char *s1, const char *s2)

- func is now declared as
int        (*func)(const char *s, const char *t)

The windows compiler does not want to assign internal_sortcmp_proc to
func :
func =&internal_sortcmp_proc;

Not sure what the LDAP_CALL macro is supposed to do, so I removed
it from internal_sortcmp_proc and at least it compiles and seems to
work... although I must admit this may be wrong.

Comment 3

[Rich Megginson]

Attachment: files for fix

Comment 4

[Rich Megginson]

Attachment: diffs for fix

Comment 5

[Rich Megginson]

I have omitted the diffs for configure itself - too large and too difficult to read.

Comment 6

[Rich Megginson]

1) Don't set the -lsvrcore in SVRCORE_LIBS in configure - instead, do it the way we do the NSPR and NSS libs, in build.mk
2) For the component LINK macros, use name.lib instead of -lname on Windows
3) The new free MSVC compiler does not have lib.exe - add an autoconf test for the lib program and use link /lib if missing
4) If using MSVC (which expects DOS style absolute paths) and cygwin (which uses unix style abs. paths), use cygpath -m on all user supplied paths to make sure they are in the correct format for cl and link.  It's better to do this during configure rather than depend on cygwin_wrapper which is much, much slower.
5) Don't link with the odbc libs (Why did we ever need these!?!??!?)
6) The free MSVC doesn't include afxwin.h, so use the other header files instead
7) Add libutil to the Windows build, including getting rid of the old Makefile and adding a new Makefile.in for the autoconf build (the Makefile.client remains)
8) getopt.c doesn't need lber.h

Comment 7

[Rich Megginson]

see also bug 325518

Comment 8

[Mark Smith [:mcs]]

(In reply to comment #6)
> 4) If using MSVC (which expects DOS style absolute paths) and cygwin (which
> uses unix style abs. paths), use cygpath -m on all user supplied paths to make
> sure they are in the correct format for cl and link.  It's better to do this
> during configure rather than depend on cygwin_wrapper which is much, much
> slower.

I am a little confused about this change.  Does this mean that cygwin_wrapper is no longer used at all?  If so, that is inconsistent with how the rest of the Mozilla code is built, which might lead to confusion.

> 5) Don't link with the odbc libs (Why did we ever need these!?!??!?)

I do not know.  Does everything still build with older MSVC (6, 7)?

> 7) Add libutil to the Windows build, including getting rid of the old Makefile
> and adding a new Makefile.in for the autoconf build (the Makefile.client
> remains)

Where was getopt picked up from before?

Comment 9

[Rich Megginson]

> I am a little confused about this change.  Does this mean that cygwin_wrapper
> is no longer used at all?  If so, that is inconsistent with how the rest of the
> Mozilla code is built, which might lead to confusion.

I'm not sure if we need cygwin-wrapper anymore with autoconf builds.  But in the code, if it wants a Windows style path, and it sees that it already has a driveletter + ":", it won't do anything, so I think it should still work as before.

> re: odbc.  I do not know.  Does everything still build with older MSVC (6, 7)?

I haven't tried it yet, but I don't see why not.

> Where was getopt picked up from before?

getopt/libutil is only used by the command line tools.  Mozilla builds don't care about the command line tools so they never even built it.  Our (Netscape/RH) internal builds use a different build process that did pick up libutil.  AFAIK, this is the first time anyone ever tried to build the command line tools on Windows using the autoconf builds.

Comment 10

[Mark Smith [:mcs]]

(In reply to comment #9)
> I'm not sure if we need cygwin-wrapper anymore with autoconf builds.  But in
> the code, if it wants a Windows style path, and it sees that it already has a
> driveletter + ":", it won't do anything, so I think it should still work as
> before.

OK.

> > re: odbc.  I do not know.  Does everything still build with older MSVC (6, 7)?
> 
> I haven't tried it yet, but I don't see why not.

Are you able to try it?  If not, let me know and I will make time to do so.  I am still using MSVC 6.


> > Where was getopt picked up from before?
> 
> getopt/libutil is only used by the command line tools.  Mozilla builds don't
> care about the command line tools so they never even built it.  Our
> (Netscape/RH) internal builds use a different build process that did pick up
> libutil.  AFAIK, this is the first time anyone ever tried to build the command
> line tools on Windows using the autoconf builds.

I think it has been done before but there were problems... this is likely one of them ;-)

Comment 11

[Rich Megginson]

I'll have to boot up my old PC that has the msvc6 on it.

Comment 12

[Rich Megginson]

Attachment: patch for testing fix

Attaching the patch file so I can test this on another machine.

Comment 13

[Rich Megginson]

Attachment: patch for configure

patch for configure

Comment 14

[Rich Megginson]

Attachment: new Makefile.in

Comment 15

[Rich Megginson]

I have confirmed that the proposed code builds and runs on Windows 2000 using MSVC 6 sp 3.  It even finds lib.exe and uses it - cool!

Comment 16

[Mark Smith [:mcs]]

Comment on attachment 210608 [details]
new Makefile.in

Looks OK but please fix the "Original Code" and "Initial Developer" portions of the license text.  You might use something like:

 * The Original Code is mozilla.org code.
 *
 * The Initial Developer of the Original Code is
 *    Rich Megginson <[e-mail]>
 * Portions created by the Initial Developer are Copyright (C) 2006
 * the Initial Developer. All Rights Reserved.

Comment 17

[Rich Megginson]

*** Bug 325518 has been marked as a duplicate of this bug. ***

Comment 18

[Rich Megginson]

Here are the checkins for the Windows/cygwin parts:
Checking in mozilla/directory/c-sdk/build.mk;
/cvsroot/mozilla/directory/c-sdk/build.mk,v  <--  build.mk
new revision: 5.27; previous revision: 5.26
done
Checking in mozilla/directory/c-sdk/configure;
/cvsroot/mozilla/directory/c-sdk/configure,v  <--  configure
new revision: 5.44; previous revision: 5.43
done
Checking in mozilla/directory/c-sdk/configure.in;
/cvsroot/mozilla/directory/c-sdk/configure.in,v  <--  configure.in
new revision: 5.45; previous revision: 5.44
done
Checking in mozilla/directory/c-sdk/config/autoconf/svrcore.m4;
/cvsroot/mozilla/directory/c-sdk/config/autoconf/svrcore.m4,v  <--  svrcore.m4
new revision: 5.2; previous revision: 5.1
done
Checking in mozilla/directory/c-sdk/ldap/clients/tools/Makefile.in;
/cvsroot/mozilla/directory/c-sdk/ldap/clients/tools/Makefile.in,v  <--  Makefile
.in
new revision: 5.12; previous revision: 5.11
done
Checking in mozilla/directory/c-sdk/ldap/clients/tools/convutf8.cpp;
/cvsroot/mozilla/directory/c-sdk/ldap/clients/tools/convutf8.cpp,v  <--  convutf
8.cpp
new revision: 5.7; previous revision: 5.6
done
Checking in mozilla/directory/c-sdk/ldap/libraries/Makefile.in;
/cvsroot/mozilla/directory/c-sdk/ldap/libraries/Makefile.in,v  <--  Makefile.in
new revision: 5.4; previous revision: 5.3
done
Checking in mozilla/directory/c-sdk/ldap/libraries/libldap/Makefile.in;
/cvsroot/mozilla/directory/c-sdk/ldap/libraries/libldap/Makefile.in,v  <--  Make
file.in
new revision: 5.18; previous revision: 5.17
done
Checking in mozilla/directory/c-sdk/ldap/libraries/libssldap/Makefile.in;
/cvsroot/mozilla/directory/c-sdk/ldap/libraries/libssldap/Makefile.in,v  <--  Ma
kefile.in
new revision: 5.12; previous revision: 5.11
done
Removing mozilla/directory/c-sdk/ldap/libraries/libutil/Makefile;
/cvsroot/mozilla/directory/c-sdk/ldap/libraries/libutil/Makefile,v  <--  Makefil
e
new revision: delete; previous revision: 5.2
done
RCS file: /cvsroot/mozilla/directory/c-sdk/ldap/libraries/libutil/Makefile.in,v
done
Checking in mozilla/directory/c-sdk/ldap/libraries/libutil/Makefile.in;
/cvsroot/mozilla/directory/c-sdk/ldap/libraries/libutil/Makefile.in,v  <--  Make
file.in
initial revision: 5.1
done
Checking in mozilla/directory/c-sdk/ldap/libraries/libutil/getopt.c;
/cvsroot/mozilla/directory/c-sdk/ldap/libraries/libutil/getopt.c,v  <--  getopt.c
new revision: 5.3; previous revision: 5.2
done

Comment 19

[Rich Megginson]

Correct.  internal_sortcmp_proc does not need to be declared LDAP_CALL because it is static.  I think LDAP_CALL is used for externally visible functions.  With this fix, perldap builds and runs on Windows with activestate perl + cygwin + the free MSVC compiler and SDK.

Mark, please reassign this to me and I will close it out.

Checking in API.xs;
/cvsroot/mozilla/directory/perldap/API.xs,v  <--  API.xs
new revision: 1.18.2.7; previous revision: 1.18.2.6
done