124244 - switch builds to use LDAP C SDK 5.0

Status: VERIFIED FIXED

(SeaMonkey :: Build Config, defect, P1)

Description

[Dan Mosedale (:dmosedale, :dmose)]

We need this for SSL-LDAP, among other things.

Comment 1

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozilla with LDAP C SDK v5.0

Using this patch requires first applying the patch in 122115, and then this
one.  After you apply this one, you _must_ run autoconf to regenerate configure
and then reconfigure.

Comment 2

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozilla with LDAP C SDK v5.0

Forgot client.mk changes.  Also my previous description of how to use was
wrong, a bit of bootstrapping will be required, since the 5.0 sources have to
be pulled from cvs before one can apply the patch in bug 122115.

Comment 3

[Jon Granrose]

this are simple changes are far as to the automation, but we're pulling a new
ldap sdk branch.  How big of a change is the new SDK? Do we need to carpool this
change?

Comment 4

[Dan Mosedale (:dmosedale, :dmose)]

This is still a short ways off.  It's a non-trivial change, overall. 
Conceivably, things may break, especially for odd platforms.  However, it's not
clear to me that a carpool actually buys us anything.  Only a few people check
in code which uses the LDAP functionality, and so it's quite unlikely that any
breakage that happens here would be easier to debug because fewer people were
checking in.

Comment 5

[Jon Granrose]

we should do it as a carpool anyway with builds after the change, just so we can
isolate any problems to the carpool rather than wondering if it was the new sdk
or some other change that is causing a problem.  Also, by putting it on the
branch landing page and sending out the notifications to mozilla-builds and
mozilla-seamonkey, we can be sure people know about it ahead of time.

Please put an entry on the branch landing page so we have a way of tracking the
progress and can estimate the landing time.  Also, keep in mind 0.9.9 freezes in
one week if you want to get this in before then.

Comment 6

[scottputterman]

Moving to 1.0 because that's when LDAP over SSL is scheduled, but if it gets in
earlier that works, too.

Comment 7

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozila with v5 C SDK, v3

OK, this patch now includes the changes necessary to work on windows as well as
unix.

Comment 8

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozila with v5 C SDK, v4

Whoops, the last patch contained _only_ the windows changes.  This patch
contains both windows and unix changes.

Comment 9

[(not reading, please use [e-mail] instead)]

heads up to dmose, I think that there might be some build order issues.

I did a clobber_all build_all (with these patches) and my build fails.

c:\moztools\bin\gmake.exe[5]: Entering directory `c:/builds/tip/mozilla/director
y/c-sdk/WIN32_D.OBJ/ldap/libraries/libssldap'
cl -Foclientinit.obj -c     -W3 -nologo -GF -Gy -MDd -Od -Z7   -UNDEBUG  -DMOZIL
LA_CLIENT=1 -DDEBUG=1 -DDEBUG_Administrator=1 -DXP_PC=1 -DWIN32=1 -D_WINDOWS=1 -
D_DEBUG=1 -DWIN95=1 -D_PR_GLOBAL_THREADS_ONLY=1 -D_X86_=1  -DFORCE_PR_LOG -DUSE_
WAITPID -DNEEDPROTOS -DLDAP_DEBUG   -DNET_SSL  -DNO_LIBLCACHE -DLDAP_REFERRALS
   -DNS_DOMESTIC  -Ic:/builds/tip/mozilla/dist/public/ldap -I../../../../ldap/in
clude -Ic:/builds/tip/mozilla/dist/./include -Ic:/builds/tip/mozilla/dist/includ
e -Ic:/builds/tip/mozilla/dist/public/security -Ic:/builds/tip/mozilla/dist/publ
ic/nss -Ic:/builds/tip/mozilla/dist/include/nspr  ../../../../ldap/libraries/lib
ssldap/clientinit.c
clientinit.c
../../../../ldap/libraries/libssldap/clientinit.c(37) : fatal error C1083: Canno
t open include file: 'cert.h': No such file or directory
c:\moztools\bin\gmake.exe[5]: *** [clientinit.obj] Error 2
c:\moztools\bin\gmake.exe[5]: Leaving directory `c:/builds/tip/mozilla/directory
/c-sdk/WIN32_D.OBJ/ldap/libraries/libssldap'
c:\moztools\bin\gmake.exe[4]: *** [install] Error 2
c:\moztools\bin\gmake.exe[4]: Leaving directory `c:/builds/tip/mozilla/directory
/c-sdk/WIN32_D.OBJ/ldap/libraries'
c:\moztools\bin\gmake.exe[3]: *** [install] Error 2
c:\moztools\bin\gmake.exe[3]: Leaving directory `c:/builds/tip/mozilla/directory
/c-sdk/WIN32_D.OBJ/ldap'
c:\moztools\bin\gmake.exe[2]: *** [install] Error 2
c:\moztools\bin\gmake.exe[2]: Leaving directory `c:/builds/tip/mozilla/directory
/c-sdk/WIN32_D.OBJ'
c:\moztools\bin\gmake.exe[1]: *** [all] Error 2
c:\moztools\bin\gmake.exe[1]: Leaving directory `c:/builds/tip/mozilla/directory
/c-sdk/WIN32_D.OBJ'
gmake: *** [build_all] Error 2
NMAKE : fatal error U1077: 'gmake' : return code '0x2'
Stop.

the build works fine if security is built first.

but, for mozilla I don't think we need to build libssldap.  perhaps things are 
ok as they are, we just need to selectively build only the libraries we really 
need.

Comment 10

[(not reading, please use [e-mail] instead)]

this might be enough to fix it for now:

-build_all: build_nspr build_ldapcsdk build_seamonkey 
+build_all: build_nspr build_seamonkey build_ldapcsdk

trying that now.

Comment 11

[(not reading, please use [e-mail] instead)]

yes,

build_all: build_nspr build_seamonkey build_ldapcsdk

is the way to go.

cc peterv, in case this information affects his mac build work for LDAP SDK 5.0

Comment 12

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to allow the LDAP C SDK to build without NSS

So after talking to cls and sspitzer, it's been determined that if the LDAP C
SDK depends on NSS, some non-trivial changes will have to be made to both the
nmake and autoconf buildsystems.  As it turns out, though, for the browser, we
don't libssldap, because we're doing all of our crypto using PSM.  So this
patch adds
an option to configure to allow the caller to build the LDAP C SDK without NSS,
and thus it doesn't build libssldap at all.

Comment 13

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozilla with v5 C SDK, v5

In combination with the LDAP C SDK patch here, this should fix the build order
problems.  Additionally, it uses libprldap to install NSPR for low-level
functions, and it updates the packager files for windows and unix.

Comment 14

[hacker formerly known as [e-mail]]

Comment on attachment 71987 [details] [diff] [review]
patch to allow the LDAP C SDK to build without NSS

r=cls

Comment 15

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 71987 [details] [diff] [review]
patch to allow the LDAP C SDK to build without NSS

Fix checked in.

Comment 16

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozilla with v5 CSK, v6

Fixed a forgotten makefile.win, and changed a reference from bash to sh, on
cls' recommendation.

Comment 17

[Jim Dunn]

Attachment: Patch for HP-UX

Basically the Makefile had an ifdef AIX that was the same as the ifdef Linux 
and HP-UX needed the same.  So instead of continuing the ifdef <OS_ARCH>
I used the ifneq $filter technigue to combine several OS_ARCH triggers
that do the same thing into a single if

Comment 18

[Peter Van der Beken [:peterv]]

Attachment: patch to v5 C SDK for Mac

Includes the changes for the Mozilla Mac build scripts.

Comment 19

[(not reading, please use [e-mail] instead)]

Comment on attachment 72158 [details] [diff] [review]
patch to pull and build Mozilla with v5 CSK, v6

r=sspitzer

how did you confirm that we were indeed using NSPR?

also, did you test both autocomplete and LDAP addressbook searches?

Comment 20

[hacker formerly known as [e-mail]]

Comment on attachment 73207 [details] [diff] [review]
Patch for HP-UX

r=cls

Comment 21

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 73207 [details] [diff] [review]
Patch for HP-UX

jdunn's HP-UX patch checked in to ldapcsdk_branch_50.

Comment 22

[Mark Smith [:mcs]]

Great work on the patch for Mac. Just a couple of comments:

1) Why remove exit 0 from the replace.pl script? I guess it just is not needed?

2) The changes that remove the OS socket code are all #ifdef macintosh, but I
think they might be useful for other platforms someday too. Can we change the
code in liblber/io.c and libldap/os-ip.c to use a more general #ifdef such as
#ifdef NSLDAPI_AVOID_OS_SOCKETS or something similar? For now, we would only
define that on the Mac.

I am not qualified to review the XML CodeWarrior project file or the
mozilla/build/mac changes.

Comment 23

[Dan Mosedale (:dmosedale, :dmose)]

sspitzer asks:

> how did you confirm that we were indeed using NSPR?

The most important test is that on the Macintosh, the OS-native socket code is
completely stubbed out.  So the only possible way it could work at all is if the
XP code for enabling NSPR were working.  And it does work.  On Linux, I've been
able to use your SSL-LDAP proto-patch which depends on NSPR working as well.

> also, did you test both autocomplete and LDAP addressbook searches?

Yes.

Comment 24

[Dan Mosedale (:dmosedale, :dmose)]

mcs asks:

> 1) Why remove exit 0 from the replace.pl script? I guess it just is not
> needed?

I talked to peterv about this, and his answer was this: replace.pl is called
from the main MacPerl build script, and one of the vagaries of MacPerl is that
it calls other scripts directly in the same Perl script-space, rather than
forking a separate perl interpreter for the script.

> 2) The changes that remove the OS socket code are all #ifdef macintosh, but I
> think they might be useful for other platforms someday too. Can we change the
> code in liblber/io.c and libldap/os-ip.c to use a more general #ifdef such as
> #ifdef NSLDAPI_AVOID_OS_SOCKETS or something similar? For now, we would 
> only define that on the Mac.

I've got this change made locally on my Mac.  I will upload an updated patch
with it shortly.  I've chosen to define NSLDAPI_AVOID_OS_SOCKETS in
libraries/macintosh/ldap-macos-defs.h.

> I am not qualified to review the XML CodeWarrior project file or the
> mozilla/build/mac changes.

No problem; when I upload the latest mac changes, I'm gonna split peterv's patch
into two: one patch for the C SDK itself (including the XML changes, which I'll
find someone else to review), and one for the Mozilla build glue.

Comment 25

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to v5 C SDK for Mac, v2

OK, here's the updated C SDK patch.  It addresses mcs' request (actually I
ended up having to put the define of NSLDAPI_AVOID_OS_SOCKETS in ldap-macos.h).
 It also updates the MANIFEST to make the proper header file links to dist.

Comment 26

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to V5 C SDK for Mac, v3

Whoops... extraneous change included in the last patch.

Comment 27

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozilla on Mac with v5 C-SDK, v1

This is the build glue for Mozilla.

Comment 28

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozilla on Mac with b5 C-SDK, v2

Fixed a build ordering issue for clobber builds: need to ensure that
ldap-standard.h is generated before attempting to create a link to dist.

Comment 29

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Updated Mac project file for LDAP XPCOM SDK, v1

No longer build nsLDAPConnectionCallbacks.c, since the SDK now provides all the
necessary code.

Comment 30

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to V5 C-SDK for Mac, v4

Need to move ber_error_print prototype in lber-int.h outside of
NSLBERI_LBER_INT_FRIEND check so that it gets prototyped for all builds,
including non-DEBUG.

Comment 31

[Mark Smith [:mcs]]

r=mcs for the "V5 C-SDK for Mac, v4 patch", with one comment: I don't see why
the ber_error_print prototype in lber-int.h needs to be moved. The
NSLBERI_LBER_INT_FRIEND macro is always defined by ldap-int.h before it includes
lber-int.h.

Comment 32

[Dan Mosedale (:dmosedale, :dmose)]

mcs: right, that's the problem.  In the current code, ber_error_print only gets
prototyped inside an "ifndef NSLBERI_LBER_INT_FRIEND" clause.

Comment 33

[Mark Smith [:mcs]]

Dan: got it. I misread the ifndef as a ifdef (doh!) r=mcs, no complaints.

Comment 34

[Simon Fraser [no longer active]]

Comment on attachment 74281 [details] [diff] [review]
patch to pull and build Mozilla on Mac with b5 C-SDK, v2

>Index: mac/build_scripts/MozillaBuildList.pm
>===================================================================
>RCS file: /cvsroot/mozilla/build/mac/build_scripts/MozillaBuildList.pm,v
>retrieving revision 1.182
>diff -u -5 -r1.182 MozillaBuildList.pm
>--- mac/build_scripts/MozillaBuildList.pm	8 Mar 2002 22:33:40 -0000	1.182
>+++ mac/build_scripts/MozillaBuildList.pm	15 Mar 2002 04:53:42 -0000
>@@ -832,10 +832,30 @@
>     InstallFromManifest(":mozilla:extensions:xml-rpc:idl:MANIFEST_IDL",                "$distdirectory:idl:");
> 
>     #LDAP
>     if ($main::options{ldap})
>     {
>+        # Create the ldap-standard.h file from the template
>+        #
>+        my($template) = ":mozilla:directory:c-sdk:ldap:include:ldap-standard-tmpl.h";
>+        my($template_modtime) = GetFileModDate($template);
>+        my($header) = ":mozilla:directory:c-sdk:ldap:include:ldap-standard.h";
>+        my($header_modtime) = (-e $header ? GetFileModDate($header) : 0);
>+        if ($template_modtime > $header_modtime) {
>+            open(STDIN, "<$template");
>+            open(STDOUT, ">$header");
>+            @ARGV = ("LDAP_VENDOR_NAME=mozilla.org", "LDAP_VENDOR_VERSION=500");
>+            do ":mozilla:directory:c-sdk:ldap:build:replace.pl";
>+            if ($@) {
>+                close(STDOUT);
>+                close(STDIN);
>+                die "Error: can't create ldap-standard.h from ldap-standard-tmpl.h";
>+            }
>+            close(STDOUT);
>+            close(STDIN);
>+        }

How is this stuff done on other platforms? Can we share some perl XP here?

>         InstallFromManifest(":mozilla:directory:c-sdk:ldap:include:MANIFEST",      		"$distdirectory:directory:");
>         InstallFromManifest(":mozilla:directory:xpcom:base:public:MANIFEST",       		"$distdirectory:directory:");
>         InstallFromManifest(":mozilla:directory:xpcom:base:public:MANIFEST_IDL",   		"$distdirectory:idl:");
>         InstallFromManifest(":mozilla:xpfe:components:autocomplete:public:MANIFEST_IDL",	"$distdirectory:idl:");
>     }
>Index: mac/build_scripts/MozillaCheckoutList.txt
...
>-DirectorySDKSourceC,    LDAPCSDK_40_BRANCH
>+DirectorySDKSourceC,    ldapcsdk_branch_50

Fine.

Comment 35

[Simon Fraser [no longer active]]

Comment on attachment 74281 [details] [diff] [review]
patch to pull and build Mozilla on Mac with b5 C-SDK, v2

r=sfraser

Comment 36

[Simon Fraser [no longer active]]

Comment on attachment 74284 [details] [diff] [review]
Updated Mac project file for LDAP XPCOM SDK, v1

r=sfraser

Comment 37

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 74281 [details] [diff] [review]
patch to pull and build Mozilla on Mac with b5 C-SDK, v2

Mac C-SDK patch checked in on ldapcsdk_branch_50.

Comment 38

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 74281 [details] [diff] [review]
patch to pull and build Mozilla on Mac with b5 C-SDK, v2

Whoops, marked the wrong attachment obsolete.

Comment 39

[hacker formerly known as [e-mail]]

Comment on attachment 72158 [details] [diff] [review]
patch to pull and build Mozilla with v5 CSK, v6

Minor nit; we use 4 space indentation in configure.in, not 2.  Fix that &&
r=cls

Comment 40

[David :Bienvenu]

Comment on attachment 74281 [details] [diff] [review]
patch to pull and build Mozilla on Mac with b5 C-SDK, v2

sr=bienvenu

Comment 41

[David :Bienvenu]

Comment on attachment 74284 [details] [diff] [review]
Updated Mac project file for LDAP XPCOM SDK, v1

sr=bienvenu

Comment 42

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozilla with v5 C-SDK, v7

Updated to CVS tip & fixed cls' indentation issue.

Comment 43

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 76144 [details] [diff] [review]
patch to pull and build Mozilla with v5 C-SDK, v7

Carrying forward r=cls,sspitzer, sr=bienvenu.

Comment 44

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: patch to pull and build Mozilla with v5 C-SDK, v8

Fix more merge conflicts with the cvs tip.

Comment 45

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 76147 [details] [diff] [review]
patch to pull and build Mozilla with v5 C-SDK, v8

carrying forward r=sspitzer,cls; sr=bienvenu

Comment 46

[Jim Dunn]

I just noticed that we check in mozilla/configure for LCHOWN and if
found we set (there) HAVE_LCHOWN which shows up in config/autoconf.mk
as -DHAVE_LCHOWN.

However, in directory/c-sdk/config/nsinstall.c there is a
#define HAVE_LCHOWN
followed by an #if...  #undef HAVE_LCHOWN... #endif

The problem is that SOME compilers won't let you redefine a previously
defined <thing>.  I am not 100% sure of the solution because I am
not sure if ldap needs to remain a separate entity... but as a 
minimum can we add a 
#ifndef HAVE_LCHOWN
#define HAVE_LCHOWN
#endif 
in nsinstall.c?

Comment 47

[Dan Mosedale (:dmosedale, :dmose)]

jdunn: config/autoconf.mk is not used by the C-SDK.  It has it's own version in
directory/c-sdk/config/autoconf.mk.  So these shouldn't interfere with each other.

Comment 48

[Jim Dunn]

doh - I knew it was getting defined but guessed at the wrong place... 

It looks like directory/c-sdk/configure 
also checks for lchown  (-> HAVE_LCHOWN)
line 1849 of directory/c-sdk/configure.in

Seeing as how ldap5's configure is checking for lchown 
(and hence defining HAVE_LCHOWN) can we remove the 
Define & undef of HAVE_LCHOWN from nsinstall.c?

Comment 49

[Dan Mosedale (:dmosedale, :dmose)]

jdunn: I just checked in your original suggestion on the branch.  Right now we
need to support both autoconf and non-autoconf builds, so it's the best
compromise.  Thanks for the fix.

Comment 50

[Asa Dotzler [:asa]]

Comment on attachment 74281 [details] [diff] [review]
patch to pull and build Mozilla on Mac with b5 C-SDK, v2

a=asa (on behalf of drivers) for checkin to the 1.0 trunk

Comment 51

[Asa Dotzler [:asa]]

Comment on attachment 74284 [details] [diff] [review]
Updated Mac project file for LDAP XPCOM SDK, v1

a=asa (on behalf of drivers) for checkin to the 1.0 trunk

Comment 52

[Asa Dotzler [:asa]]

Comment on attachment 76147 [details] [diff] [review]
patch to pull and build Mozilla with v5 C-SDK, v8

a=asa (on behalf of drivers) for checkin to the 1.0 trunk

Comment 53

[Dan Mosedale (:dmosedale, :dmose)]

Patches checked in, with the tiny tweak that the checkin versions use the newly
minted stable ldapcsdk_50_client_branch, rather than the old development branch
like the patches here did.

Comment 54

[Michael Buckland]

Discussed in Mail News bug meeting. Decided to [ADT2} this bug.

Comment 55

[Aleksey Nogin]

Please see bug 126462 for compiler warnings that were introduced on the new branch.

Comment 56

[Paul Wyskoczka]

Verifying (rs)