245467 - nsNegotiateAuthGSSAPI.h compile error using Heimdal 0.6

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[ib]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a1) Gecko/20040520
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a1) Gecko/20040520

Compiling from source fails with the following error at nsNegotiateAuthFactory.cpp:

In file included from nsNegotiateAuthFactory.cpp:44:
nsNegotiateAuthGSSAPI.h:82: Fehler: »gss_OID« bezeichnet keinen Typ
nsNegotiateAuthGSSAPI.h:85: Fehler: »gss_ctx_id_t« bezeichnet keinen Typ
nsNegotiateAuthGSSAPI.h:86: Fehler: »gss_OID« bezeichnet keinen Typ

This happens on a SuSE 9.0 box with its included version of heimdal 0.6.

Reproducible: Always
Steps to Reproduce:
1.
2.
3.

Comment 1

[Daniel Veditz [:dveditz]]

I'll count bug 256888 as confirmation. That one used the newer heimdal 0.6.1rc3
on SUSE 9.1

From that bug:
--- CUT ---
c++ -o nsNegotiateAuthFactory.o -c -DOSTYPE=\"Linux2.6.5-7\" -DOSARCH=\"Linux\"
-DHAVE_DEPENDENT_LIBS -DUSE_GSSAPI -I/usr/include/heimdal 
-I../../dist/include/xpcom -I../../dist/include/string
-I../../dist/include/necko -I../../dist/include/pref
-I../../dist/include/negotiateauth -I../../dist/include
-I/home/beaufour/Documents/Mozilla/mozilla/dist/include/nspr     -I.
-I/usr/X11R6/include   -fPIC  -I/usr/X11R6/include -fno-rtti -fno-exceptions
-Wall -Wconversion-Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth
-Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -pedantic
-fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -O  -I/usr/X11R6/include
-DMOZILLA_CLIENT -include ../../mozilla-config.h
-Wp,-MD,.deps/nsNegotiateAuthFactory.pp nsNegotiateAuthFactory.cpp
In file included from nsNegotiateAuthFactory.cpp:64:
nsNegotiateAuthGSSAPI.h:74: error: syntax error before `)' token
nsNegotiateAuthGSSAPI.h:76: error: semicolon missing after declaration of `
   nsNegotiateAuth'
nsNegotiateAuthGSSAPI.h:76: error: syntax error before `:' token
nsNegotiateAuthGSSAPI.h:78: error: 'gss_OID' is used as a type, but is not
   defined as a type.
nsNegotiateAuthGSSAPI.h:81: error: syntax error before `}' token
gmake[1]: *** [nsNegotiateAuthFactory.o] Error 1
gmake[1]: Leaving directory
`/home/beaufour/Documents/Mozilla/mozilla/extensions/negotiateauth'
make: *** [all] Error 2

Comment 2

[Daniel Veditz [:dveditz]]

*** Bug 256888 has been marked as a duplicate of this bug. ***

Comment 3

[Darin Fisher]

Attachment: v1 patch

With this patch to configure.in I am able to build using Heimdal 0.6.2 or the
MIT Kerberos 5 package installed on my Fedora Core 2 system.

The problem I had with the current code in the trunk is that it would pick up
some headers from MIT and others from Heimdal.	That led to great confusion for
the compiler while trying to build the source.	I didn't get the same errors
shown in this bug report, but I got similar ones.

I'm hoping that this patch will fix the problems people are seeing, so please
try it out and let me know if it does the trick.  Thanks!

Comment 4

[Darin Fisher]

Would be good if folks could try this patch on other systems, like AIX and
Solaris, to make sure it doesn't break things there.  Thanks!

Comment 5

[Kyle Yuan]

It will be great if Darin or someone else can also provide a patch against
configure (for testing). My autoconf 2.13 can't generate the correct configure
for me.

Comment 6

[Darin Fisher]

Attachment: configure.diff.gz for v1 patch

Comment 7

[Philip K. Warren]

(In reply to comment #4)
> Would be good if folks could try this patch on other systems, like AIX and
> Solaris, to make sure it doesn't break things there.  Thanks!

Darin: I have tested this patch with AIX's kerberos, and everything is working
well. Thanks for the heads up!

Comment 8

[Kyle Yuan]

Still doesn't work on SuSE 9.1 with Heimdal 0.6.1. configure outputs:

<snip>
checking for gssapi.h... no
checking for gssapi_generic.h... no
checking for gssapi/gssapi.h... no
checking for gssapi/gssapi_generic.h... no
checking for gss_init_sec_context in -lgss... no
checking for gss_init_sec_context in -lgssapi... yes
configure: warning: GSS_C_NT_HOSTBASED_SERVICE not found
<snip>

The error is no gssapi.h included in nsNegotiateAuthGSSAPI.h.

Heimdal installs gssapi.h into /usr/include/heimdal. I guess that's why
configure can't find gssapi.h.

Comment 9

[Darin Fisher]

thanks for the configure output kyle.  did you use the --enable-gssapi configure
argument?

Comment 10

[Kyle Yuan]

Is there any option called "--enable-gssapi". I thought there are only
--with-gssapi and --without-gssapi. I used to try --with-gssapi, but failed. On
SuSE 9.1, gssapi.h is in /usr/include/heimdal, libgssapi.so is in /usr/lib, I
don't know which dir should be given to --with-gssapi.

Comment 11

[Darin Fisher]

> Is there any option called "--enable-gssapi". I thought there are only
> --with-gssapi and --without-gssapi. I used to try --with-gssapi, but failed. On

whoops.. yeah, i meant --with-gssapi.  i don't think the current configure.in is
suitable for the SuSE configuration.  we need to make sure that krb5-config is
used, but unfortunately "krb5-config --libs gssapi" outputs more libraries than
we need, and as a result we limit mozilla's portability.  see bug 242029.  i'm
not sure how to deal with this problem other than to special case MIT kerberos
maybe.  ugh! :-(

Comment 12

[Wyllys Ingersoll]

I made the original configure.in changes and I had tried to accomodate the 3
most common GSSAPI configurations - MIT, Heimdal, and Solaris.   I think its ok
in this case to "special case" Heimdal or MIT in order to find the headers.

The other option is to add new config options like "--with-gssapi-headers=<dir>"
and "-with-gssapi-libs=<dir>", but that gets confusing.

Only Heimdal includes the "krb5-config" script, so you cannot expect that to be
present on all systems.   

Im also concerned that whatever fix is made, that the check for native Solaris
GSSAPI is not skipped or made to take lower precedence when building on a
Solaris system (especially Solaris 9 or 10).

Comment 13

[Darin Fisher]

> I think its ok
> in this case to "special case" Heimdal or MIT in order to find the headers.

The problem it seems is that Heimdal is installed in a very odd way under SuSE.
 If you download the Heimdal source yourself it'll install itself in a much more
consistent way (i.e., $prefix/{include,lib}) and hence the v1 patch in this bug
would work great.  Unfortunately, it looks like we cannot assume that the
headers and libraries will appear under a common prefix.


> The other option is to add new config options like 
> "--with-gssapi-headers=<dir>"
> and "-with-gssapi-libs=<dir>", but that gets confusing.

yeah, i was just about to give in and implement those :-(


> Only Heimdal includes the "krb5-config" script, so you cannot expect that to be
> present on all systems.   

Fedora Core 2 (and older versions of Redhat Linux) includes a krb5-config script
(presumably part of MIT Krb5), and if we use it to build Mozilla, then our
libnegotiateauth.so will suffer from bug 242029.


> Im also concerned that whatever fix is made, that the check for native Solaris
> GSSAPI is not skipped or made to take lower precedence when building on a
> Solaris system (especially Solaris 9 or 10).

Ok.

Comment 14

[Darin Fisher]

*** Bug 264905 has been marked as a duplicate of this bug. ***

Comment 15

[Martin Mokrejs]

OK, curent mozilla cvs with standard heimdal installation in
/usr/heimdal/{bin,include,lib,libexec,man} gives me:

checking for gssapi.h... yes
checking for gss_init_sec_context in -lgss... no
checking for gss_init_sec_context... no
checking for gss_init_sec_context in -lgssapi... yes
checking for gssapi_generic.h... no
checking for gssapi/gssapi_generic.h... no


configure:13309: checking for gssapi.h
configure:13319: gcc -E   -I/usr/X11R6/include -I/usr/heimdal/include conftest.c
>/dev/null 2>conftest.out
configure:13390: checking for gss_init_sec_context in -lgss
configure:13409: gcc -o conftest  -pthread   -I/usr/X11R6/include
-I/usr/heimdal/include  -L/usr/heimdal/lib conftest.c -lgss
  -ldl -lm  1>&5
/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/../../../../i686-pc-linux-gnu/bin/ld:
cannot find -lgss
collect2: ld returned 1 exit status
configure: failed program was:
#line 13398 "configure"
#include "confdefs.h"
/* Override any gcc2 internal prototype to avoid an error.  */
/* We use char because int might match the return type of a gcc2
    builtin and then its argument prototype would still apply.  */
char gss_init_sec_context();

int main() {
gss_init_sec_context()
; return 0; }
configure:13434: checking for gss_init_sec_context
configure:13462: gcc -o conftest  -pthread   -I/usr/X11R6/include
-I/usr/heimdal/include  -L/usr/heimdal/lib conftest.c -ldl 
-lm  -L/usr/heimdal/lib -lgssapi_krb5 1>&5
/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/../../../../i686-pc-linux-gnu/bin/ld:
cannot find -lgssapi_krb5
collect2: ld returned 1 exit status
configure: failed program was:
#line 13439 "configure"
#include "confdefs.h"
/* System header to define __stub macros and hopefully few prototypes,
    which can conflict with char gss_init_sec_context(); below.  */
#include <assert.h>
/* Override any gcc2 internal prototype to avoid an error.  */
/* We use char because int might match the return type of a gcc2
    builtin and then its argument prototype would still apply.  */
char gss_init_sec_context();

int main() {

/* The GNU C library defines this for functions which it implements
    to always fail with ENOSYS.  Some functions are actually named
    something starting with __ and the normal name is an alias.  */
#if defined (__stub_gss_init_sec_context) || defined (__stub___gss_init_sec_context)
choke me
#else
gss_init_sec_context();
#endif

; return 0; }
configure:13492: checking for gss_init_sec_context in -lgssapi
configure:13511: gcc -o conftest  -pthread   -I/usr/X11R6/include
-I/usr/heimdal/include  -L/usr/heimdal/lib conftest.c -lgss
api  -ldl -lm  -L/usr/heimdal/lib -lgssapi -lkrb5 -lasn1 -lcrypto -lroken
-lcrypt -ldb -lresolv 1>&5
configure:13547: checking for gssapi_generic.h
configure:13557: gcc -E   -I/usr/X11R6/include -I/usr/heimdal/include 
conftest.c >/dev/null 2>conftest.out
configure:13553:28: gssapi_generic.h: No such file or directory
configure: failed program was:
#line 13552 "configure"
#include "confdefs.h"
#include <gssapi_generic.h>
configure:13587: checking for gssapi/gssapi_generic.h
configure:13597: gcc -E   -I/usr/X11R6/include -I/usr/heimdal/include 
conftest.c >/dev/null 2>conftest.out
configure:13593:35: gssapi/gssapi_generic.h: No such file or directory
configure: failed program was:
#line 13592 "configure"
#include "confdefs.h"
#include <gssapi/gssapi_generic.h>
configure:13636: gcc -c  -pthread   -I/usr/X11R6/include -I/usr/heimdal/include
 conftest.c 1>&5


# ls -la /usr/heimdal/include
total 297
drwxr-xr-x  4 root root  1040 Oct  6 14:25 .
drwxr-xr-x  9 root root   216 Sep 15 14:06 ..
-rw-r--r--  1 root root  1137 Oct  6 14:25 asn1_err.h
-rw-r--r--  1 root root  1867 Oct  6 14:25 base64.h
-rw-r--r--  1 root root  5895 Oct  6 14:25 der.h
-rw-r--r--  1 root root  1296 Oct  6 14:25 editline.h
-rw-r--r--  1 root root  2285 Oct  6 14:25 fnmatch.h
-rw-r--r--  1 root root  2808 Oct  6 14:25 getarg.h
-rw-r--r--  1 root root  3800 Oct  6 14:25 glob.h
-rw-r--r--  1 root root 25424 Oct  6 14:25 gssapi.h
-rw-r--r--  1 root root   464 Oct  6 14:25 hdb-private.h
-rw-r--r--  1 root root  3236 Oct  6 14:25 hdb-protos.h
-rw-r--r--  1 root root  3558 Oct  6 14:25 hdb.h
-rw-r--r--  1 root root  5523 Oct  6 14:25 hdb_asn1.h
-rw-r--r--  1 root root  1228 Oct  6 14:25 hdb_err.h
-rw-r--r--  1 root root  1788 Oct  6 14:25 heim_err.h
-rw-r--r--  1 root root  1020 Oct  6 14:25 k524_err.h
drwxr-xr-x  2 root root   200 Oct  6 14:25 kadm5
-rw-r--r--  1 root root  6755 Oct  6 14:25 kafs.h
-rw-r--r--  1 root root  1830 Oct  6 14:25 krb5-private.h
-rw-r--r--  1 root root 60804 Oct  6 14:25 krb5-protos.h
-rw-r--r--  1 root root   403 Oct  6 14:25 krb5-types.h
-rw-r--r--  1 root root 20861 Oct  6 14:25 krb5.h
-rw-r--r--  1 root root 35715 Oct  6 14:25 krb5_asn1.h
-rw-r--r--  1 root root 10876 Oct  6 14:25 krb5_err.h
-rw-r--r--  1 root root  3531 Oct  6 14:25 otp.h
-rw-r--r--  1 root root  1977 Oct  6 14:25 parse_bytes.h
-rw-r--r--  1 root root  2003 Oct  6 14:25 parse_time.h
-rw-r--r--  1 root root  2454 Oct  6 14:25 parse_units.h
-rw-r--r--  1 root root  4320 Oct  6 14:25 resolve.h
-rw-r--r--  1 root root  7367 Oct  6 14:25 roken-common.h
-rw-r--r--  1 root root  5903 Oct  6 14:25 roken.h
-rw-r--r--  1 root root  2134 Oct  6 14:25 rtbl.h
-rw-r--r--  1 root root  2239 Oct  6 14:25 sl.h
drwxr-xr-x  2 root root    72 Oct  6 14:25 ss
-rw-r--r--  1 root root  3417 Oct  6 14:25 vis.h
-rw-r--r--  1 root root  2006 Oct  6 14:25 xdbm.h
# ls -la /usr/heimdal/lib    
total 22287
drwxr-xr-x  2 root root     768 Oct  6 14:25 .
drwxr-xr-x  9 root root     216 Sep 15 14:06 ..
-rw-r--r--  1 root root 1998542 Oct  6 14:25 libasn1.a
-rwxr-xr-x  1 root root     759 Oct  6 14:25 libasn1.la
-rw-r--r--  1 root root  251264 Oct  6 14:25 libeditline.a
-rwxr-xr-x  1 root root     761 Oct  6 14:25 libeditline.la
-rw-r--r--  1 root root 3465368 Oct  6 14:25 libgssapi.a
-rwxr-xr-x  1 root root     872 Oct  6 14:25 libgssapi.la
-rw-r--r--  1 root root  956418 Oct  6 14:25 libhdb.a
-rwxr-xr-x  1 root root     863 Oct  6 14:25 libhdb.la
-rw-r--r--  1 root root 1423388 Oct  6 14:25 libkadm5clnt.a
-rwxr-xr-x  1 root root     908 Oct  6 14:25 libkadm5clnt.la
-rw-r--r--  1 root root 2183332 Oct  6 14:25 libkadm5srv.a
-rwxr-xr-x  1 root root     905 Oct  6 14:25 libkadm5srv.la
-rw-r--r--  1 root root  466236 Oct  6 14:25 libkafs.a
-rwxr-xr-x  1 root root     866 Oct  6 14:25 libkafs.la
-rw-r--r--  1 root root 7780140 Oct  6 14:25 libkrb5.a
-rwxr-xr-x  1 root root     839 Oct  6 14:25 libkrb5.la
-rw-r--r--  1 root root  939476 Oct  6 14:25 libotp.a
-rwxr-xr-x  1 root root     797 Oct  6 14:25 libotp.la
-rw-r--r--  1 root root 2663802 Oct  6 14:25 libroken.a
-rwxr-xr-x  1 root root     758 Oct  6 14:25 libroken.la
-rw-r--r--  1 root root  271920 Oct  6 14:25 libsl.a
-rwxr-xr-x  1 root root     785 Oct  6 14:25 libsl.la
-rw-r--r--  1 root root  324810 Oct  6 14:25 libss.a
-rwxr-xr-x  1 root root     795 Oct  6 14:25 libss.la
#

Briefly, there is no libgssapi_kr5.a but libgssapi.a. If this test is for MIT
Kerberos, then proper test for heimdal is missing.

Comment 16

[Martin Mokrejs]

I had a look at that library not resolving der_get_length (see comment #14).

gmake[1]: Entering directory `/scratch/mozilla/extensions/negotiateauth'
Creating .deps
echo "#define DEPENDENT_LIBS \"libxpcom.so\", \"libplds4.so\", \"libplc4.so\",
\"libnspr4.so\", \"libpthread.so\", \"libdl.so\", \"libgssapi.so\",
\"libkrb5.so\", \"libasn1.so\", \"libcrypto.so\", \"libroken.so\",
\"libcrypt.so\", \"libdb.so\", \"libresolv.so\", " > dependentLibs.h

Wow! But by default, heimdal installs only static libs. It's said because of
security. ;) What's this then? I don't have libgssapi.so.

Comment 17

[Martin Mokrejs]

The problem described in comment #14 can be fixed by:

# cd /scratch/mozilla/extensions/negotiateauth
# g++ -I/usr/X11R6/include -fno-rtti -fno-exceptions -Wall -Wconversion
-Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy
-Wno-non-virtual-dtor -Wno-long-long -pedantic -fshort-wchar -pthread -pipe 
-DDEBUG -D_DEBUG -DDEBUG_root -DTRACING -g -fno-inline -fPIC -shared -Wl,-h
-Wl,libnegotiateauth.so -o libnegotiateauth.so  nsNegotiateAuthFactory.o
nsHttpNegotiateAuth.o nsNegotiateAuthGSSAPI.o -L../../dist/bin -lxpcom 
-L../../dist/bin -L/scratch/mozilla/dist/lib -lplds4 -lplc4 -lnspr4 -lpthread
-ldl /usr/heimdal/lib/libgssapi.a /usr/heimdal/lib/libkrb5.a
/usr/heimdal/lib/libasn1.a /usr/lib/libcrypto.a /usr/heimdal/lib/libroken.a
-lcrypt -ldb -lresolv  -Wl,-Bsymbolic -ldl -lm
# cp libnegotiateauth.so /usr/local/lib/mozilla-1.8a5/components/libnegotiateauth.so


Simply, by including *.a libs from heimdal on the link line. Please note that
libcrypto is from openssl and nout heimdal. I believe
# /usr/heimdal/bin/krb5-config --cflags
-I/usr/heimdal/include
# /usr/heimdal/bin/krb5-config --libs
-L/usr/heimdal/lib -lkrb5 -lasn1 -lcrypto -lroken -lcrypt -ldb -lresolv
#
would print also full path to openssl lib and headers if they would be present
in non-standard location at the time heimdal has been compiled.



This would mean configure actually did it's job fine, only
mozilla/extensions/negotiateauth/ (Makefile?) has wrong expectations.

Comment 18

[Iain MacDonnell]

This is still quite broken on SuSE (SLES9). Not only is there no easy way to
find the headers in /usr/include/heimdal, krb5-config is only looked-for in
GSSAPI_DIR, if specified. On SLES9, krb5-config is in /usr/bin

The best kludge I've come up with is:

# ls -l /usr/heimdal/
total 8
drwxr-xr-x   2 root root 4096 Nov 29 16:10 .
drwxr-xr-x  21 root root 4096 Nov 29 15:52 ..
lrwxrwxrwx   1 root root    6 Nov 29 16:10 bin -> ../bin
lrwxrwxrwx   1 root root   18 Nov 29 15:52 include -> ../include/heimdal
lrwxrwxrwx   1 root root    6 Nov 29 15:53 lib -> ../lib
# 

and --with-gssapi=/usr/heimdal

Comment 19

[Darin Fisher]

Downgrading to Severity:minor since this impacts relatively few users.  I doubt
I will have time to improve the build fu for this in time for Gecko 1.8. 
Patches welcome.  Help wanted.  Yadda yadda.

Comment 20

[Martin Mokrejs]

With current cvs I get (it must be related to some changes, say introduced
during last month):

c++ -o nsNegotiateAuthFactory.o -c -I../../dist/include/system_wrappers -include
../../config/gcc_hidden.h -DMOZILLA_INTERNAL_API -DOSTYPE=\"Linux2.6\"
-DOSARCH=\"Linux\" -DBUILD_ID=0000000000 -DHAVE_DEPENDENT_LIBS -DUSE_GSSAPI
-I/usr/heimdal/include  -I../../dist/include/xpcom -I../../dist/include/string
-I../../dist/include/necko -I../../dist/include/pref
-I../../dist/include/negotiateauth -I../../dist/include
-I../../dist/include/nspr    -I.    -fPIC   -fno-rtti -fno-exceptions -Wall
-Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth
-Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -pedantic -g3 -O0
-ggdb -fshort-wchar -pthread -pipe  -DDEBUG -D_DEBUG -DDEBUG_root -DTRACING -g
-fno-inline   -DMOZILLA_CLIENT -include ../../mozilla-config.h
-Wp,-MD,.deps/nsNegotiateAuthFactory.pp nsNegotiateAuthFactory.cpp
In file included from nsNegotiateAuthFactory.cpp:64:
nsNegotiateAuthGSSAPI.h:74: error: `gss_OID' does not name a type
nsNegotiateAuthGSSAPI.h:77: error: `gss_ctx_id_t' does not name a type
nsNegotiateAuthGSSAPI.h:77: error: extra semicolon
nsNegotiateAuthGSSAPI.h:78: error: `gss_OID' does not name a type
nsNegotiateAuthGSSAPI.h:78: error: extra semicolon
gmake[3]: *** [nsNegotiateAuthFactory.o] Error 1
gmake[3]: Leaving directory `/scratch/mozilla/extensions/negotiateauth'


Am on linux with heimdal. 
MOZ_DEBUG_FLAGS="-g3 -O0 -ggdb" CFLAGS="-g3 -O0 -ggdb" CXXFLAGS="-g3 -O0 -ggdb"
./configure --enable-debug --disable-optimize --enable-debug-modules=all
--enable-debugger-info-modules --enable-detect-webshell-leaks --enable-svg
--enable-svg-renderer-libart --enable-image-decoders=all --with-qtdir=/usr/qt/3
--enable-application=suite --disable-freetype2 --enable-jprof
--enable-default-toolkit=gtk2 --enable-xft

Comment 21

[Simon Wilkinson]

Negotiateauth has now been changed to use an internal version of the gssapi.h
header file, so any compilation problems with gssapi.h should be a thing of the
past. The changes were made as part of the patch for bug #280792.