299305 - New GSSAPI detection in configure breaks things

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Mark Mentovai]

Following bug 295109, the way GSS headers are located has changed.  In the past,
configure would check for the headers and a GSS library, and upon finding them,
it would set the proper variables.  Subsequent attempts to find GSS got wrapped
in tests based on the GSSAPI_LIBS variable.

Now that GSS libraries are loaded at runtime, GSSAPI_LIBS is no longer
necessary.  The configure code was changed to use GSSAPI_INCLUDES to determine
whether or not a suitable header had been found.

The problem here is that GSSAPI_INCLUDES can be empty even after headers are
found.  This is the case if the headeres are found in system directories.  In
this case, even after finding headers, configure continues searching.

This breaks Mac builds on 10.4 that use SDK 10.2.8 and probably 10.3.9. 
Although GSS headers are found in the SDK, configure doesn't realize, and pulls
in the headers from /usr/include.  Mixing the root /usr/include with an SDK is
never a good idea.  This problem was initially fixed in bug 292530, but hey,
what Mac build problem wasn't?

Comment 1

[Mark Mentovai]

Attachment: GSS header detector fix

Use a separate _GSSAPI_FOUND variable to track whether or not GSS headers were
located.

Also: I'm pulling out the last remaining traces of GSSAPI_LIBS in this patch.

Also also: CPPFLAGS was being set in a certain code path only to be restored
shortly thereafter without being used.	Fixing.

The patch was hand-edited to remove irrelevant hunks.  Line numbers are
probably off, context is still OK, should be no factor.

This really should make 1.8b3, the leading bug causes a lot of bustage.  The
tinderboxen only escape burning because none are yet on 10.4.

Comment 2

[Mark Mentovai]

Attachment: Mac GSSAPI fixes, v2

GSSAPI_INCLUDES was being used in a test in a makefile, too.

But wait!  That's not all!  Order in the next ten minutes and...

nsNegotiateAuthGSSAPI.cpp has a call to KLCacheHasValidTickets, a Mac-only
function in the GSS library.  Now that -lgssapi_krb5 is gone, that caused a
major problem on the Mac because the call wasn't wrapped in pointer magic. 
Here, I'm supplying the missing magic.	I didn't use the gssFunPtr array
because the array members are accessed by index, and maintaining those indices
will become a nightmare if the same kind of thing ever becomes needed for other
platform-specific functions.

I didn't test this because I don't have any Kerberized web logins.  I can't
help but think that the leading patch wasn't tested on the Mac, either.

Comment 3

[Mark Mentovai]

a=#developers for Mac bustage fix

Checking in configure.in;
/cvsroot/mozilla/configure.in,v  <--  configure.in
new revision: 1.1483; previous revision: 1.1482
done
Checking in config/autoconf.mk.in;
/cvsroot/mozilla/config/autoconf.mk.in,v  <--  autoconf.mk.in
new revision: 3.356; previous revision: 3.355
done
Checking in extensions/negotiateauth/Makefile.in;
/cvsroot/mozilla/extensions/negotiateauth/Makefile.in,v  <--  Makefile.in
new revision: 1.8; previous revision: 1.7
done
Checking in extensions/negotiateauth/nsNegotiateAuthGSSAPI.cpp;
/cvsroot/mozilla/extensions/negotiateauth/nsNegotiateAuthGSSAPI.cpp,v  <-- 
nsNegotiateAuthGSSAPI.cpp
new revision: 1.7; previous revision: 1.6
done

Comment 4

[Mark Mentovai]

*** Bug 299284 has been marked as a duplicate of this bug. ***

Comment 5

[Christopher Nebergall]

Negotiateauth on Mac OS X broken Bug 301030

Comment 6

[Mark Mentovai]

Declared KLCacheHasValidTicketsPtr incorrectly, fixing in bug 301030.