242870 - Statically link libIDL, glib with xpidl on Windows

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, defect)

Description

[Braden]

Gecko SDK binaries ought to be self-sufficient; that is it ought to be the 
case that one can get the Gecko SDK and not have to fish for libIDL and glib 
DLLs. The current situation is worsened by the fact that Visal C++ 7.0 builds 
of these libraries don't play nicely with Visual C++ 7.1 builds of xpidl, and 
vice versa.

Comment 1

[Braden]

Attachment: nmake makefile to build glib-1.2.10 as a static library

Comment 2

[Braden]

Attachment: nmake makefile to build libIDL-0.6.8 as a static library

Comment 3

[Braden]

Attachment: glib-1.2.10 and libIDL-0.6.8 static library binaries (and public headers)

These binaries were built with Visual C++ 7.1. However, since they are static
libraries, I suspect they may work with previous compiler versions. Obviously
that should be tested; however, I'm not in a position to do so.

Also, note that the glib binary includes the glib library, and not any of its
accessory libraries (gthread, gmodule, gplugin). Mozilla does not appear to
require those; and not building them avoids the pthreads-win32 dependency.

Comment 4

[Braden]

Using these static libraries to build xpidl does not appear to require any 
changes to Mozilla's build system. Simply make them available (and make sure 
no crufty .dll or .exp files are hanging around), and it should Just Work.

Comment 5

[Braden]

Comment on attachment 148350 [details]
glib-1.2.10 and libIDL-0.6.8 static library binaries (and public headers)

leaf, could you test this against Visual C++ 6.0 and 7.0? If all goes well, I
propose distributing these binaries instead of the DLLs currently in the
moztools distribution and using them to create mozilla.org's binary
distributions.

Comment 6

[David Bradley]

It may work with 7.0, but I'm pretty sure it won't work with 6.0 untouched. I
don't remeber the precise problem, could be packing or calling convention. If
either of those, you might be able to tweak the compiler settings to get
something that would work on all existing versions.

Is this going to be SDK only, or are you planning on using this in all things
that build xpidl? I'd prefer making it the same across the board.

Comment 7

[Braden]

I'd be surprised if Visual C++ 7.x is completely incapable of using libraries
that work with Visual C++ 6.0, so I think this approach should be workable. It
may be that someone using Visual C++ 6.0 needs to build the libraries using the
makefiles I've attached.

Building with these static libraries rather than the dynamic ones in use now
should make everyones life easier, I think. So I figure it makes sense to use
them everywhere.

Comment 8

[Benjamin Smedberg]

dbaron, darin, thoughts? I like the idea. I don't think that changing from
dynamic to static linking affects the licensing issues at all, libIDL is still LGPL.

Comment 9

[Brian Haskin (Janzert)]

Tested this using vc6, it died with the below unresolved external symbol.

/cygdrive/c/mozdev/mozilla/build/cygwin-wrapper link /NOLOGO /OUT:xpidl.exe /PDB
:xpidl.pdb     -SUBSYSTEM:CONSOLE -NODEFAULTLIB:MSVCRTD xpidl.obj xpidl_idl.obj
xpidl_util.obj xpidl_header.obj xpidl_typelib.obj xpidl_doc.obj xpidl_java.obj .
/module.res ../../../dist/lib/xpt.lib c:/mozdev/moztools/lib/libidl-0.6.lib c:/m
ozdev/moztools/lib/glib-1.2.lib  kernel32.lib user32.lib gdi32.lib winmm.lib wso
ck32.lib advapi32.lib
   Creating library xpidl.lib and object xpidl.exp
glib-1.2.lib(gstrfuncs.obj) : error LNK2001: unresolved external symbol __ftol2
xpidl.exe : fatal error LNK1120: 1 unresolved externals

My moztools directory was made up of the zip from comment #3 and a bin directory
containing nsinstall.exe.

Brian

Comment 10

[Brian Haskin (Janzert)]

Tested this using vc6, it died with the below unresolved external symbol.

/cygdrive/c/mozdev/mozilla/build/cygwin-wrapper link /NOLOGO /OUT:xpidl.exe /PDB
:xpidl.pdb     -SUBSYSTEM:CONSOLE -NODEFAULTLIB:MSVCRTD xpidl.obj xpidl_idl.obj
xpidl_util.obj xpidl_header.obj xpidl_typelib.obj xpidl_doc.obj xpidl_java.obj .
/module.res ../../../dist/lib/xpt.lib c:/mozdev/moztools/lib/libidl-0.6.lib c:/m
ozdev/moztools/lib/glib-1.2.lib  kernel32.lib user32.lib gdi32.lib winmm.lib wso
ck32.lib advapi32.lib
   Creating library xpidl.lib and object xpidl.exp
glib-1.2.lib(gstrfuncs.obj) : error LNK2001: unresolved external symbol __ftol2
xpidl.exe : fatal error LNK1120: 1 unresolved externals

My moztools directory was made up of the zip from comment #3 and a bin directory
containing nsinstall.exe.

Brian

Comment 11

[Brian Haskin (Janzert)]

Attachment: static libraries built with vc6

This contains the same thing as attachment #148350 [details], but it is built using
Visual C++ 6. Could someone test this with vc 7.0 and 7.1?

Brian

Comment 12

[Son Le]

I've successfully build firefox using the new static libraries using Visual C++
7.1. Runs fine.

Comment 13

[Darin Fisher]

(In reply to comment #8)
> dbaron, darin, thoughts? I like the idea. I don't think that changing from
> dynamic to static linking affects the licensing issues at all, libIDL is still
LGPL.

provided licensing isn't an issue, i think we should definitely do this.

Comment 14

[David Bradley]

Yes, it should be fine. I don't think a lot of DLL's would be linked against 
this, so footprint increase should be minimal.

Comment 15

[Gervase Markham [:gerv]]

The LGPLed code only goes into tools which are part of the build chain, and not
into Mozilla/Firefox/Thunderbird binaries, right? 

We may still run into problems. Currently, the MPL gives us permission to
distribute binaries of MPLed code under any license we choose. By default, this
has been the MPL itself (which is a bit weird, but there you go). However, if we
statically link in LGPLed code, that might change the situation. The LGPL has no
such provision.

The LGPL also has requirements different to that of the MPL regarding source
code distribution, which we would have to obey.

Gerv

Comment 16

[Benjamin Smedberg]

The only thing in the source tree that uses glib/libidl is xpcom/typelib/xpidl,
which is part of the toolchain and SDK. I think that we do also ship it with
mozilla/firefox tarballs, but not installers. But static/dynamic linkage doesn't
change that situation, does it?

Comment 17

[Darin Fisher]

> But static/dynamic linkage doesn't change that situation, does it?

I had always thought that there was a difference between static and dynamic
linkage as far as source code licenses (or at least the LGPL) is concerned.  I'd
be happy if that were not the case.

Comment 18

[Braden]

No, it doesn't. And people should stop raising that boogeyman here without
evidence to the contrary.

If Mozilla has issues with the LGPL license attached to glib and libIDL, that
belongs in a different bug report. It is not related to this one.

Comment 19

[cls]

The issue isn't the use of the LGPL library but how the license interprets
static vs dynamic linking.  This is a valid bug for that discussion.

We have always err'd on the side of caution when it comes to the LGPL/GPL.  I'd
like to see some evidence that statically linking against a LGPL library
*doesn't* violate the LGPL and that we can still ship the binaries under the MPL
with this change.  IIRC, the libart library has been kept as a dynamic library
for this reason.

Comment 20

[Braden]

While we're at it, why don't we look for evidence the LGPL causes acne, bad
breath, and is the real reason for the commercial success of the Moon Pie?

How about the persons asserting the *does* differentiate between static and
dynamic linking produce evidence to that effect?

Comment 21

[Benjamin Smedberg]

Please note that there is a significant difference in the static-linking scenario. 

With dynamic linking, we are not shipping any LGPL code, we are only shipping
code that *links* to LGPL code (because we don't ship the glib/libidl DLLs).

With static linking, we actually end up shipping LGPL code.

Comment 22

[Braden]

> With static linking, we actually end up shipping LGPL code.

No, you end up shipping object code that is the compiled result of LGPL code.
And this technical distinction is absolutely irrelevant in the absence of a
citation of text in the license that indicates otherwise.

Comment 23

[Benjamin Smedberg]

Braden, you are ignoring the fine distinctions of the LGPL #5. As it stands, the
XPIDL binary is *not* (legally) a derivative work of libidl, and is therefore
not subject to any of the provisions of the LGPL. This is precisely because of
the dynamic linkage, and because we do not incorporate significant inline
functions. However, if we statically link against libidl and distribute that
code, we are most certainly a derivative work and must obey the provisions of
the LGPL #6, some of which provisions are unacceptable to commercial providers,
or are incompatible with licenses such as the MSVC.NET redistributable license.

Comment 24

[Braden]

bsmedberg: Section 5 says:

  If such an object file uses only numerical parameters, data structure layouts
  and accessors, and small macros and small inline functions (ten lines or less
  in length), then the use of the object file is unrestricted, regardless of
  whether it is legally a derivative work. (Executables containing this object
  code plus portions of the Library will still fall under Section 6.)

So use of the *object file* resulting from compiling code that includes a libIDL
header is unrestricted. The executable incorporating that object
code--xpidl--still falls under section 6. The term "static linking" (or similar)
does not even occur in section 5.

If there's is a problem with distribution of an xpidl binary invoking section 6
of the LGPL, *Mozilla is already there*.

Comment 25

[Gervase Markham [:gerv]]

bsmedberg said:
> some of which provisions are unacceptable to commercial providers, or are 
> incompatible with licenses such as the MSVC.NET redistributable license.

Do we link MSVC.NET redistributables into xpidl? I wasn't aware that the
MSVC.NET redistributable licence affected works which were merely distributed
alongside the redistributables.

(I actually think we have a problem with that licence anyway, as I think I've
mentioned in another bug somewhere.)

(In reply to comment #24)
> So use of the *object file* resulting from compiling code that includes a 
> libIDL header is unrestricted. The executable incorporating that object
> code--xpidl--still falls under section 6. 

Actually, the LGPL says:

"(Executables containing this object code *plus portions of the Library* will
still fall under Section 6.)"

Currently, our executables contain the object code but no portions of the
Library (outside of those in the section 5 list of things which make no
difference). If we statically linked, then the executables would contain both
our object code and portions of the library. That is where static linking comes
in, and why static vs. dynamic linking does make a difference in this situation.

This is not necessarily a showstopper. We could meet the requirements of Section
6 by providing a source code tarball download from the same place as the Gecko
SDK. The LGPL still allows us to distribute the SDK under "terms of our choice",
as long as we meet the LGPL requirements for the libraries we use - which we
could do by providing the aforementioned tarball.

Gerv

Comment 26

[Geoff "temporarily off bugmail" Beier]

building with these libraries:
https://bugzilla.mozilla.org/attachment.cgi?id=168489
instead of the vc71 libraries from ftp.mozilla.org solved the following
application error from xpidl.exe:

Application popup: xpidl.exe - Application Error : The application failed to
initialize properly (0xc0000022). Click on OK to terminate the application.

Windows XP Pro SP2, Visual Studio .Net 2003

Comment 27

[Son Le]

Discovered a side-effect of linking with the static libs - when source-level
tracing the line numbers are always off by a few lines.

Comment 28

[Cedric Vivier [:cedricv]]

Attachment: static libs with MT (do not link with msvcr...)

Comment 29

[Benjamin Smedberg]

Attachment: Static libIDL with -MT and fixes for VC8 runtime libs

Comment 30

[Benjamin Smedberg]

Attachment: Patch to glib 1.2.10

Comment 31

[Benjamin Smedberg]

Attachment: Patch to libIDL 0.6.8

Comment 32

[Benjamin Smedberg]

Attachment: Detect static libIDL and patch xpidl.exe accordingly, rev. 1

This patches configure to recognize static libIDL and pass -MT when compiling xpidl against static libIDL.

Comment 33

[cls]

Comment on attachment 209380 [details] [diff] [review]
Detect static libIDL and patch xpidl.exe accordingly, rev. 1

Just a handful of nits:
1) We should check for the static libs in the cross-compile case as well.
2) Why do we need to link against advapi32 now? 
3) And why aren't we linking against it for win32-gcc?
4) Should we be using -NODEFAULTLIB:LIBC when building against a non-static libIDL?
5) There's an extra glib-1.2_s.lib on the xpidl link line.  That should be taken care of in LIBIDL_LIBS.

Comment 34

[Benjamin Smedberg]

Attachment: Detect static libIDL and patch xpidl.exe accordingly, rev. 1.1 [fixed on trunk]

Damn, the extra hardcoded C:\builds\glib thing was for testing and I forgot to remove it. And it turns out that hardcoding advapi is also not needed. We want nodefaultlib:libc because that is the static-non-multithread version of libc and we never want to link against it (something pulled it in through directives and I couldn't figure out what).

Comment 35

[Benjamin Smedberg]

Chase, can you please make the following changes on stage:

1) rename pub/mozilla.org/mozilla/source/wintools.zip to pub/mozilla.org/mozilla/source/wintools-20060124.zip

2) upload attachment 209373 [details] to pub/mozilla.org/mozilla/libraries/win32/wintools.zip

3) upload attachment 209376 [details] [diff] [review] to pub/mozilla.org/libraries/source/glib-1.2.10-static.patch
and attachment 209379 [details] [diff] [review] to pub/mozilla.org/libraries/source/libIDL-0.6.8-static.patch

Comment 36

[Benjamin Smedberg]

Comment on attachment 209448 [details] [diff] [review]
Detect static libIDL and patch xpidl.exe accordingly, rev. 1.1 [fixed on trunk]

Attachment 209448 [details] [diff] checked in on trunk.

Comment 37

[cls]

Attachment: Fix casing of libIDL references for cross-compiled builds

Comment 38

[Benjamin Smedberg]

Comment on attachment 209697 [details] [diff] [review]
Fix casing of libIDL references for cross-compiled builds

Of course.

Comment 39

[Benjamin Smedberg]

preed, can you perform the stage activities in comment 35?

Comment 40

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Shaver has staged the "Static libIDL with -MT and fixes for VC8 runtime libs" as http://ftp.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/moztools-static.zip -- it would be good if someone were to also create a src dir and populate it with the patches/nmake files/etc. just in case someone wants to go through that fun in the future.  All the old gunk got moved to the "old" dir.

Comment 41

[Benjamin Smedberg]

This is fixed, with staged bits and client-side code checked in on trunk and 1.8branch.