Statically link libIDL, glib with xpidl on Windows

RESOLVED FIXED

Status

SeaMonkey
Build Config
RESOLVED FIXED
14 years ago
12 years ago

People

(Reporter: Braden, Assigned: preed)

Tracking

({fixed1.8})

Trunk
x86
Windows XP
fixed1.8
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(7 attachments, 4 obsolete attachments)

1.01 KB, text/plain
Details
915 bytes, text/plain
Details
236.47 KB, application/x-zip
Details
6.12 KB, patch
Details | Diff | Splinter Review
5.09 KB, patch
Details | Diff | Splinter Review
10.69 KB, patch
cls
: review+
Brian Ryner (not reading)
: approval-branch-1.8.1+
Details | Diff | Splinter Review
1.32 KB, patch
bsmedberg
: review+
Details | Diff | Splinter Review
(Reporter)

Description

14 years ago
Gecko SDK binaries ought to be self-sufficient; that is it ought to be the 
case that one can get the Gecko SDK and not have to fish for libIDL and glib 
DLLs. The current situation is worsened by the fact that Visal C++ 7.0 builds 
of these libraries don't play nicely with Visual C++ 7.1 builds of xpidl, and 
vice versa.
(Reporter)

Comment 1

13 years ago
Created attachment 148348 [details]
nmake makefile to build glib-1.2.10 as a static library
(Reporter)

Comment 2

13 years ago
Created attachment 148349 [details]
nmake makefile to build libIDL-0.6.8 as a static library
(Reporter)

Comment 3

13 years ago
Created attachment 148350 [details]
glib-1.2.10 and libIDL-0.6.8 static library binaries (and public headers)

These binaries were built with Visual C++ 7.1. However, since they are static
libraries, I suspect they may work with previous compiler versions. Obviously
that should be tested; however, I'm not in a position to do so.

Also, note that the glib binary includes the glib library, and not any of its
accessory libraries (gthread, gmodule, gplugin). Mozilla does not appear to
require those; and not building them avoids the pthreads-win32 dependency.
(Reporter)

Comment 4

13 years ago
Using these static libraries to build xpidl does not appear to require any 
changes to Mozilla's build system. Simply make them available (and make sure 
no crufty .dll or .exp files are hanging around), and it should Just Work.
(Reporter)

Comment 5

13 years ago
Comment on attachment 148350 [details]
glib-1.2.10 and libIDL-0.6.8 static library binaries (and public headers)

leaf, could you test this against Visual C++ 6.0 and 7.0? If all goes well, I
propose distributing these binaries instead of the DLLs currently in the
moztools distribution and using them to create mozilla.org's binary
distributions.
Attachment #148350 - Flags: review?(leaf)
(Reporter)

Updated

13 years ago
Assignee: nobody → braden
(Reporter)

Updated

13 years ago
Status: NEW → ASSIGNED
Blocks: 249194

Comment 6

13 years ago
It may work with 7.0, but I'm pretty sure it won't work with 6.0 untouched. I
don't remeber the precise problem, could be packing or calling convention. If
either of those, you might be able to tweak the compiler settings to get
something that would work on all existing versions.

Is this going to be SDK only, or are you planning on using this in all things
that build xpidl? I'd prefer making it the same across the board.
(Reporter)

Comment 7

13 years ago
I'd be surprised if Visual C++ 7.x is completely incapable of using libraries
that work with Visual C++ 6.0, so I think this approach should be workable. It
may be that someone using Visual C++ 6.0 needs to build the libraries using the
makefiles I've attached.

Building with these static libraries rather than the dynamic ones in use now
should make everyones life easier, I think. So I figure it makes sense to use
them everywhere.
dbaron, darin, thoughts? I like the idea. I don't think that changing from
dynamic to static linking affects the licensing issues at all, libIDL is still LGPL.
Product: Browser → Seamonkey
Tested this using vc6, it died with the below unresolved external symbol.

/cygdrive/c/mozdev/mozilla/build/cygwin-wrapper link /NOLOGO /OUT:xpidl.exe /PDB
:xpidl.pdb     -SUBSYSTEM:CONSOLE -NODEFAULTLIB:MSVCRTD xpidl.obj xpidl_idl.obj
xpidl_util.obj xpidl_header.obj xpidl_typelib.obj xpidl_doc.obj xpidl_java.obj .
/module.res ../../../dist/lib/xpt.lib c:/mozdev/moztools/lib/libidl-0.6.lib c:/m
ozdev/moztools/lib/glib-1.2.lib  kernel32.lib user32.lib gdi32.lib winmm.lib wso
ck32.lib advapi32.lib
   Creating library xpidl.lib and object xpidl.exp
glib-1.2.lib(gstrfuncs.obj) : error LNK2001: unresolved external symbol __ftol2
xpidl.exe : fatal error LNK1120: 1 unresolved externals

My moztools directory was made up of the zip from comment #3 and a bin directory
containing nsinstall.exe.

Brian
Tested this using vc6, it died with the below unresolved external symbol.

/cygdrive/c/mozdev/mozilla/build/cygwin-wrapper link /NOLOGO /OUT:xpidl.exe /PDB
:xpidl.pdb     -SUBSYSTEM:CONSOLE -NODEFAULTLIB:MSVCRTD xpidl.obj xpidl_idl.obj
xpidl_util.obj xpidl_header.obj xpidl_typelib.obj xpidl_doc.obj xpidl_java.obj .
/module.res ../../../dist/lib/xpt.lib c:/mozdev/moztools/lib/libidl-0.6.lib c:/m
ozdev/moztools/lib/glib-1.2.lib  kernel32.lib user32.lib gdi32.lib winmm.lib wso
ck32.lib advapi32.lib
   Creating library xpidl.lib and object xpidl.exp
glib-1.2.lib(gstrfuncs.obj) : error LNK2001: unresolved external symbol __ftol2
xpidl.exe : fatal error LNK1120: 1 unresolved externals

My moztools directory was made up of the zip from comment #3 and a bin directory
containing nsinstall.exe.

Brian
Created attachment 168489 [details]
static libraries built with vc6

This contains the same thing as attachment #148350 [details], but it is built using
Visual C++ 6. Could someone test this with vc 7.0 and 7.1?

Brian

Updated

13 years ago
Blocks: 274221

Comment 12

13 years ago
I've successfully build firefox using the new static libraries using Visual C++
7.1. Runs fine.

Comment 13

13 years ago
(In reply to comment #8)
> dbaron, darin, thoughts? I like the idea. I don't think that changing from
> dynamic to static linking affects the licensing issues at all, libIDL is still
LGPL.

provided licensing isn't an issue, i think we should definitely do this.

Comment 14

13 years ago
Yes, it should be fine. I don't think a lot of DLL's would be linked against 
this, so footprint increase should be minimal.
The LGPLed code only goes into tools which are part of the build chain, and not
into Mozilla/Firefox/Thunderbird binaries, right? 

We may still run into problems. Currently, the MPL gives us permission to
distribute binaries of MPLed code under any license we choose. By default, this
has been the MPL itself (which is a bit weird, but there you go). However, if we
statically link in LGPLed code, that might change the situation. The LGPL has no
such provision.

The LGPL also has requirements different to that of the MPL regarding source
code distribution, which we would have to obey.

Gerv
The only thing in the source tree that uses glib/libidl is xpcom/typelib/xpidl,
which is part of the toolchain and SDK. I think that we do also ship it with
mozilla/firefox tarballs, but not installers. But static/dynamic linkage doesn't
change that situation, does it?

Comment 17

13 years ago
> But static/dynamic linkage doesn't change that situation, does it?

I had always thought that there was a difference between static and dynamic
linkage as far as source code licenses (or at least the LGPL) is concerned.  I'd
be happy if that were not the case.
(Reporter)

Comment 18

13 years ago
No, it doesn't. And people should stop raising that boogeyman here without
evidence to the contrary.

If Mozilla has issues with the LGPL license attached to glib and libIDL, that
belongs in a different bug report. It is not related to this one.

Comment 19

13 years ago
The issue isn't the use of the LGPL library but how the license interprets
static vs dynamic linking.  This is a valid bug for that discussion.

We have always err'd on the side of caution when it comes to the LGPL/GPL.  I'd
like to see some evidence that statically linking against a LGPL library
*doesn't* violate the LGPL and that we can still ship the binaries under the MPL
with this change.  IIRC, the libart library has been kept as a dynamic library
for this reason.
(Reporter)

Comment 20

13 years ago
While we're at it, why don't we look for evidence the LGPL causes acne, bad
breath, and is the real reason for the commercial success of the Moon Pie?

How about the persons asserting the *does* differentiate between static and
dynamic linking produce evidence to that effect?
Please note that there is a significant difference in the static-linking scenario. 

With dynamic linking, we are not shipping any LGPL code, we are only shipping
code that *links* to LGPL code (because we don't ship the glib/libidl DLLs).

With static linking, we actually end up shipping LGPL code.
(Reporter)

Comment 22

13 years ago
> With static linking, we actually end up shipping LGPL code.

No, you end up shipping object code that is the compiled result of LGPL code.
And this technical distinction is absolutely irrelevant in the absence of a
citation of text in the license that indicates otherwise.
Braden, you are ignoring the fine distinctions of the LGPL #5. As it stands, the
XPIDL binary is *not* (legally) a derivative work of libidl, and is therefore
not subject to any of the provisions of the LGPL. This is precisely because of
the dynamic linkage, and because we do not incorporate significant inline
functions. However, if we statically link against libidl and distribute that
code, we are most certainly a derivative work and must obey the provisions of
the LGPL #6, some of which provisions are unacceptable to commercial providers,
or are incompatible with licenses such as the MSVC.NET redistributable license.
(Reporter)

Comment 24

13 years ago
bsmedberg: Section 5 says:

  If such an object file uses only numerical parameters, data structure layouts
  and accessors, and small macros and small inline functions (ten lines or less
  in length), then the use of the object file is unrestricted, regardless of
  whether it is legally a derivative work. (Executables containing this object
  code plus portions of the Library will still fall under Section 6.)

So use of the *object file* resulting from compiling code that includes a libIDL
header is unrestricted. The executable incorporating that object
code--xpidl--still falls under section 6. The term "static linking" (or similar)
does not even occur in section 5.

If there's is a problem with distribution of an xpidl binary invoking section 6
of the LGPL, *Mozilla is already there*.
bsmedberg said:
> some of which provisions are unacceptable to commercial providers, or are 
> incompatible with licenses such as the MSVC.NET redistributable license.

Do we link MSVC.NET redistributables into xpidl? I wasn't aware that the
MSVC.NET redistributable licence affected works which were merely distributed
alongside the redistributables.

(I actually think we have a problem with that licence anyway, as I think I've
mentioned in another bug somewhere.)

(In reply to comment #24)
> So use of the *object file* resulting from compiling code that includes a 
> libIDL header is unrestricted. The executable incorporating that object
> code--xpidl--still falls under section 6. 

Actually, the LGPL says:

"(Executables containing this object code *plus portions of the Library* will
still fall under Section 6.)"

Currently, our executables contain the object code but no portions of the
Library (outside of those in the section 5 list of things which make no
difference). If we statically linked, then the executables would contain both
our object code and portions of the library. That is where static linking comes
in, and why static vs. dynamic linking does make a difference in this situation.

This is not necessarily a showstopper. We could meet the requirements of Section
6 by providing a source code tarball download from the same place as the Gecko
SDK. The LGPL still allows us to distribute the SDK under "terms of our choice",
as long as we meet the LGPL requirements for the libraries we use - which we
could do by providing the aforementioned tarball.

Gerv
building with these libraries:
https://bugzilla.mozilla.org/attachment.cgi?id=168489
instead of the vc71 libraries from ftp.mozilla.org solved the following
application error from xpidl.exe:

Application popup: xpidl.exe - Application Error : The application failed to
initialize properly (0xc0000022). Click on OK to terminate the application.

Windows XP Pro SP2, Visual Studio .Net 2003

Comment 27

13 years ago
Discovered a side-effect of linking with the static libs - when source-level
tracing the line numbers are always off by a few lines.
Created attachment 195746 [details]
static libs with MT (do not link with msvcr...)
Attachment #148350 - Flags: review?(leaf)
Created attachment 209373 [details]
Static libIDL with -MT and fixes for VC8 runtime libs
Assignee: braden → benjamin
Attachment #148350 - Attachment is obsolete: true
Attachment #168489 - Attachment is obsolete: true
Attachment #195746 - Attachment is obsolete: true
Created attachment 209376 [details] [diff] [review]
Patch to glib 1.2.10
Created attachment 209379 [details] [diff] [review]
Patch to libIDL 0.6.8
Created attachment 209380 [details] [diff] [review]
Detect static libIDL and patch xpidl.exe accordingly, rev. 1

This patches configure to recognize static libIDL and pass -MT when compiling xpidl against static libIDL.
Attachment #209380 - Flags: review?(cls)

Comment 33

12 years ago
Comment on attachment 209380 [details] [diff] [review]
Detect static libIDL and patch xpidl.exe accordingly, rev. 1

Just a handful of nits:
1) We should check for the static libs in the cross-compile case as well.
2) Why do we need to link against advapi32 now? 
3) And why aren't we linking against it for win32-gcc?
4) Should we be using -NODEFAULTLIB:LIBC when building against a non-static libIDL?
5) There's an extra glib-1.2_s.lib on the xpidl link line.  That should be taken care of in LIBIDL_LIBS.
Attachment #209380 - Flags: review?(cls) → review-
Created attachment 209448 [details] [diff] [review]
Detect static libIDL and patch xpidl.exe accordingly, rev. 1.1 [fixed on trunk]

Damn, the extra hardcoded C:\builds\glib thing was for testing and I forgot to remove it. And it turns out that hardcoding advapi is also not needed. We want nodefaultlib:libc because that is the static-non-multithread version of libc and we never want to link against it (something pulled it in through directives and I couldn't figure out what).
Attachment #209380 - Attachment is obsolete: true
Attachment #209448 - Flags: review?(cls)

Updated

12 years ago
Attachment #209448 - Flags: review?(cls) → review+
Chase, can you please make the following changes on stage:

1) rename pub/mozilla.org/mozilla/source/wintools.zip to pub/mozilla.org/mozilla/source/wintools-20060124.zip

2) upload attachment 209373 [details] to pub/mozilla.org/mozilla/libraries/win32/wintools.zip

3) upload attachment 209376 [details] [diff] [review] to pub/mozilla.org/libraries/source/glib-1.2.10-static.patch
and attachment 209379 [details] [diff] [review] to pub/mozilla.org/libraries/source/libIDL-0.6.8-static.patch
Comment on attachment 209448 [details] [diff] [review]
Detect static libIDL and patch xpidl.exe accordingly, rev. 1.1 [fixed on trunk]

Attachment 209448 [details] [diff] checked in on trunk.
Attachment #209448 - Attachment description: Detect static libIDL and patch xpidl.exe accordingly, rev. 1.1 → Detect static libIDL and patch xpidl.exe accordingly, rev. 1.1 [fixed on trunk]
Blocks: 323931

Comment 37

12 years ago
Created attachment 209697 [details] [diff] [review]
Fix casing of libIDL references for cross-compiled builds
Comment on attachment 209697 [details] [diff] [review]
Fix casing of libIDL references for cross-compiled builds

Of course.
Attachment #209697 - Flags: review+
preed, can you perform the stage activities in comment 35?
Assignee: benjamin → preed
Status: ASSIGNED → NEW
Blocks: 255460
Attachment #209448 - Flags: branch-1.8.1?(bryner)
Attachment #209448 - Flags: branch-1.8.1?(bryner) → branch-1.8.1+
Shaver has staged the "Static libIDL with -MT and fixes for VC8 runtime libs" as http://ftp.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/moztools-static.zip -- it would be good if someone were to also create a src dir and populate it with the patches/nmake files/etc. just in case someone wants to go through that fun in the future.  All the old gunk got moved to the "old" dir.
This is fixed, with staged bits and client-side code checked in on trunk and 1.8branch.
Status: NEW → RESOLVED
Last Resolved: 12 years ago
Keywords: fixed1.8
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.