79681 - mozilla should statically link libstdc++ under (at least) Linux

Status: VERIFIED WONTFIX

(SeaMonkey :: Build Config, defect, P5)

Description

[jim_nance]

Due to the problems we are having with incompatable g++ libraries under Linux, I
decided to see how much bloat it would add if we statically linked everything
with libstdc++.  I did some experiments and was supprised to find that the size
of our component libraries actually goes DOWN if we statically link.  My
hypothesis is that we dont actualy use anything from this library, and we dont
have to store the library name in the component if we statically link.

Given that there is a code size advantage and that it makes our binaries much
more portable, I can see no reason not to statically link libstdc++. 
Unfortunatly there does not seem to be an easy way to tell the compiler to do
this.  You can configure a gcc build with --disable-shared and then all the
binaries it builds will have a static libstdc++, but this does not seem like a
good or a practicle solution.  I played with -Wl,-Bstatic,-lstdc++,-Bdynamic
options on the gcc command line but I could not get this to work.  I finally
found that just putting libstdc++.a into dist/bin at the beginning of a build
would cause everything to be statically linked.  This seems simple enough to do:

ln -s `g++ -print-file-name=libstdc++.a` dist/bin

cls also had some suggestions about linking the libraries with gcc instead of
g++ which would eleminate the use of libstdc++.  I have not checked to see if
this works, but it is something to think about.

Comment 1

[jim_nance]

cls,
    I am trying to figure out where the build system creates the $(DIST) 
directory and its children.  I looked in the top level makefile as well as in 
rules.mk but I dont see it.  Can you point me in the right direction?

Comment 2

[Brendan Eich [:brendan]]

The nsinstall program creates prerequisite directories.  There is no particular
makefile rule to create $(DIST), therefore.  Instead, whichever installable gets
nsintalled into a subdir of $(DIST) first (prolly a .h file) causes the ancestor
line to be created as needed.

Why did you want to find a makefile that created $(DIST), anyhoo?

/be

Comment 3

[jim_nance]

Thanks Brendan.  My proposed way to fix this is to symlink libstdc++.a into the 
dist/bin directory.  It seemed like a good place to put this code would be 
whereever that directory is created.

Comment 4

[Jens-Uwe]

is this connected to bug 46775?

Comment 5

[cls]

No.

Comment 6

[olaf]

Two notes: 
1. The proposed solution of linking with gcc instead of g++ does not work with
all compiler versions (perhaps even configurations?) so the symlink solution
seems to be more appropriate.
2. The probably right place to generate the symlink is the EXTRA_CMDS section of
the AC_OUTPUT clause in configure.in, after making sure that the directory
exists (by creating a dummy dist/bin/Makefile via allmakefiles.sh?). It probably
needs to be considered in acoutput-fast.pl too.

Comment 7

[robbe]

Please keep this feature optional. Distributors using proper package management
can easily make their package depend on the package providing the right shared
lib. Those don't need/want static libstdc++.

I understand that jim's hack links code from libstdc++.a into shared library
components. Are you sure that libstdc++.a contains proper PIC code? Because that
would be evil.

> My hypothesis is that we dont actualy use anything from this library,

In this case we shouldn't link with it at all! I seriously doubt that theory,
though.

Comment 8

[cls]

It would definitely be optional.  There's no guarantee that libstdc++.a would
contain the proper PIC code (I believe we've run into that problem before). 
I've compiled the entire app using CXX=gcc which does not implicitly link
against libstdc++.  With the exception of a few console apps like regchrome,
Mozilla worked fine without libstdc++.  And configure would be the wrong place
to make the symlink as the link would be blown away on a clobber and not
recreated.  It would most likely be created from config/Makefile.in .

Now, I just thought about this.  Since libstdc++ is LGPL'd, what sort of license
complications would arise if mozilla.org chose to link it statically?  My
initial feeling is that it cannot be done without tainting the project.

Comment 9

[robbe]

> With the exception of a few console apps like regchrome, Mozilla worked fine
> without libstdc++.

Perhaps it should be avoided throughout, except when linking these programs.
Either by using "gcc", or (I'm not sure whether replacing g++ by gcc has no
undesired side-effects) "g++ -nodefaultlibs -lgcc -lc" as the compiler.

The few exceptions that need libstdc++ could get a
$(full_path_of_static_libstdc++) [detected by autoconf] added to their link
command, avoiding any symlink tricks. Since this it explicit, you'd be
reasonably sure that code from the static lib would not end up in shared libs.

> Since libstdc++ is LGPL'd, what sort of license complications would arise if
> mozilla.org chose to link it statically?

None. The Lesser GPL is explicitly designed so that "non-free" (or
non-GPL-compatible in this case) applications can link to code under it. It does
not matter whether linking is dynamic or static.

Comment 10

[cls]

> None. The Lesser GPL is explicitly designed so that "non-free" (or
> non-GPL-compatible in this case) applications can link to code under it. It
> does not matter whether linking is dynamic or static.

I recall this being a hot topic in previous *GPL discussions I've had.  A
re-reading of section 5 of the LGPL implies (to me anyways) that static linking
would cause the "executable" to contain portions of the LGPL'd library and
thereof be defined as a derivative work and must be LGPL'd.  (Yes, I know we
just said that the majority of the project does not use symbols from libstdc++
but some parts do.  And this seems far too sticky of an issue to take lightly.)

Mitchell, as the resident lawyer & license expert, can you short-circuit this
discussion and let us know if this is going to be a problem?

http://www.gnu.org/copyleft/lgpl.html

Comment 11

[jim_nance]

Attachment: patch to link libstdc++ statically

Comment 12

[jim_nance]

I just added a patch the makes static c++ library linking a configure options. 
I did not read the comments that have been posted here before I wrote it. 
Hopefully it does not contradict any of them.

Comment 13

[cls]

Sorry, I meant to update this last week.  I talked to Mitchell in person last
week and she agreed that the LGPL was unclear on this particular matter.   So
until this particular legal issue is cleared up (presumably by speaking with RMS
himself), we will not be able to statically link against libstdc++.

Comment 14

[Dan Mosedale (:dmosedale, :dmose)]

So instead of dropping the priority and pushing it out, isn't the right thing to
do reassign this bug to mitchell?  This seems pretty important, after all...

Comment 15

[Dan Mosedale (:dmosedale, :dmose)]

Or, alternately, pursuing the path already suggested by cls and robbe:
eliminating libstdc++ from the linkage entirely.

Comment 16

[Ben Bucksch (:BenB)]

Will post my comments to license question to .license newsgroup.

Comment 17

[Ben Bucksch (:BenB)]

> Or, alternately, pursuing the path already suggested by cls and robbe

This seems to be the better choice, if and only if we don't intend to ever use
the C*+ standard lib (libstdc++). Better, because executables that are linked
with it, dynamically or statically, are subject to a small subset of the GPL
terms (Section 6). Or so I understood.

Comment 18

[jim_nance]

I see some discussion about the LGPL here, but I dont actually think that
libstdc++ is an LGPL library (it used to be).  Its a GPL library that has
an escape clause that lets you use it in non-GPL programs iff you are using
it with gcc.  For more details go to a Linux box and read the top of:

    /usr/include/g++-3/iostream.h

Comment 19

[Jens-Uwe]

just for those without linux box nearby, I cite the most interesting part from
the iostream.h - header.

This file is part of the GNU IO Library.  This library is free
software; you can redistribute it and/or modify it under the
terms of the GNU General Public License as published by the
Free Software Foundation; either version 2, or (at your option)
any later version.

[...]

As a special exception, if you link this library with files
compiled with a GNU compiler to produce an executable, this does not cause
the resulting executable to be covered by the GNU General Public License.
This exception does not however invalidate any other reasons why
the executable file might be covered by the GNU General Public License. */

Comment 20

[cls]

Nifty.  They buried the legalese in the src files instead of the toplevel
document files.  Looking at the gcc 2.95.3 src tarball, not all of the src files
under libstdc++ nor libio contain that notice.  Mitchell, is this going to be a
problem or do we have sufficient coverage to proceed?

Comment 21

[Craig Kelley]

Straight from the horse's mouth: 

http://gcc.gnu.org/ml/libstdc++/2000-q2/msg00055.html

In short, libstdc++ has a special "runtime" GPL license that allows it to be
used in closed projects;  I don't think Mozilla is even an issue, because anyone
can get the source and re-link as needed -- but furthermore, I believe that it
could be statically linked into commercial code if some provisos were followed
(allowing re-linking ala Oracle).  IANAL, of course.

Comment 22

[cls]

Also note that the post is referring to libstdc++-v3, ie the libstdc++ that will
be shipped with gcc-3.0 when it's released later this month (hopefully).  If you
find anything conclusive referring to the libstdc++ that's shipped with
gcc-2.95.x (and covers all relevant src files), let us know.

Comment 23

[Jens-Uwe]

christopher: 
the text I cited in my comment from 2001-05-30 09:53 was taken from my Linux
system, which was shipped with gcc2.95.2.

I am really sorry for getting off topic, but shouldn't a tracking bug for gcc3
be created to make the change in the midterm? Once this bug here is fixed, one
of the heavy stones is taken out of the way :-)

Comment 24

[cls]

Jens-Uwe, I checked the files in the gcc 2.95.3 release and while some of the
files contain that notice, not all of them do.  Nor can I find any toplevel
documentation that claims the change is in effect for the entire library.  Until
we have clarification on that issue, we shouldn't proceed with the change.

I believe there are already several bugs upon on building with gcc3.  I know
dbaron has done quite a bit of work in that area.  Not sure if there's a
tracking bug though.

Comment 25

[jim_nance]

We need to clarify any license issues before mozilla.org can distribute binaries
linked with a static c++ library.  However I dont think that should affect the
status of the patch which enables static linking of the c++ library.  There is
already a precident for this too, as we have a --with-static-gtk configure
option right now.  The patch adds a similar option for linking with the c++
library.

Comment 26

[Mitchell Baker]

Well, this all still seems pretty murky to me, even with the exception. 
Cathleen and I have discussed, but i'm still not that clear on how we use this.
 chris????  cathleen and I await your help

mitchell

Comment 27

[Mitchell Baker]

cls and I had a long talk, where chris explained the basics of reality to me. 
So I now feel pretty comfortable (that's a relative term) that the exception
mentioned is intended and could cover the use we have in mind.

So I've reached the same point as cls.  If we had confirmation that the
exception applied to all the files of the version we're using life would be good.

believe cls is trying to get this clarification.  

Comment 28

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

To answer the question of how Mozilla does with gcc3:  I don't know of any
problems for x86, but we still need to fix bug 71627 and bug 86446 to use gcc
3.0 on other architectures.  (gcc 3.0 does not have bug 83388, and it also has
the advantage of allowing the good form of NS_LITERAL_STRING and handles GetIID
better.)

There is still one known problem for Netscape's commercial build with gcc3:
http://bugscape.netscape.com/show_bug.cgi?id=6876

Comment 29

[Ben Bucksch (:BenB)]

The patch doesn't work for me (at least not straigh away). I manually created
the link, and it works fine: No links to libstdc++, no size increase. Apart from
2 libraries: libgtksuperwin.so and libgtkxtbin.so; they still link libstdc++
dynamically!
drepper said that s/g++/gcc/ would do the trick. gcc can compile C++ code, the
only difference is that gcc doesn't link to libstdc++ automatically.

Comment 30

[Ben Bucksch (:BenB)]

I'll attach a new, additional patch which fixes the problem with libgtk* for me.
(you will still need the link trick.) Many thanks for Myth and drepper for
telling me how to do it.

Comment 31

[Ben Bucksch (:BenB)]

Attachment: Fix, version 1, part 2. Fixes libgtk*.

Comment 32

[cls]

Just FYI:

From: Mark Mitchell <[e-mail]>
To: Chris Seawood <[e-mail]>
cc: "[e-mail]" <[e-mail]>

--On Thursday, July 19, 2001 02:18:02 PM -0700 Chris Seawood 
<[e-mail]> wrote:

>
> Hi Mark,

I really don't have a good answer.  I know that the GCC 2.95.x licensing
terms were unclear to say the least.  I guess you should contact the
FSF directly.

-- 
Mark Mitchell                [e-mail]
CodeSourcery, LLC            http://www.codesourcery.com

Comment 33

[(Don't use, old account) - Kai Engert]

Having read sections 5 and 6 of the LGPL, I want to try a conclusion.
As I'm no lawyer, I might be wrong.

I assume the libstdc++ library is indeed licensed under LGPL.
The home page of this library is at http://gcc.gnu.org/libstdc++/

To avoid confusion between the section numbering of the LGPL and this comment,
I'm numbering this comment using roman numbers.


I.) General requirements for statically linked executables.
===========================================================
The program parts which display licenses (for example the "about Mozilla"
screen, or the "license step" of the "installation wizard") must display a list
of the names of all the statically linked libraries. And it must include the
license(s) of the included libraries, e.g. the LGPL.


II.) General requirements for dynamically linked executables.
=============================================================
When linking dynamically, the LGPL says in section 5, that the whole license
does not apply.
Therefore mentioning the library or including the license does not seem to be
necessary.

However, linking dynamically still contains the risk of having added library
contents statically. This happens if the header files include not only
declarations, but also function definitions. As soon as this happens (for at
least one used function of at least 10 lines in size), the executable may no
longer be considered a dynamic one, although the bunch of code is still
dynamically. It must be considered a static one, and the rules from sections
I./III. do apply.


III.) Statically linked executables of the open source version of Mozilla.
==========================================================================
If we follow section 6a of the LGPL, we may distribute a statically linked version.

If Mozilla is distributed as downloads only, we can do so by providing
downloadable archives that contain:
- the complete Mozilla source code
- the complete libstdc++ that was used to compile.
  To make that easier for everyone, we might just provide a source code copy of
the used gcc compiler (which includes the libstdc++).
- of course, all the sources of all other used LPGL'ed libraries we link with
statically.

As soon as Mozilla executables are provided on different media, those media must
contain the sources right away, or the sources may be provided on request free
of charge.


IV.) Dynamically linked executables from vendors who add closed source portions
to Mozilla.
===========================================================================================
Same as II.) above.

Regarding license restrictions, this is the easiest way for a vendor to go.
Regarding usable system environments, this imposes restrictions on which
environments will work, as the environment must match the requirements.
However, as vendors often want to give support for customers, they often certify
for special platforms only, to ease giving support.
If necessary a vendor can produce multiple executables for all supported
environments of any given operating system.


V.) Statically linked executables from vendors who add closed source portions to
Mozilla.
=========================================================================================
This is allowed, but introduces additional overhead for the vendor.
The license requires the vendor to include the object files of the closed source
portions, and any compilation/build tools used. Of course, in addition to above
section III.).

a.)
===
Speaking in terms of Mozilla, this could mean compiling the modified Mozilla
tree "out of the source tree", i.e. in a separate object directory.
The vendor could strip all debug information from the object files, and provide
that bunch of files to the users/customers.
However, providing the object files for the closed source shows symbol names
which are required to link successfully. In addition to header files. A vendor
who wants to disclose source portions might not want to do that.

b.)
===
An alternative and maybe easier way of providing the closed source in linkable
form: Create one or more shared libraries that contain the closed source portions.
Provide all the sources that can be given out, including the header files
(definitions) to access the shared library containing the closed source portions.


==========
Conclusion
==========
- Currently mozilla.org uses II. (at least on Linux).

- Mozilla.org could do I. + III. easily.
  This introduced flexibility for private users, who use a variety of different
distributions.

- Vendors are free to chose between either:

  - II. + IV.         (good)
  or
  - I. + III. + V. a  (seems inappropriate)
  or
  - I. + III. + V. b  (good)

Comment 34

[(Don't use, old account) - Kai Engert]

Sorry for the previous comment to be so long.

I'm personally interested in having a statically linked version of Mozilla, and
I think resolving this bug is the first step to reach this goal.

It would be great if there were a option to the "configure" script, which
selects whether to link with all the compiler libraries either statically or
dynamically.

I say "all", because gcc 3.0 introduces an additional library, "libgcc", which
must be linked statically, too, if we want to resolve this bug.

Comment 35

[hacker formerly known as [e-mail]]

This has been stagnating because of the legal issues for so long that it
would've been faster to get users accustomed to the extra dependencies.  Marking
WONTFIX.

Comment 36

[Ben Bucksch (:BenB)]

Chris, this bug is a major problem for distributors. Depending on a non-standard
C++ stdlib is baaad. (OK, I have a workaround, but it is inconvient and very
error-prone.)
I don't think, it's a good idea to close the bug wontfix because no one wants to
sort of the legal crap. Why can't we just accept a working patch and warn users
of the compile flag that the legal status is unclear?

Comment 37

[Ben Bucksch (:BenB)]

No answer. I'll go forward and reopen.

Unfortunately, the workaround / patches presented here stopped to work for me,
no idea why. I'm compiling Mozilla 0.9.4.1 with Debian's gcc 2.95.4.

Comment 38

[hacker formerly known as [e-mail]]

If a distributor depends upon a non-standard c++ lib, then they pay the burden
of making sure that their users have access to that non-standard c++ lib or they
hack *their* copy of mozilla to deal with the static link (/legal) issues.  In
other words, they pay the burden of dealing with the non-standard configuration,
not us.  RedHat is a good example of a distributor who carries their own burden
in this regard.  

Also, now that gcc 3 is out, we have the additional shared lib dependency of
libgcc_s, which is recommened to only be statically linked against static apps (
see http://gcc.gnu.org/gcc-3.0/libgcc.html ).  Even our "static" version of
mozilla isn't completely static so we'd be asking for problems there if we tried
to statically link libgcc. So if we're going to have to get users accustomed to
a dynamic libgcc dependency (eventually), then they can handle the dynamic
libstdc++ dependency as well.

You're using gcc 2.95.4 and presumably a stock release of it.  I don't see any
standard gcc/libstdc++ release a being a "non-standard" c++ lib but that'll be
forever up to debate, I suppose.  If you want something to work out of the box
for your users, then you need to create a "better" box like Netscape & Redhat do
for their releases.

Trying to skim around the legal issues and deal with the potential breakage of
this static libstdc++ hack is not worth it.

Comment 39

[Ben Bucksch (:BenB)]

> In other words, they pay the burden of dealing with the non-standard
> configuration, not us.  RedHat is a good example of a distributor who
> carries their own burden in this regard.  

No, any application that wants to be used widely has to support Redhat. The fact
that Redhat ships a Mozilla version doesn't matter, because I think that I can
do a better job, as does Netscape seem to think (the only Linux version approved
by Netscape is Redhat).

So, it is not only Redhat's problem, it is also mine, unfortunately.

(Even worse, this problem restricts mozilla.org and Netscape to egcs 1.1, which
is an inferiour compiler, which means that our C++ language used is severely
crippled.)

> Also, now that gcc 3 is out

gcc 3.0 is another matter. Luckily, we don't have to care about it yet.

> If you want something to work out of the box
> for your users, then you need to create a "better" box like Netscape &
> Redhat do for their releases.

What, do you think, should that "box" look like, if it wants to be "better"? I
certainly don't consider using an age-old compiler (as Netscape does) being
better. Redhat compiles for its own compiler version, which is out of question
for me, unless I want to produce 6+ packages for Linux.

I consider that "better box" being a build that doesn't needlessly link to libs
that might not be available, i.e. fixes this bug.


If anybody has a completely different proposal how to avoid that burden on
either users (for installing another lib) or distributors (for producing one
package for each distribution), I'd be happy to hear it.

Comment 40

[Jens-Uwe]

>> Also, now that gcc 3 is out

> gcc 3.0 is another matter. Luckily, we don't have to care about it yet.

the latest linux nightlies are compiled with gcc3

Comment 41

[Jon Granrose]

verified wontfix.

Comment 42

[devphil]

If you are using gcc3 to build mozilla these days, then it may be worthwhile to
find out which parts of libstdc++ you're using (i.e., which symbols are being
pulled in and why).

I recall reading on the mozilla coding guide that exceptions shouldn't be used,
most of the STL subset shouldn't be used, etc, etc.  Without commenting on those
decisions, I will point out that gcc 3.x ships a libsupc++.a library, containing
only the runtime support routines.  Basically, new/delete, some RTTI, and some EH.

This library is included in libstdc++, both .so and .a, but is also provided
seperately in a .so form.  If you do a static link with -lsupc++, you may not
need libstdc++ at all.