386445 - [Mac] Clean checkout dies with "multiple definitions of symbol nsINIParser::GetStrings" and others

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, defect)

Description

[Karsten Düsterloh]

Doing a clean checkout of client.mk and building suite from that results in the following build failure (I'm CCing some Makefile and Mac savvies):

g++ -I/usr/X11R6/include -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -fpascal-strings -no-cpp-precomp -fno-common -fshort-wchar -I/Developer/Headers/FlatCarbon -pipe  -DDEBUG -D_DEBUG -DDEBUG_karsten -DTRACING -g -fPIC  -o libsuite.dylib  nsSuiteModule.o  nsSuiteDirectoryProvider.o nsProfileMigrator.o nsSuiteProfileMigratorUtils.o nsNetscapeProfileMigratorBase.o nsSeamonkeyProfileMigrator.o nsThunderbirdProfileMigrator.o           -L../../dist/bin -L../../dist/lib  -L../../dist/lib -lunicharutil_external_s ../../dist/../modules/libreg/src/libmozreg_s.a -L../../dist/bin -lmozjs ../../dist/lib/libxpcomglue_s.a -L../../dist/bin -lxpcom -lxpcom_core -L../../dist/bin -L../../dist/lib -lplds4 -lplc4 -lnspr4  -framework Carbon   -bundle    
/usr/bin/ld: multiple definitions of symbol nsINIParser::GetStrings(char const*, int (*)(char const*, char const*, void*), void*)
../../dist/bin/libxpcom_core.dylib(nsINIParser.o) definition of nsINIParser::GetStrings(char const*, int (*)(char const*, char const*, void*), void*)
../../dist/lib/libxpcomglue_s.a(nsINIParser.o) private external definition of nsINIParser::GetStrings(char const*, int (*)(char const*, char const*, void*), void*)in section (__TEXT,__text)
/usr/bin/ld: multiple definitions of symbol nsINIParser::GetSections(int (*)(char const*, void*), void*)
../../dist/bin/libxpcom_core.dylib(nsINIParser.o) definition of nsINIParser::GetSections(int (*)(char const*, void*), void*)
../../dist/lib/libxpcomglue_s.a(nsINIParser.o) private external definition of nsINIParser::GetSections(int (*)(char const*, void*), void*)in section (__TEXT,__text)
/usr/bin/ld: multiple definitions of symbol nsINIParser::InitFromFILE(__sFILE*)
../../dist/bin/libxpcom_core.dylib(nsINIParser.o) definition of nsINIParser::InitFromFILE(__sFILE*)
../../dist/lib/libxpcomglue_s.a(nsINIParser.o) private external definition of nsINIParser::InitFromFILE(__sFILE*)       in section (__TEXT,__text)
/usr/bin/ld: multiple definitions of symbol nsINIParser::GetSectionsCB(char const*, nsINIParser::INIValue*, void*)
../../dist/bin/libxpcom_core.dylib(nsINIParser.o) definition of nsINIParser::GetSectionsCB(char const*, nsINIParser::INIValue*, void*)
../../dist/lib/libxpcomglue_s.a(nsINIParser.o) private external definition of nsINIParser::GetSectionsCB(char const*, nsINIParser::INIValue*, void*)in section (__TEXT,__text)
/usr/bin/ld: multiple definitions of symbol nsINIParser::Init(nsILocalFile*)
../../dist/bin/libxpcom_core.dylib(nsINIParser.o) definition of nsINIParser::Init(nsILocalFile*)
../../dist/lib/libxpcomglue_s.a(nsINIParser.o) private external definition of nsINIParser::Init(nsILocalFile*)       in section (__TEXT,__text)
/usr/bin/ld: multiple definitions of symbol nsINIParser::Init(char const*)
../../dist/bin/libxpcom_core.dylib(nsINIParser.o) definition of nsINIParser::Init(char const*)
../../dist/lib/libxpcomglue_s.a(nsINIParser.o) private external definition of nsINIParser::Init(char const*)in section (__TEXT,__text)
/usr/bin/ld: multiple definitions of symbol nsINIParser::GetString(char const*, char const*, char*, unsigned int)
../../dist/bin/libxpcom_core.dylib(nsINIParser.o) definition of nsINIParser::GetString(char const*, char const*, char*, unsigned int)
../../dist/lib/libxpcomglue_s.a(nsINIParser.o) private external definition of nsINIParser::GetString(char const*, char const*, char*, unsigned int)in section (__TEXT,__text)
collect2: ld returned 1 exit status
make[6]: *** [libsuite.dylib] Error 1
make[5]: *** [libs] Error 2
make[4]: *** [libs_tier_app] Error 2
make[3]: *** [tier_app] Error 2
make[2]: *** [default] Error 2
make[1]: *** [build] Error 2
make: *** [build] Error 2

Comment 1

[Benjamin Smedberg]

1) what compiler version
2) what's the target CPU?

Comment 2

[Karsten Düsterloh]

1) powerpc-apple-darwin8-gcc-4.0.1 (GCC) 4.0.1 (Apple Computer, Inc. build 5363)
2) PPC G4

My .mozconfig:

CC=gcc
CXX=g++
mk_add_options MOZ_OBJDIR=/Users/karsten/Projekte/Mozilla/obj/soctest
mk_add_options MOZ_CO_PROJECT=suite
ac_add_options --enable-application=suite
MOZ_PKG_SPECIAL=suiterunner
ac_add_options --enable-debug
ac_add_options --disable-optimize
ac_add_options --enable-chrome-format=flat
ac_add_options --enable-crypto
ac_add_options --enable-extensions=default
ac_add_options --enable-svg
ac_add_options --enable-canvas
ac_add_app_options ppc --enable-prebinding

Comment 3

[Anant Narayanan [:anant]]

I can confirm this error. I'm using:
i686-apple-darwin8-gcc-4.0.1 (GCC) 4.0.1 (Apple Computer, Inc. build 5367)
on an Intel Macbook Pro.

My .mozconfig:
ac_add_options --disable-optimize --disable-tests --enable-debug --enable-xpctools
mk_add_options MOZ_BUILD_PROJECTS="suite xulrunner"
mk_add_options MOZ_CO_PROJECT="suite xulrunner"
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@

ac_add_app_options suite --enable-application=suite
ac_add_app_options xulrunner --enable-application=xulrunner

Comment 4

[Markus Hossner]

I have the same problem. Clean checkouts < 16 Jun. work and clean checkouts > 21 Jun. don't work.

Comment 5

[Benjamin Smedberg]

By design there are two definitions of this symbol in play:

* The version in libxpcom_core.dylib. Not safe for external linkage.

* The version in libxpcomglue_s.a, which is safe for external linkage, and also should be private extern.

The link command lists libxpcomglue_s.a before libxpcom_core.dylib, so I would expect the first (statically linked) definition to be picked up, and not the second.

And I'd also expect this to be a problem for many more symbols than nsINIParser... I don't understand why these are special. I can certainly do a little #define renaming magic to change the symbols for external linkage, but I'm worried about why I'd have to do that.

Comment 6

[Anant Narayanan [:anant]]

Perhaps looking at the changes that were made between 16th and 21st June (as specified in a previous comment) would help. I wonder why Tinderbox doesn't show a failed build?

Comment 7

[Benjamin Smedberg]

I added support for hidden visibility (private extern) symbols on mac.

Comment 8

[jag (Peter Annema)]

I grepped the xpcomglue_s.a private external definition of lines from a build.log and the same build.log with the last batch (the error) removed | sort | uniq, and then diffed the two output files (assuming that symbols that previously didn't cause an error wouldn't be the cause of the error in the last batch), and that came up with this:

77a78,84
> nsINIParser.o nsINIParser::GetSections(int (*)(char const*, void*), void*)
> nsINIParser.o nsINIParser::GetSectionsCB(char const*, nsINIParser::INIValue*, void*)
> nsINIParser.o nsINIParser::GetString(char const*, char const*, char*, unsigned int)
> nsINIParser.o nsINIParser::GetStrings(char const*, int (*)(char const*, char const*, void*), void*)
> nsINIParser.o nsINIParser::Init(char const*)
> nsINIParser.o nsINIParser::Init(nsILocalFile*)       
> nsINIParser.o nsINIParser::InitFromFILE(__sFILE*)       
82a90,98
> nsTArray.o nsTArray_base::EnsureCapacity(unsigned int, unsigned int)
> nsTArray.o nsTArray_base::EnsureNotUsingAutoArrayBuffer(unsigned int)
> nsTArray.o nsTArray_base::InsertSlotsAt(unsigned int, unsigned int, unsigned int)
> nsTArray.o nsTArray_base::ShiftData(unsigned int, unsigned int, unsigned int, unsigned int)
> nsTArray.o nsTArray_base::ShrinkCapacity(unsigned int)
> nsTArray.o nsTArray_base::SwapArrayElements(nsTArray_base&, unsigned int)
> nsTArray.o nsTArray_base::nsTArray_base()
> nsTArray.o nsTArray_base::sEmptyHdr       
> nsTArray.o nsTArray_base::~nsTArray_base()

I just noticed that for the other multiple definitions, it's something like:

/usr/bin/ld: warning multiple definitions of symbol NS_NewGenericModule(char const*, unsigned int, nsModuleComponentInfo*, void (*)(nsIModule*), nsIModule**)../../dist/lib/libxpcomglue_s.a(nsGenericFactory.o) private external definition of NS_NewGenericModule(char const*, unsigned int, nsModuleComponentInfo*, void (*)(nsIModule*), nsIModule**)in section (__TEXT,__text)../../dist/bin/libxpcom_core.dylib(nsGenericFactory.o) definition of NS_NewGenericModule(char const*, unsigned int, nsModuleComponentInfo*, void (*)(nsIModule*), nsIModule**)

A warning, and it's looking in libxpcomglue_s.a first.

while for INIParser it's:

/usr/bin/ld: multiple definitions of symbol nsINIParser::GetStrings(char const*, int (*)(char const*, char const*, void*), void*)
../../dist/bin/libxpcom_core.dylib(nsINIParser.o) definition of nsINIParser::GetStrings(char const*, int (*)(char const*, char const*, void*), void*)
../../dist/lib/libxpcomglue_s.a(nsINIParser.o) private external definition of nsINIParser::GetStrings(char const*, int (*)(char const*, char const*, void*), void*)in section (__TEXT,__text)

It's not a warning, and for some reason it's looking in libxpcom_core.dylib first (which is probably why it's not a warning).

Comment 9

[jag (Peter Annema)]

Note that both of these came from the same ld command, so why the mixed up order?

Comment 10

[Markus Hossner]

Checkout with Tag D2007.06.19.00.00.00 works fine.

Comment 11

[jag (Peter Annema)]

Comment 7 seems to imply the checkin for bug 384513 around 7:53 on June 19th is the cause of this.

Comment 12

[Benjamin Smedberg]

Yes, sorry if comment #7 wasn't clear enough about that.

Comment 13

[jag (Peter Annema)]

shebs, any ideas?

Comment 14

[Stan Shebs]

Symbols can get picked up in a different order because of cross-references to them from elsewhere. People have been thrashing around with symbol visibility on Macs since forever it seems, defeats me as much as everybody else.

Comment 15

[jag (Peter Annema)]

Hmm, so the obvious one is nsINIParserImpl.cpp, but even with that stubbed out (i.e. all references to nsINIParser removed) those symbols still got linked in a different order.

Is there any way to have the linker spell out for us why it's picking symbols from libxpcom_core.dylib instead of libxpcomglue_s.a, despite the relative order of these two?

Comment 16

[Benjamin Smedberg]

Attachment: Rename nsINIParser, rev. 1

I haven't finished compiling this, but I think it will work. I added single_module to the dylib link line because it is recommended and has the potential to make libxul load a lot faster (I found it in "man ld" while researching this bug).

Comment 17

[jag (Peter Annema)]

Comment on attachment 270786 [details] [diff] [review]
Rename nsINIParser, rev. 1

With just the rename I can do a clean build. With the -single_module bit applied I get:

c++  -fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -fpascal-strings -no-cpp-precomp -fno-common -fshort-wchar -I/Developer/Headers/FlatCarbon -pipe  -DDEBUG -D_DEBUG -DDEBUG_jag -DTRACING -g -I/Developer/Headers/FlatCarbon -fPIC  -o libgkgfxthebes.dylib  nsThebesDeviceContext.o nsThebesImage.o nsThebesRegion.o nsThebesGfxFactory.o nsThebesRenderingContext.o nsThebesFontMetrics.o nsThebesFontEnumerator.o nsSystemFontsMac.o             ../shared/libgfxshared_s.a -L../../../dist/bin -L../../../dist/lib -lgkgfx -lthebes ../../../modules/libutil/src/libmozutil_s.a -L../../../dist/bin -lxpcom -lxpcom_core -L../../../dist/bin -L../../../dist/lib -lplds4 -lplc4 -lnspr4 ../../../dist/lib/libunicharutil_s.a -L../../../dist/bin -lmozjs -framework Carbon   -bundle    
/usr/bin/ld: warning can't open dynamic library: @executable_path/libmozz.dylib referenced from: ../../../dist/bin/libthebes.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2)
/usr/bin/ld: Undefined symbols:
_MOZ_Z_deflate referenced from libthebes expected to be defined in @executable_path/libmozz.dylib
_MOZ_Z_deflateEnd referenced from libthebes expected to be defined in @executable_path/libmozz.dylib
_MOZ_Z_deflateInit_ referenced from libthebes expected to be defined in @executable_path/libmozz.dylib
collect2: ld returned 1 exit status

r=jag on the rename, though I'd really like us (collectively) to understand why this is needed.

Maybe file a new bug on getting the single_module thing working?

Comment 18

[Benjamin Smedberg]

Attachment: Rename nsINIParser, rev. 2

This fixes the additional issues in dynamic builds.

Comment 19

[(not currently active) Ted Mielczarek]

Comment on attachment 271712 [details] [diff] [review]
Rename nsINIParser, rev. 2

Looks reasonable to me, from what I understand.

Comment 20

[Benjamin Smedberg]

Fixed on trunk, and -single_module bought us back a sizable perf win, probably the perf that was lost when libxul was turned on, but that's still mostly unexplained.