319416 - IRIX MIPSPro static build requires reordering of libraries and alternate archive commands

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Frank Everdij]

User-Agent:       Mozilla/5.0 (X11; U; IRIX IP32; en-US; rv:1.8) Gecko/20051206 Firefox/1.5
Build Identifier: Mozilla/5.0 (X11; U; IRIX IP32; en-US; rv:1.8) Gecko/20051206 Firefox/1.5

A --enable-static build of firefox-1.5 release on SGI/IRIX 6.5.22 with MIPSPro 7.4.2 needs the following patch to compile correctly:

--- configure.save      Thu Dec  1 21:45:25 2005
+++ configure   Mon Dec  5 14:52:28 2005
@@ -5801,10 +5801,10 @@
     else
        MKSHLIB_FORCE_ALL='-all'
        MKSHLIB_UNFORCE_ALL='-none'
-          AR_LIST="$AR t"
-          AR_EXTRACT="$AR x"
-          AR_DELETE="$AR d"
-          AR='$(CXX) -ar'
+          AR_LIST="ar t"
+          AR_EXTRACT="ar x"
+          AR_DELETE="ar d"
+          AR='$(CXX) -ar -WR,-suv'
           AR_FLAGS='-o $@'
        CFLAGS="$CFLAGS -woff 3262 -G 4"
        CXXFLAGS="$CXXFLAGS -woff 3262 -G 4"
--- config/rules.mk.save        Tue Dec  6 15:55:53 2005
+++ config/rules.mk     Tue Dec  6 16:04:08 2005
@@ -841,7 +841,7 @@
        $(LD) -NOLOGO -OUT:$@ -PDB:$(PDBFILE) $(WIN32_EXE_LDFLAGS) $(LDFLAGS) $(
PROGOBJS) $(RESFILE) $(LIBS) $(EXTRA_LIBS) $(OS_LIBS)
 else
 ifeq ($(CPP_PROG_LINK),1)
-       $(CCC) -o $@ $(CXXFLAGS) $(WRAP_MALLOC_CFLAGS) $(PROGOBJS) $(RESFILE) $(
WIN32_EXE_LDFLAGS) $(LDFLAGS) $(LIBS_DIR) $(LIBS) $(OS_LIBS) $(EXTRA_LIBS) $(BIN
_FLAGS) $(WRAP_MALLOC_LIB) $(PROFILER_LIBS) $(EXE_DEF_FILE)
+       $(CCC) -o $@ $(CXXFLAGS) $(WRAP_MALLOC_CFLAGS) $(PROGOBJS) $(RESFILE) $(
WIN32_EXE_LDFLAGS) $(LDFLAGS) $(LIBS_DIR) $(EXTRA_LIBS) $(LIBS) $(OS_LIBS) $(BIN
_FLAGS) $(WRAP_MALLOC_LIB) $(PROFILER_LIBS) $(EXE_DEF_FILE)
 else # ! CPP_PROG_LINK
        $(CC) -o $@ $(CFLAGS) $(PROGOBJS) $(RESFILE) $(WIN32_EXE_LDFLAGS) $(LDFL
AGS) $(LIBS_DIR) $(LIBS) $(OS_LIBS) $(EXTRA_LIBS) $(BIN_FLAGS) $(EXE_DEF_FILE)
 endif # CPP_PROG_LINK

The default archive for C++ code under MIPSPro should be CC -ar -WR,-suv because  this is the only way MIPSPro will instantiate templates in static objects. However, enabling this archiver choice for AR_LIST AR_EXTRACT and AR_DELETE will break the subsequent 'for' loops in Makefiles, when extracting an archive, because the command then reads CC -ar -WR,-suv x <archive.a> which is incorrect syntax. Rather than specifying the C++ archive commands, which even if its syntax is done correctly, also breaks the 'for' loops in the makefile, one can simply define AR_EXTRACT as "ar x" because no template instantiation is need for extracting/listing objects. I'm not sure if AR_DELETE 'ar d' requires building a new symbol table, but i tested it and it seems to work fine.

The reordering of libraries at link stage is a bit odd for SGI machines. Usually it wants shared objects defined after static object so i shifted the $(EXTRA_LIBS) two places up front.

With both of these corrections, i have sucessfully compiled Firefox 1.5 statically on my IRIX systems, and am typing from it :)

Yours,

Frank Everdij

Reproducible: Always




build env:
setenv CC c99
setenv CXX CC
setenv CFLAGS "-O3 -mips4 -n32 -INLINE -woff 1174"
setenv CXXFLAGS "-O3 -mips4 -n32 -INLINE -woff 1110,1171,1201,1355,3201"
setenv LDFLAGS "-L/usr/nekoware/lib"
setenv LIBS -lfastm -lm
setenv PERL "/usr/nekoware/bin/perl"
setenv MOZ_PHOENIX "1"
setenv BUILD_OFFICIAL "1"
setenv CPPFLAGS "-I/usr/nekoware/include -I/usr/nekoware/include/glib-1.2 -I/usr
/nekoware/lib/glib/include"

.mozconfig:
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/fb-opt-static
ac_add_options --prefix=/usr/local/firefox-test
ac_add_options --enable-application=browser
ac_add_options --without-pthread
ac_add_options --enable-optimize=-O3
ac_add_options --disable-debug
ac_add_options --enable-static
ac_add_options --disable-shared
ac_add_options --disable-tests
ac_add_options --disable-composer
ac_add_options --enable-default-toolkit=gtk
ac_add_options --enable-freetype2
ac_add_options --enable-plaintext-editor-only
ac_add_options --disable-accessibility

Comment 1

[Ludovic Hirlimann [:Usul]]

Confirming bug because a patch is present. Reporter could you attach the patch in the bug ?

Comment 2

[Frank Everdij]

Attachment: diff -u patch to mozilla root

Comment 3

[Benjamin Smedberg]

Comment on attachment 205224 [details] [diff] [review]
diff -u patch to mozilla root

I'm more than slightly worried about the EXTRA_LIBS change in rules.mk, I'm asking cls for his opinion of this.

Comment 4

[cls]

Comment on attachment 205224 [details] [diff] [review]
diff -u patch to mozilla root

I'd rather not hardcode the ar values if we don't have to or at least, we need a way to be able to override the location of the ar binary.

The -WR,-suv flags should really be added to AR_FLAGS not AR.  How does using '$(CC) -ar' cause the build to break when it's not breaking right now?

I don't understand how this fixes the link reordering as EXTRA_LIBS aren't guaranteed to be static or even from the Mozilla tree.  Plus, you only changed the link order in one instance.  We should use a consistent link order for all cases/platforms.

I'd prefer to introduce a new variable to handle the static link libraries that come from static-rules.mk .  I'm assuimg that's the only reason you need to move EXTRA_LIBS.

Comment 5

[Frank Everdij]

(In reply to comment #4)
> (From update of attachment 205224 [details] [diff] [review] [edit])
> I'd rather not hardcode the ar values if we don't have to or at least, we need
> a way to be able to override the location of the ar binary.

On IRIX you don't have much choice when compiling with MIPSPro compilers. I elaborate:

CC -ar is used solely and exclusively to create or update archives. I quote from the man pages of CC:

     -ar         Creates an archive using ar(1) instead of a shared object
                 or executable.  The name of the archive is specified by
                 using the -o option.  Template entities required by the
                 objects being archived are instantiated before creating
                 the archive.  The CC command implicitly passes the -r and
                 -c options of ar to ar in addition to the name of the
                 archive and the objects being created.  Any other option
                 that can be used in conjunction with the -c option of ar
                 can be passed to ar using -WR option_name.  For more
                 information, see the ar(1) man page.

Archive extraction/listing/deletion cannot be used with -c and -r and are therefore invalid options when specifying with the -WR,<options> flag.
This means that CC -ar only has one function, namely to provide template instantiation when creating an archive, because the options -c and -r are implicitely added.

The full command: CC -ar -WR,-suv -o <archive> <object>... therefore reads as:
"Create a static archive and instantiate templates. Create a symbol table (-s), update objects if already present (-u) and provide verbose output (-v)"
Consequently, one has to fall back to the standard "vanilla" ar t, ar x and ar d commands when listing, extracting or deleting objects from the archive. On IRIX/MIPSPro that is...

> The -WR,-suv flags should really be added to AR_FLAGS not AR.  How does using
> '$(CC) -ar' cause the build to break when it's not breaking right now?

Whether the part of '-WR,-suv' should be included in the definition of AR or AR_FLAGS is anyone's guess. Clearly AR_FLAGS is not used in combination with AR_LIST AR_EXTRACT or AR_DELETE, so '-WR,-suv' can either be left in AR or absorbed in AR_FLAGS. Same story for the '-o', this option, when used with CC -ar, should be followed by the archive itself. All this limits the useability of these environment variables in IRIX to only one combination: $AR $AR_FLAGS <archive> <objects>...

My initial comment of "Rather than specifying the C++ archive commands, which even if its syntax is done correctly, also breaks the 'for' loops in the makefile, one can simply define AR_EXTRACT as "ar x" because no template instantiation is need for extracting/listing objects." wasn't entirely correct, because i thought the for-loop breaking had something to do with using CC -ar as archive command in general, forgetting that no valid combination between CC -ar and any extract/listing/delete option flag could ever be made. Sorry for that...

> I don't understand how this fixes the link reordering as EXTRA_LIBS aren't
> guaranteed to be static or even from the Mozilla tree.  Plus, you only changed
> the link order in one instance.  We should use a consistent link order for all
> cases/platforms.
> 
> I'd prefer to introduce a new variable to handle the static link libraries that
> come from static-rules.mk .  I'm assuimg that's the only reason you need to
> move EXTRA_LIBS.

 I agree that we need a consistent order accross all the different build flavors. I have no idea whether reordering of libraries affects other platform builds. As suggested, adding a flag to enforce a certain ordering would be a solution.
Just to indicate what problem i encountered:

cd mozilla/fb-opt-static/browser/app
gmake

With the vanilla mozilla/config/rules.mk with order $(LIBS) $(OS_LIBS) $(EXTRA_LIBS) i get 

CC -o firefox-bin -I/usr/nekoware/include -I/usr/nekoware/include/glib-1.2 -I/usr/nekoware/lib/glib/include  -LANG:exceptions=OFF -g -n32 -woff 3262 -G 4 -n32  -DNDEBUG -DTRIMMED -O3  nsBrowserApp.o nsStaticComponents.o   -L/usr/nekoware/lib -n32   -Wl,-LD_LAYOUT:lgot_buffer=80 -L../../dist/bin -L../../dist/lib -L../../dist/lib/components  ../../dist/lib/libxulapp_s.a -L../../dist/bin -lmozjs -L../../dist/bin -lxpcom -lxpcom_core -L../../dist/lib -lplds4 -lplc4 -lnspr4 -L/usr/nekoware/lib -lpthread -ldl -L/usr/nekoware/lib -lgtk -lgdk -lgmodule -lglib -lintl -lXext -lX11 -lm   -lX11  -lsocket -ldl -lm -lfastm -lm -lpthread ../../dist/lib/components/libxpcom_compat_c.a ../../dist/lib/components/libxpconnect.a ../../dist/lib/components/libuconv.a ../../dist/lib/components/libucvmath.a ../../dist/lib/components/libi18n.a ../../dist/lib/components/libnecko.a ../../dist/lib/components/libnecko2.a ../../dist/lib/components/libjar50.a ../../dist/lib/components/libpref.a ../../dist/lib/components/libcaps.a ../../dist/lib/components/librdf.a ../../dist/lib/components/libhtmlpars.a ../../dist/lib/components/libgfxps.a ../../dist/lib/components/libgfx_gtk.a ../../dist/lib/components/libimglib2.a ../../dist/lib/components/libgkplugin.a ../../dist/lib/components/libwidget_gtk.a ../../dist/lib/components/libgklayout.a ../../dist/lib/components/libdocshell.a ../../dist/lib/components/libembedcomponents.a ../../dist/lib/components/libwebbrwsr.a ../../dist/lib/components/libeditor.a ../../dist/lib/components/libtxmgr.a ../../dist/lib/components/libnsappshell.a ../../dist/lib/components/liboji.a ../../dist/lib/components/libchrome.a ../../dist/lib/components/libmork.a ../../dist/lib/components/libmozfind.a ../../dist/lib/components/libappcomps.a ../../dist/lib/components/libfileview.a ../../dist/lib/components/libremoteservice.a ../../dist/lib/components/libcommandlines.a ../../dist/lib/components/libtoolkitcomps.a ../../dist/lib/components/libpipboot.a ../../dist/lib/components/libpipnss.a ../../dist/lib/components/libpippki.a ../../dist/lib/components/libcookie.a ../../dist/lib/components/libxmlextras.a ../../dist/lib/components/libautoconfig.a ../../dist/lib/components/libtransformiix.a ../../dist/lib/components/libuniversalchardet.a ../../dist/lib/components/libwebsrvcs.a ../../dist/lib/components/libauth.a ../../dist/lib/components/libpermissions.a ../../dist/lib/components/libsearchservice.a ../../dist/lib/components/libbrowserdirprovider.a ../../dist/lib/components/libbrowsercomps.a ../../dist/lib/libunicharutil_s.a ../../dist/lib/libucvutil_s.a ../../dist/lib/libgtksuperwin.a ../../dist/lib/libgtkxtbin.a ../../dist/lib/libgfxshared_s.a ../../dist/lib/libgfxft2_s.a ../../dist/lib/libgfxpsshar.a ../../dist/lib/libgkgfx.a ../../dist/lib/libjsj.a ../../dist/lib/libxulapp_s.a  -L../../dist/lib -lmozpng -lm -L../../dist/lib -lmozjpeg -L../../dist/lib -lmozz  -L../../dist/bin -L../../dist/lib ../../dist/lib/libcrmf.a -lsmime3 -lssl3 -lnss3 -lsoftokn3    -lXt -L../../dist/lib -lxpcom_compat

which will bomb out on ld with 1617 ERROR messages, whereas with reordering into $(EXTRA_LIBS) $(LIBS) $(OS_LIBS) gives

CC -o firefox-bin -I/usr/nekoware/include -I/usr/nekoware/include/glib-1.2 -I/usr/nekoware/lib/glib/include  -LANG:exceptions=OFF -g -n32 -woff 3262 -G 4 -n32  -DNDEBUG -DTRIMMED -O3  nsBrowserApp.o nsStaticComponents.o   -L/usr/nekoware/lib -n32   -Wl,-LD_LAYOUT:lgot_buffer=80 -L../../dist/bin -L../../dist/lib ../../dist/lib/components/libxpcom_compat_c.a ../../dist/lib/components/libxpconnect.a ../../dist/lib/components/libuconv.a ../../dist/lib/components/libucvmath.a ../../dist/lib/components/libi18n.a ../../dist/lib/components/libnecko.a ../../dist/lib/components/libnecko2.a ../../dist/lib/components/libjar50.a ../../dist/lib/components/libpref.a ../../dist/lib/components/libcaps.a ../../dist/lib/components/librdf.a ../../dist/lib/components/libhtmlpars.a ../../dist/lib/components/libgfxps.a ../../dist/lib/components/libgfx_gtk.a ../../dist/lib/components/libimglib2.a ../../dist/lib/components/libgkplugin.a ../../dist/lib/components/libwidget_gtk.a ../../dist/lib/components/libgklayout.a ../../dist/lib/components/libdocshell.a ../../dist/lib/components/libembedcomponents.a ../../dist/lib/components/libwebbrwsr.a ../../dist/lib/components/libeditor.a ../../dist/lib/components/libtxmgr.a ../../dist/lib/components/libnsappshell.a ../../dist/lib/components/liboji.a ../../dist/lib/components/libchrome.a ../../dist/lib/components/libmork.a ../../dist/lib/components/libmozfind.a ../../dist/lib/components/libappcomps.a ../../dist/lib/components/libfileview.a ../../dist/lib/components/libremoteservice.a ../../dist/lib/components/libcommandlines.a ../../dist/lib/components/libtoolkitcomps.a ../../dist/lib/components/libpipboot.a ../../dist/lib/components/libpipnss.a ../../dist/lib/components/libpippki.a ../../dist/lib/components/libcookie.a ../../dist/lib/components/libxmlextras.a ../../dist/lib/components/libautoconfig.a ../../dist/lib/components/libtransformiix.a ../../dist/lib/components/libuniversalchardet.a ../../dist/lib/components/libwebsrvcs.a ../../dist/lib/components/libauth.a ../../dist/lib/components/libpermissions.a ../../dist/lib/components/libsearchservice.a ../../dist/lib/components/libbrowserdirprovider.a ../../dist/lib/components/libbrowsercomps.a ../../dist/lib/libunicharutil_s.a ../../dist/lib/libucvutil_s.a ../../dist/lib/libgtksuperwin.a ../../dist/lib/libgtkxtbin.a ../../dist/lib/libgfxshared_s.a ../../dist/lib/libgfxft2_s.a ../../dist/lib/libgfxpsshar.a ../../dist/lib/libgkgfx.a ../../dist/lib/libjsj.a ../../dist/lib/libxulapp_s.a  -L../../dist/lib -lmozpng -lm -L../../dist/lib -lmozjpeg -L../../dist/lib -lmozz  -L../../dist/bin -L../../dist/lib ../../dist/lib/libcrmf.a -lsmime3 -lssl3 -lnss3 -lsoftokn3    -lXt -L../../dist/lib -lxpcom_compat -L../../dist/lib/components  ../../dist/lib/libxulapp_s.a -L../../dist/bin -lmozjs -L../../dist/bin -lxpcom -lxpcom_core -L../../dist/lib -lplds4 -lplc4 -lnspr4 -L/usr/nekoware/lib -lpthread -ldl -L/usr/nekoware/lib -lgtk -lgdk -lgmodule -lglib -lintl -lXext -lX11 -lm   -lX11  -lsocket -ldl -lm -lfastm -lm -lpthread

will give me firefox-bin.

Note BTW that there already is a difference in reordering in rules.ml. Check lines 840-844:
ifeq (_WINNT,$(GNU_CC)_$(OS_ARCH))
	$(LD) -NOLOGO -OUT:$@ -PDB:$(PDBFILE) $(WIN32_EXE_LDFLAGS) $(LDFLAGS) $(PROGOBJS) $(RESFILE) $(LIBS) $(EXTRA_LIBS) $(OS_LIBS)
else
ifeq ($(CPP_PROG_LINK),1)
	$(CCC) -o $@ $(CXXFLAGS) $(WRAP_MALLOC_CFLAGS) $(PROGOBJS) $(RESFILE) $(WIN32_EXE_LDFLAGS) $(LDFLAGS) $(LIBS_DIR) $(LIBS) $(OS_LIBS) $(EXTRA_LIBS) $(BIN_FLAGS) $(WRAP_MALLOC_LIB) $(PROFILER_LIBS) $(EXE_DEF_FILE)

i'll stop here since this is already getting too long. Hope this all makes sense...

Comment 6

[Matthias Versen [:Matti]]

IRIX build support got removed with bug 680792