648134 - [OS/2] follow-up Stop building intermediate static libs with fakelibs

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Walter Meinl]

User-Agent:       Mozilla/5.0 (OS/2; Warp 4.5; rv:2.0b13pre) Gecko/20110321 Firefox/4.0b13pre
Build Identifier: 

The new python-based linking system does not work for OS/2. First, the build breaks due to having ".a" and not a ".lib" as the archive extension for ffi. After fixing this, the build breaks when expandlibs_exec.py tries to remove a (not closed) temporary file. The reason for that is that the configure check for the EXPAND_LIBS_LIST macro doesn't work properly and the macro is set to "none". I'll attach a WIP patch that corrects the extension for ffi, and let the ar_extract function work properly... but fails to link xul.dll, finally.

Reproducible: Always

Comment 1

[Walter Meinl]

Attachment: WIP - fails to link xul.dll

Comment 2

[Walter Meinl]

Comment on attachment 524282 [details] [diff] [review]
WIP - fails to link xul.dll

>diff --git a/configure.in b/configure.in
>--- a/configure.in
>+++ b/configure.in
>@@ -2639,16 +2639,17 @@ ia64*-hpux*)
>     LIB_PREFIX=
>     LIB_SUFFIX=lib
>     BIN_SUFFIX=".exe"
>     DLL_SUFFIX=".dll"
>     IMPORT_LIB_SUFFIX=lib
>     DSO_PIC_CFLAGS=
>     AR=emxomfar
>     AR_FLAGS='r $@'
>+    AR_EXTRACT=                <- this is needed to reset AR_EXTRACT
>     CFLAGS="$CFLAGS -Zomf"
>     CXXFLAGS="$CXXFLAGS -Zomf"
>     DSO_LDOPTS='-Zdll'
>     BIN_FLAGS='-Zlinker /ST:0x100000'
>     IMPLIB='emximp -o'
>     FILTER='true'
>     LDFLAGS='-Zmap'
>     WARNINGS_AS_ERRORS='-Werror'
>@@ -8205,17 +8206,17 @@ AC_CACHE_CHECK(what kind of list files a
>          echo "INPUT(conftest.${OBJ_SUFFIX})" > conftest.list
>          if AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS conftest.list $LIBS 1>&2) && test -s conftest${ac_exeext}; then
>              EXPAND_LIBS_LIST_STYLE=linkerscript
>          else
>              echo "conftest.${OBJ_SUFFIX}" > conftest.list
>              if AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS @conftest.list $LIBS 1>&2) && test -s conftest${ac_exeext}; then
>                  EXPAND_LIBS_LIST_STYLE=list
>              else
>-                 EXPAND_LIBS_LIST_STYLE=none
>+                 EXPAND_LIBS_LIST_STYLE=list  <-- I know that this is trash but yet I couldn't find a way to get gcc convinced to use emxomfld in this test, it takes always ld.exe that in turn fails due to the -Zomf CFLAG
>              fi
>          fi
>      else
>          dnl We really don't expect to get here, but just in case
>          AC_ERROR([couldn't compile a simple C file])
>      fi
>      rm -rf conftest*])
> 
>diff --git a/js/src/Makefile.in b/js/src/Makefile.in
>--- a/js/src/Makefile.in
>+++ b/js/src/Makefile.in
>@@ -448,17 +448,17 @@ CPPSRCS += \
> 
> LOCAL_INCLUDES = \
>     -Ictypes/libffi/include \
>     -I. \
>     $(NULL)
> 
> ifeq ($(OS_ARCH),OS2)
> SHARED_LIBRARY_LIBS += \
>-    ctypes/libffi/.libs/ffi.a \
>+    ctypes/libffi/.libs/ffi.$(LIB_SUFFIX) \
>     $(NULL)
> else
> SHARED_LIBRARY_LIBS += \
>     ctypes/libffi/.libs/libffi.$(LIB_SUFFIX) \
>     $(NULL)
> endif
> 
> endif # JS_HAS_CTYPES
>diff --git a/js/src/config/rules.mk b/js/src/config/rules.mk
>--- a/js/src/config/rules.mk
>+++ b/js/src/config/rules.mk
>@@ -1124,17 +1124,17 @@ endif
> $(filter-out %.$(LIB_SUFFIX),$(LIBRARY)): $(filter %.$(LIB_SUFFIX),$(LIBRARY)) $(OBJS) $(LOBJS) $(SHARED_LIBRARY_LIBS_DEPS) $(EXTRA_DEPS) $(GLOBAL_DEPS)
> 	$(EXPAND_LIBS_GEN) $(OBJS) $(LOBJS) $(SHARED_LIBRARY_LIBS) > $@
> 
> ifeq (,$(filter-out WINNT WINCE, $(OS_ARCH)))
> $(IMPORT_LIBRARY): $(SHARED_LIBRARY)
> endif
> 
> ifeq ($(OS_ARCH),OS2)
>-$(DEF_FILE): $(OBJS) $(SHARED_LIBRARY_LIBS)
>+$(DEF_FILE): $(OBJS) $(SHARED_LIBRARY_LIBS_DEPS)
> 	rm -f $@
> 	echo LIBRARY $(SHARED_LIBRARY_NAME) INITINSTANCE TERMINSTANCE > $@
> 	echo PROTMODE >> $@
> 	echo CODE    LOADONCALL MOVEABLE DISCARDABLE >> $@
> 	echo DATA    PRELOAD MOVEABLE MULTIPLE NONSHARED >> $@
> 	echo EXPORTS >> $@
> 
> 	$(ADD_TO_DEF_FILE)
>diff --git a/js/src/configure.in b/js/src/configure.in
>--- a/js/src/configure.in
>+++ b/js/src/configure.in
>@@ -2533,16 +2533,17 @@ ia64*-hpux*)
>     LIB_PREFIX=
>     LIB_SUFFIX=lib
>     BIN_SUFFIX=".exe"
>     DLL_SUFFIX=".dll"
>     IMPORT_LIB_SUFFIX=lib
>     DSO_PIC_CFLAGS=
>     AR=emxomfar
>     AR_FLAGS='r $@'
>+    AR_EXTRACT=
>     CFLAGS="$CFLAGS -Zomf"
>     CXXFLAGS="$CXXFLAGS -Zomf"
>     DSO_LDOPTS='-Zdll'
>     BIN_FLAGS='-Zlinker /ST:0x100000'
>     IMPLIB='emximp -o'
>     FILTER='true'
>     LDFLAGS='-Zmap'
>     WARNINGS_AS_ERRORS='-Werror'
>@@ -5413,17 +5414,17 @@ AC_CACHE_CHECK(what kind of list files a
>          echo "INPUT(conftest.${OBJ_SUFFIX})" > conftest.list
>          if AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS conftest.list $LIBS 1>&2) && test -s conftest${ac_exeext}; then
>              EXPAND_LIBS_LIST_STYLE=linkerscript
>          else
>              echo "conftest.${OBJ_SUFFIX}" > conftest.list
>              if AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS @conftest.list $LIBS 1>&2) && test -s conftest${ac_exeext}; then
>                  EXPAND_LIBS_LIST_STYLE=list
>              else
>-                 EXPAND_LIBS_LIST_STYLE=none
>+                 EXPAND_LIBS_LIST_STYLE=list
>              fi
>          fi
>      else
>          dnl We really don't expect to get here, but just in case
>          AC_ERROR([couldn't compile a simple C file])
>      fi
>      rm -rf conftest*])
> 
>diff --git a/js/src/ctypes/libffi/configure b/js/src/ctypes/libffi/configure
>old mode 100755
>new mode 100644
>--- a/js/src/ctypes/libffi/configure
>+++ b/js/src/ctypes/libffi/configure
>@@ -9138,17 +9138,17 @@ rm -f core conftest.err conftest.$ac_obj
>       fi
>       ;;
> 
>     os2*)
>       hardcode_libdir_flag_spec='-L$libdir'
>       hardcode_minus_L=yes
>       allow_undefined_flag=unsupported
>       archive_cmds='$ECHO "LIBRARY $libname INITINSTANCE" > $output_objdir/$libname.def~$ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~$ECHO DATA >> $output_objdir/$libname.def~$ECHO " SINGLE NONSHARED" >> $output_objdir/$libname.def~$ECHO EXPORTS >> $output_objdir/$libname.def~emxexp $libobjs >> $output_objdir/$libname.def~$CC -Zdll -Zcrtdll -o $lib $libobjs $deplibs $compiler_flags $output_objdir/$libname.def'
>-      old_archive_from_new_cmds='emximp -o $output_objdir/$libname.a $output_objdir/$libname.def'
>+      old_archive_from_new_cmds='emximp -o $output_objdir/$libname.${libext} $output_objdir/$libname.def'
>       ;;
> 
>     osf3*)
>       if test "$GCC" = yes; then
> 	allow_undefined_flag=' ${wl}-expect_unresolved ${wl}\*'
> 	archive_cmds='$CC -shared${allow_undefined_flag} $libobjs $deplibs $compiler_flags ${wl}-soname ${wl}$soname `test -n "$verstring" && $ECHO "X${wl}-set_version ${wl}$verstring" | $Xsed` ${wl}-update_registry ${wl}${output_objdir}/so_locations -o $lib'
>       else
> 	allow_undefined_flag=' -expect_unresolved \*'
>@@ -10096,20 +10096,23 @@ openbsd*)
> 	;;
>       esac
>   else
>     shlibpath_overrides_runpath=yes
>   fi
>   ;;
> 
> os2*)
>+  if test "$AR" = emxomfar; then
>+    libext=lib
>+  fi
>   libname_spec='$name'
>   shrext_cmds=".dll"
>   need_lib_prefix=no
>-  library_names_spec='$libname${shared_ext} $libname.a'
>+  library_names_spec='$libname${shared_ext} $libname.${libext}'
>   dynamic_linker='OS/2 ld.exe'
>   shlibpath_var=LIBPATH
>   ;;
> 
> osf3* | osf4* | osf5*)
>   version_type=osf
>   need_lib_prefix=no
>   need_version=no

Comment 3

[Walter Meinl]

Using this patch I get when linking xul.dll:
> g++: error trying to exec 'emxomfld.exe': execvp: Invalid argument
indicating that the input list is too long for emxomfld

Comment 4

[Mike Hommey [:glandium]]

Comment on attachment 524282 [details] [diff] [review]
WIP - fails to link xul.dll

>--- a/configure.in
>+++ b/configure.in
>@@ -2639,16 +2639,17 @@ ia64*-hpux*)
>     LIB_PREFIX=
>     LIB_SUFFIX=lib
>     BIN_SUFFIX=".exe"
>     DLL_SUFFIX=".dll"
>     IMPORT_LIB_SUFFIX=lib
>     DSO_PIC_CFLAGS=
>     AR=emxomfar
>     AR_FLAGS='r $@'
>+    AR_EXTRACT=

Doesn't emxomfar work like ar ?

>              if AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS @conftest.list $LIBS 1>&2) && test -s conftest${ac_exeext}; then
>                  EXPAND_LIBS_LIST_STYLE=list
>              else
>-                 EXPAND_LIBS_LIST_STYLE=none
>+                 EXPAND_LIBS_LIST_STYLE=list
>              fi

This is just plain wrong. Maybe there's something wrong with the AC_TRY_COMMAND, but if that command doesn't work, it is *meant* to set EXPAND_LIBS_LIST_STYLE to none, because it means the "@list" syntax can't be used.

Anyways, for ffi, don't you actually "only" need to change the default LIB_SUFFIX to "a" instead of "lib" ?

Comment 5

[Mike Hommey [:glandium]]

(In reply to comment #3)
> After fixing this, the build breaks when expandlibs_exec.py tries to remove a
> (not closed) temporary file. The reason for that is that the configure check
> for the EXPAND_LIBS_LIST macro doesn't work properly and the macro is set to
> "none".

Which is expected, as said in my previous comment. What you want is this:

diff --git a/config/expandlibs_exec.py b/config/expandlibs_exec.py
--- a/config/expandlibs_exec.py
+++ b/config/expandlibs_exec.py
@@ -114,8 +114,9 @@ class ExpandArgsMore(ExpandArgs):
         elif conf.EXPAND_LIBS_LIST_STYLE == "list":
             content = ["%s\n" % obj for obj in objs]
             ref = "@" + tmp
         else:
+            os.close(fd)
             os.remove(tmp)
             return
         self.tmp.append(tmp)
         f = os.fdopen(fd, "w")

Comment 6

[Walter Meinl]

(In reply to comment #4)
> Comment on attachment 524282 [details] [diff] [review]
> WIP - fails to link xul.dll
> 
> >--- a/configure.in
> >+++ b/configure.in
> >@@ -2639,16 +2639,17 @@ ia64*-hpux*)
> >     LIB_PREFIX=
> >     LIB_SUFFIX=lib
> >     BIN_SUFFIX=".exe"
> >     DLL_SUFFIX=".dll"
> >     IMPORT_LIB_SUFFIX=lib
> >     DSO_PIC_CFLAGS=
> >     AR=emxomfar
> >     AR_FLAGS='r $@'
> >+    AR_EXTRACT=
> 
> Doesn't emxomfar work like ar ?
> 
> >              if AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS @conftest.list $LIBS 1>&2) && test -s conftest${ac_exeext}; then
> >                  EXPAND_LIBS_LIST_STYLE=list
> >              else
> >-                 EXPAND_LIBS_LIST_STYLE=none
> >+                 EXPAND_LIBS_LIST_STYLE=list
> >              fi
> 
> This is just plain wrong. Maybe there's something wrong with the
> AC_TRY_COMMAND, but if that command doesn't work, it is *meant* to set
> EXPAND_LIBS_LIST_STYLE to none, because it means the "@list" syntax can't be
> used.
I know that it is wrong and it _will_ be changed in the final patch. However the problem here is that the AC_TRY_COMMAND invokes ld.exe instead of emxomfld.exe. The latter is needed to get a properly linked file since g++ builds conftest.o with the -Zomf flag. ld.exe cannot link OMF objects and therefore it fails. When I don't use -Zomf then the macro works with ld and @list file is chosen correctly. Maybe Dave has an idea how we can convince g++ at this point to call emxomfld instead of ld.
However, for the purpose to see how far we can come with using a list file at all, I forced it for now to use a list.

> Anyways, for ffi, don't you actually "only" need to change the default
> LIB_SUFFIX to "a" instead of "lib" ?
Yes, but since the sources were imported and have their own libtool based configure we faced the problem that the standalone build would build a.out formatted objects and thus creates an .a archive. We didn't want to change too much in the upstream sources and decided to take the .a extension even though the library is archived by emxomfar (which we could pass from the mozilla main configure). Unfortunately, it was impossible to pass $libext=lib as well from mozilla/configure to libffi/configure. Therefore, we will have now to change some lines in libffi/configure to build an archive with the correct lib extension.

Comment 7

[Walter Meinl]

(In reply to comment #5)
> (In reply to comment #3)
> > After fixing this, the build breaks when expandlibs_exec.py tries to remove a
> > (not closed) temporary file. The reason for that is that the configure check
> > for the EXPAND_LIBS_LIST macro doesn't work properly and the macro is set to
> > "none".
> 
> Which is expected, as said in my previous comment. What you want is this:
> 
> diff --git a/config/expandlibs_exec.py b/config/expandlibs_exec.py
> --- a/config/expandlibs_exec.py
> +++ b/config/expandlibs_exec.py
> @@ -114,8 +114,9 @@ class ExpandArgsMore(ExpandArgs):
>          elif conf.EXPAND_LIBS_LIST_STYLE == "list":
>              content = ["%s\n" % obj for obj in objs]
>              ref = "@" + tmp
>          else:
> +            os.close(fd)
>              os.remove(tmp)
>              return
>          self.tmp.append(tmp)
>          f = os.fdopen(fd, "w")

We tested this also, and, yes, it works. However, when linking xul.dll without a listfile at all I get a subprocess error 22 invalid argument (yet I couldn't find out which argument is invalid)

Comment 8

[Mike Hommey [:glandium]]

(In reply to comment #6)
> (In reply to comment #4)
> > Comment on attachment 524282 [details] [diff] [review]
> > WIP - fails to link xul.dll
> > 
> > >--- a/configure.in
> > >+++ b/configure.in
> > >@@ -2639,16 +2639,17 @@ ia64*-hpux*)
> > >     LIB_PREFIX=
> > >     LIB_SUFFIX=lib
> > >     BIN_SUFFIX=".exe"
> > >     DLL_SUFFIX=".dll"
> > >     IMPORT_LIB_SUFFIX=lib
> > >     DSO_PIC_CFLAGS=
> > >     AR=emxomfar
> > >     AR_FLAGS='r $@'
> > >+    AR_EXTRACT=
> > 
> > Doesn't emxomfar work like ar ?
> > 
> > >              if AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS @conftest.list $LIBS 1>&2) && test -s conftest${ac_exeext}; then
> > >                  EXPAND_LIBS_LIST_STYLE=list
> > >              else
> > >-                 EXPAND_LIBS_LIST_STYLE=none
> > >+                 EXPAND_LIBS_LIST_STYLE=list
> > >              fi
> > 
> > This is just plain wrong. Maybe there's something wrong with the
> > AC_TRY_COMMAND, but if that command doesn't work, it is *meant* to set
> > EXPAND_LIBS_LIST_STYLE to none, because it means the "@list" syntax can't be
> > used.
> I know that it is wrong and it _will_ be changed in the final patch. However
> the problem here is that the AC_TRY_COMMAND invokes ld.exe instead of
> emxomfld.exe. The latter is needed to get a properly linked file since g++
> builds conftest.o with the -Zomf flag. ld.exe cannot link OMF objects and
> therefore it fails. When I don't use -Zomf then the macro works with ld and
> @list file is chosen correctly. Maybe Dave has an idea how we can convince g++
> at this point to call emxomfld instead of ld.

It probably needs -Zomf as well, and I see that MKSHLIB adds CXXFLAGS... so maybe that's all it'd take: add CXXFLAGS to the AC_TRY_COMMAND. (and check it doesn't break other platforms)

> However, for the purpose to see how far we can come with using a list file at
> all, I forced it for now to use a list.
> 
> > Anyways, for ffi, don't you actually "only" need to change the default
> > LIB_SUFFIX to "a" instead of "lib" ?
> Yes, but since the sources were imported and have their own libtool based
> configure we faced the problem that the standalone build would build a.out
> formatted objects and thus creates an .a archive. We didn't want to change too
> much in the upstream sources and decided to take the .a extension even though
> the library is archived by emxomfar (which we could pass from the mozilla main
> configure). Unfortunately, it was impossible to pass $libext=lib as well from
> mozilla/configure to libffi/configure. Therefore, we will have now to change
> some lines in libffi/configure to build an archive with the correct lib
> extension.

My question was about the other way around: why not change the lib suffix in mozilla to be "a" instead of "lib" ?

Comment 9

[Dave Yeo]

(In reply to comment #8)

> 
> It probably needs -Zomf as well, and I see that MKSHLIB adds CXXFLAGS... so
> maybe that's all it'd take: add CXXFLAGS to the AC_TRY_COMMAND. (and check it
> doesn't break other platforms)
> 

Better to add it to LDFLAGS as the AC_TRY_COMMAND is already using LDFLAGS and it seems to work here.
Unluckily with ar_extract= commented out the build dies early with unresolved symbols from libfii which I take as a failure to extract the object files from libfii.
Testing emxomfar x fii.lib works fine except the object files have a .obj suffix instead of .o

> 
> My question was about the other way around: why not change the lib suffix in
> mozilla to be "a" instead of "lib" ?

I tried this a couple of years ago. I don't remember the specifics but IIRC there were quite a few hardcoded uses of .lib in nspr.
Our toolchain automatically uses .lib for OMF so really the proper way is to remove -Zomf from C[XX]FLAGS and add it to LDFLAGS and change $AR to ar

Comment 10

[Walter Meinl]

(In reply to comment #9)

> Our toolchain automatically uses .lib for OMF so really the proper way is to
> remove -Zomf from C[XX]FLAGS and add it to LDFLAGS and change $AR to ar

Tried this one, passed $AR=ar to libffi configure (passed no LDFLAGS to get ffi.a in a.out format for linking later), renamed ffi.a to ffi.lib.
js_static.lib was build (and probably converted to omf during linkage) but then I got when linking js.exe
emxomf: Input file `../js_static.lib(ffi.lib)' is not an a.out file
emxomfld: a.out to omf conversion failed for '../js_static.lib'.
The same error occurred when I tried to link ffi.a.

BTW, does LD.EXE has also limitations in the maximum of lines passed over?

Comment 11

[Mike Hommey [:glandium]]

(In reply to comment #10)
> (In reply to comment #9)
> 
> > Our toolchain automatically uses .lib for OMF so really the proper way is to
> > remove -Zomf from C[XX]FLAGS and add it to LDFLAGS and change $AR to ar
> 
> Tried this one, passed $AR=ar to libffi configure (passed no LDFLAGS to get
> ffi.a in a.out format for linking later), renamed ffi.a to ffi.lib.
> js_static.lib was build (and probably converted to omf during linkage) but then
> I got when linking js.exe
> emxomf: Input file `../js_static.lib(ffi.lib)' is not an a.out file
> emxomfld: a.out to omf conversion failed for '../js_static.lib'.
> The same error occurred when I tried to link ffi.a.

It looks like js_static.lib literally contains ffi.lib, instead of the extracted objects.

Comment 12

[Mike Hommey [:glandium]]

(In reply to comment #11)
> It looks like js_static.lib literally contains ffi.lib, instead of the
> extracted objects.

(which suggests AR_EXTRACT was not used)

Comment 13

[Dave Yeo]

(In reply to comment #10)
> (In reply to comment #9)
> 
> > Our toolchain automatically uses .lib for OMF so really the proper way is to
> > remove -Zomf from C[XX]FLAGS and add it to LDFLAGS and change $AR to ar
> 
> Tried this one, passed $AR=ar to libffi configure (passed no LDFLAGS to get
> ffi.a in a.out format for linking later), renamed ffi.a to ffi.lib.

Actually I meant changing the whole build system. Doing it like you did seems to be hit and miss whether the toolchain gets the formats right and does the correct a.out to OMF conversion.

> js_static.lib was build (and probably converted to omf during linkage) but then
> I got when linking js.exe
> emxomf: Input file `../js_static.lib(ffi.lib)' is not an a.out file
> emxomfld: a.out to omf conversion failed for '../js_static.lib'.
> The same error occurred when I tried to link ffi.a.
> 
> BTW, does LD.EXE has also limitations in the maximum of lines passed over?

As far as I can see emxomfld will only fail with out of memory error if the response file is too big.

Comment 14

[Walter Meinl]

Attachment: errors when linking xul.dll

(In reply to comment #13)
> (In reply to comment #10)
> > (In reply to comment #9)
> > 

> 
> Actually I meant changing the whole build system. Doing it like you did seems
> to be hit and miss whether the toolchain gets the formats right and does the
> correct a.out to OMF conversion.
Yes, after several tries it worked setting -Zomf in the LDFLAGS, it still needs to build a ffi.lib then the extraction works as well (dunno what was wrong in my last try).
> > BTW, does LD.EXE has also limitations in the maximum of lines passed over?
> 
> As far as I can see emxomfld will only fail with out of memory error if the
> response file is too big.
Nevertheless, even when changing the whole build system to use ar.exe and ld.exe and do the OMF conversion during link time, I see still the same error as described in comment #3

The attachment contains the log when doing 
make -C $objdir/toolkit/library
To reach maximal verbosity I added -v to $CXXFLAGS and --verbose at the end of EXPAND_LIBS_EXEC in config.mk.

Comment 15

[Justin Wood (:Callek)]

Comment on attachment 524282 [details] [diff] [review]
WIP - fails to link xul.dll

Please be sure you do NOT change the chmod on the libffi configure (and other files)

Comment 16

[Dave Yeo]

Seems we've run into a toolchain bug where g++ does not use a response file to call the linker. KNut says he hopes to find time to fix it over Easter.
http://svn.netlabs.org/libc/ticket/216

Comment 17

[Walter Meinl]

Attachment: WIP-v1

Still until the toolchain problems are fixed the better version of the patch fails to link xul.dll. However, I took several comments into account. In order to change nothing in the libffi configure, ar.exe is used to create ffi.a, ffi.a is then converted to omf with the correct "lib" LIB_SUFFIX by emxomf. Note I had to add +libs:: ctypes/libffi/.libs/ffi.$(LIB_SUFFIX) in the Makefile, because otherwise it would not be converted at the right moment.
I added $CFLAGS in configure for the determination of the EXPAND_LIBS_LIST_STYLE, this works for OS/2 and also for linux (I cannot test whether this breaks the determination on windows)

Comment 18

[Walter Meinl]

Attachment: final patch updated to apply cleanly to m-c (8.0a1)

the linking issues with XUL.DLL will be handled in a different bug

Comment 19

[Justin Wood (:Callek)]

http://hg.mozilla.org/integration/mozilla-inbound/rev/b6dfc500248d

Comment 20

[Justin Wood (:Callek)]

(In reply to Justin Wood (:Callek) from comment #19)
> http://hg.mozilla.org/integration/mozilla-inbound/rev/b6dfc500248d

...and backed out due to Windows burning.

Please do a try run before requesting checkin for next time.

Error On:
http://tinderbox.mozilla.org/showlog.cgi?log=Mozilla-Inbound/1312963862.1312966797.11770.gz

Comment 21

[Mike Hommey [:glandium]]

(In reply to Justin Wood (:Callek) from comment #20)
> Error On:
> http://tinderbox.mozilla.org/showlog.cgi?log=Mozilla-Inbound/1312963862.
> 1312966797.11770.gz

The python error message is misleading, the problem is that the command line either is too long or has too many arguments, which is due to:
"checking what kind of list files are supported by the linker... none"

The configure.in change made the test fail.

Comment 22

[Walter Meinl]

(In reply to Justin Wood (:Callek) from comment #20)
> (In reply to Justin Wood (:Callek) from comment #19)
> > http://hg.mozilla.org/integration/mozilla-inbound/rev/b6dfc500248d
> 
> ...and backed out due to Windows burning.
> 
> Please do a try run before requesting checkin for next time.
Sorry, this was not to be expected
(In reply to Mike Hommey [:glandium] from comment #21)
> (In reply to Justin Wood (:Callek) from comment #20)
> > Error On:
> > http://tinderbox.mozilla.org/showlog.cgi?log=Mozilla-Inbound/1312963862.
> > 1312966797.11770.gz
> 
> The python error message is misleading, the problem is that the command line
> either is too long or has too many arguments, which is due to:
> "checking what kind of list files are supported by the linker... none"
> 
> The configure.in change made the test fail.
That's odd: Windows chokes on $CFLAGS $LDFLAGS and OS/2 on $LDFLAGS alone - with the same result that EXPAND_LIBS_LIST_STYLE = none is chosen and linking of XUL.DLL is then impossible.

I see 4 possible solutions for the lines in question:
1) (not very likely) how important is the usage of $CFLAGS and $CPPFLAGS in the very first test? If they could be skipped we would be happy too (then LDFLAGS alone would work)

2) Save LDFLAGS for OS/2, add -Zomf to LDFLAGS for the test, Set LDFLAGS to their saved state after running the test - a lot of lines for a little goal...

3) duplicate the test line 
> if (AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS @conftest.list $LIBS 1>&2) ||
>     AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $CFLAGS $LDFLAGS @conftest.list $LIBS 1>&2)) &&  -s conftest${ac_exeext}; then 

4) duplicate test and result line, use elif
>  if AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $LDFLAGS @conftest.list $LIBS 1>&2) -s conftest${ac_exeext}; then
>      EXPAND_LIBS_LIST_STYLE=list
>  elif AC_TRY_COMMAND(${CC-cc} -o conftest${ac_exeext} $CFLAGS $LDFLAGS @conftest.list $LIBS 1>&2) -s conftest${ac_exeext}; then
>      EXPAND_LIBS_LIST_STYLE=list
> else

If 1) is not an option, I'd suggest 3)

Comment 23

[Mike Hommey [:glandium]]

Why exactly is -Zomf *not* in LDFLAGS ?

Comment 24

[Walter Meinl]

(In reply to Mike Hommey [:glandium] from comment #23)
> Why exactly is -Zomf *not* in LDFLAGS ?

It's a switch for gcc to call emxomfld (and not the default a.out ld.exe) for linking. emxomfld can not handle -Zomf. Thus, if we're going to use emxomfld directly for linking xul with a list file, emxomfld will choke on the presence of -Zomf.

Comment 25

[Mike Hommey [:glandium]]

So why aren't we linking with gcc instead of emxomfld?

Comment 26

[Walter Meinl]

(In reply to Mike Hommey [:glandium] from comment #25)
> So why aren't we linking with gcc instead of emxomfld?

The problem is that our gcc fails to handle over correctly the list file (due to it's size) created for xul.dll. Calling emxomfld directly for linking works with the list file.

Comment 27

[Wayne Mery (:wsmwk)]

OS/2 is no longer a supported platform.