Fix clang builds on OpenBSD

RESOLVED FIXED in mozilla18

Status

defect
RESOLVED FIXED
7 years ago
Last year

People

(Reporter: gaston, Assigned: gaston)

Tracking

Trunk
mozilla18
All
OpenBSD
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(5 attachments, 3 obsolete attachments)

Assignee

Description

7 years ago
Since osx builds switched to clang, openbsd is one of the last still using the ancient-and-soon-to-be-unsupported gcc 4.2.1. Let's try to switch to clang (3.1 here) !

After a hint from espindola, i've added two missing headers to system-headers to fix a visibility problem (libxul failed to link and was complaining about missing syms, coming from the aforementioned headers).

Added to that a TLS configure check hack (OpenBSD doesn't have TLS support yet), for some subtle reason the following code (from http://mxr.mozilla.org/mozilla-central/source/configure.in#3641) :

__thread bool tlsIsMainThread = false;
int main() {
return tlsIsMainThread;
}

compiles with clang++ (but fails to run!), but fails with g++.

[12:27] sunrise:/tmp/ $g++ t.C
t.C:1: error: thread-local storage not supported for this target
[12:27] sunrise:/tmp/ $clang++ t.C
[12:27] sunrise:/tmp/$./a.out
./a.out: unsupported TLS program header

Dunno if that's an issue in our clang port, but since the test succeeds ac_cv_thread_keyword is yes, and MOZ_TLS ends up with 1. Lets declare that openbsd fully doesnt have TLS at the moment.
Assignee

Comment 1

7 years ago
This patch allows me to build m-c with clang on OpenBSD/i386, and it even runs!
Assignee: nobody → landry
Attachment #646831 - Flags: review?(mh+mozilla)
Assignee

Comment 2

7 years ago
Now, a more twisted issue on OpenBSD/amd64 (on OpenBSD/ppc clang fails to build nsinstall.c -ie segfaults hard-, although it builds all the configure tests, so i'll investigate on our side for clang/ppc. Work for later.)

The failure happens when linking js shell against libffi.a (that's not https://bugzilla.mozilla.org/show_bug.cgi?id=631928) :

../libjs_static.a(ffi64.o)(.text+0x561): In function `ffi_call':
/home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c:485: undefined reference to `ffi_call_unix64'
../libjs_static.a(ffi64.o)(.text+0x5c8): In function `ffi_prep_closure_loc':
/home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c:504: undefined reference to `ffi_closure_unix64'
clang-3: error: linker command failed with exit code 1 (use -v to see invocation)

If i look at libffi.a, unix64.S is correctly built and linked in the lib:

libtool: compile:  clang -DHAVE_CONFIG_H -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src -MT src/x86/unix64.lo -MD -MP -MF src/x86/.deps/unix64.Tpo -c /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S  -fPIC -DPIC -o src/x86/unix64.o
/bin/sh ./libtool --tag=CC   --mode=link clang -Wall -g -fexceptions  -O2  -version-info `grep -v '^#' /home/landry/src/mozilla-central/js/src/ctypes/libffi/libtool-version`   -pthread -Wl,--no-keep-memory -o libffi.la -rpath /usr/local/lib src/debug.lo src/prep_cif.lo src/types.lo src/raw_api.lo src/java_raw_api.lo src/closures.lo                      src/x86/ffi64.lo src/x86/unix64.lo src/x86/ffi.lo src/x86/sysv.lo      
libtool: link: rm -fr  .libs/libffi.a .libs/libffi.la .libs/libffi.lai
libtool: link: ar cru .libs/libffi.a  src/debug.o src/prep_cif.o src/types.o src/raw_api.o src/java_raw_api.o src/closures.o src/x86/ffi64.o src/x86/unix64.o src/x86/ffi.o src/x86/sysv.o

attaching readelf -a -W / nm -a output, not sure but everything looks here :

File: /usr/obj/m-c/js/src/ctypes/libffi/.libs/libffi.a(unix64.o)
....
     5: 0000000000000000   366 FUNC    GLOBAL DEFAULT    1 ffi_call_unix64
     6: 000000000000016e   314 FUNC    GLOBAL DEFAULT    1 ffi_closure_unix64

Of course if i build with gcc js shell links fine against libffi. The only other occurence of that failure i've found was in a freebsd 7/amd64/gcc 4.6 build, see http://tb.smeets.im/tb/errors/7-FreeBSD/firefox-13.0,1.log
Assignee

Comment 4

7 years ago
More information to narrow down the issue.. on 32 bits, sysv.S is used in ffi.c and ffi_call_SYSV is called (fine apparently). on 64 bits, both sysv.S and unix64.S are used but with the #ifndef at the top of sysv.S it ends up empty.

Interestingly, the 'undefined ref' issue shows up with the gnu libtool script provided/created  in top_builddir/libtool. If i use OpenBSD's /usr/bin/libtool (a perl rewrite aimed at simplifying the maze) the failure is not exactly the same, libffi fails to link :

/usr/bin/libtool --tag=CC   --mode=link clang -Wall -g -fexceptions  -O2  -version-info `grep -v '^#' /home/landry/src/mozilla-central/js/src/ctypes/libffi/libtool-version`   -pthread -Wl,--no-keep-memory -o libffi.la -rpath /usr/local/lib src/debug.lo src/prep_cif.lo src/types.lo src/raw_api.lo src/java_raw_api.lo src/closures.lo                      src/x86/ffi64.lo src/x86/unix64.lo src/x86/ffi.lo src/x86/sysv.lo      
libtool: link: clang -shared -fPIC -DPIC -o .libs/libffi.so.5.10 -pthread -Wall -g -fexceptions -O2 -Wl,--no-keep-memory src/.libs/debug.o src/.libs/prep_cif.o src/.libs/types.o src/.libs/raw_api.o src/.libs/java_raw_api.o src/.libs/closures.o src/x86/.libs/ffi64.o src/x86/.libs/unix64.o src/x86/.libs/ffi.o src/x86/.libs/sysv.o
clang-3: warning: argument unused during compilation: '-g'
src/x86/.libs/unix64.o: file not recognized: File format not recognized


But file/nm -a/readelf -a -W says me the object looks valid, like the other objects in the same dir:
/usr/obj/m-c/js/src/ctypes/libffi/src/x86/.libs/unix64.o: ELF 64-bit LSB relocatable, x86-64, version 1, not stripped

Istr gnu libtool silently ignores invalid object files, while our libtool errors our on them, so this might be related to the issue. Wild guess, for some reason, clang compiles unix64.S as an object that doesnt please clang itself when linking, gnu libtool silently discards it from the link line, and our libtool barfs on it.
Assignee

Comment 5

7 years ago
Another sidenote/data point: libffi 3.0.9 from our portstree builds with clang and runs its regress tests fine.
Assignee

Comment 6

7 years ago
And if i compare the output of readelf -a -W on libffi.a 3.0.9 built with clang within our portstree (so with our libtool) and the libffi.a from js/ctypes built with clang (and gnu libtool), i can see a difference in the .rela.text section for ffi64.o :

on the latter,  ffi_call_unix64 is of type R_X86_64_PLT32 and ffi_closure_unix64 is of type R_X86_64_GOTPCREL. That one is the failing one.

A 'working one' has ffi_call_unix64 as R_X86_64_PC32 and ffi_closure_unix64 as R_X86_64_32S. Dunno if that matters since this is all cryptic to me, but that's a difference.
Assignee

Comment 7

7 years ago
With --enable-system-ffi, m-c builds fine wth clang on OpenBSD/amd64. So the problem can be narrowed down to bundled libffi build...
Attachment #646831 - Flags: review?(mh+mozilla) → review+
(In reply to Landry Breuil (:gaston) from comment #0)
> Dunno if that's an issue in our clang port.

It should probably fail to accept __thread if there is no underlying TLS support.
(In reply to Landry Breuil (:gaston) from comment #6)
> And if i compare the output of readelf -a -W on libffi.a 3.0.9 built with
> clang within our portstree (so with our libtool) and the libffi.a from
> js/ctypes built with clang (and gnu libtool), i can see a difference in the
> .rela.text section for ffi64.o :
> 
> on the latter,  ffi_call_unix64 is of type R_X86_64_PLT32 and
> ffi_closure_unix64 is of type R_X86_64_GOTPCREL. That one is the failing one.
> 
> A 'working one' has ffi_call_unix64 as R_X86_64_PC32 and ffi_closure_unix64
> as R_X86_64_32S. Dunno if that matters since this is all cryptic to me, but
> that's a difference.

Smells like a visibility issue.
(In reply to Mike Hommey [:glandium] from comment #8)
> (In reply to Landry Breuil (:gaston) from comment #0)
> > Dunno if that's an issue in our clang port.
> 
> It should probably fail to accept __thread if there is no underlying TLS
> support.


I agree. Gastor, would you mind reporting a bug in llvm.org/bugs?
> Smells like a visibility issue.

Note that the ffi build system integration is not all that good. When running make from the top level directory I get:

$ gcc -DHAVE_CONFIG_H -I. -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi -I. -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/src -Wall -g -fexceptions -O2 -MT src/x86/ffi64.lo -MD -MP -MF src/x86/.deps/ffi64.Tpo -c /home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c  -fPIC -DPIC -o src/x86/ffi64.o

which produces:

readelf  -rW src/x86/ffi64.o  | grep ffi_call_u
0000000000000826  0000001600000004 R_X86_64_PLT32         0000000000000000 ffi_call_unix64 - 4

If I just run "make  src/x86/ffi64.o" I get

gcc -DHAVE_CONFIG_H -I. -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi  -I. -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/src  -Wall -g -fexceptions  -O2  -MT src/x86/ffi64.o -MD -MP -MF $depbase.Tpo -c -o src/x86/ffi64.o /home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c

and without the -fPIC we get:

readelf  -rW src/x86/ffi64.o  | grep ffi_call_u
0000000000000816  0000001500000002 R_X86_64_PC32          0000000000000000 ffi_call_unix64 - 4

Do you see a -fPIC when building from the top level?
Assignee

Comment 13

7 years ago
(In reply to Rafael Ávila de Espíndola (:espindola) from comment #11)
> > Smells like a visibility issue.
> 
> Note that the ffi build system integration is not all that good. When
> running make from the top level directory I get:
> 
> $ gcc -DHAVE_CONFIG_H -I.
> -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi -I.
> -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/include
> -Iinclude -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/src
> -Wall -g -fexceptions -O2 -MT src/x86/ffi64.lo -MD -MP -MF
> src/x86/.deps/ffi64.Tpo -c
> /home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c
> -fPIC -DPIC -o src/x86/ffi64.o
> 
> which produces:
> 
> readelf  -rW src/x86/ffi64.o  | grep ffi_call_u
> 0000000000000826  0000001600000004 R_X86_64_PLT32         0000000000000000
> ffi_call_unix64 - 4
> 
> If I just run "make  src/x86/ffi64.o" I get
> 
> gcc -DHAVE_CONFIG_H -I.
> -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi  -I.
> -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/include
> -Iinclude -I/home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/src
> -Wall -g -fexceptions  -O2  -MT src/x86/ffi64.o -MD -MP -MF $depbase.Tpo -c
> -o src/x86/ffi64.o
> /home/espindola/mozilla/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c
> 
> and without the -fPIC we get:
> 
> readelf  -rW src/x86/ffi64.o  | grep ffi_call_u
> 0000000000000816  0000001500000002 R_X86_64_PC32          0000000000000000
> ffi_call_unix64 - 4
> 
> Do you see a -fPIC when building from the top level?

I see the same thing (ie -fPIC when in a full build, and no -fPIC when building only that file). I have the impression that -fPIC -DPIC is added by libtool invocation, which doesnt seem used when building only a single file. The working gcc builds also have -fPIC -DPIC on the build commands.
> I see the same thing (ie -fPIC when in a full build, and no -fPIC when
> building only that file). I have the impression that -fPIC -DPIC is added by
> libtool invocation, which doesnt seem used when building only a single file.
> The working gcc builds also have -fPIC -DPIC on the build commands.

And you get different relocations when you use gcc or clang? Strange, on linux the difference is only if -fPIC is used or not.
Whiteboard: [leave open]
Assignee

Comment 16

7 years ago
Build with clang from the top dir : (ie -fPIC -DPIC)

ffi64.o has
    Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
...
0000000000000561  0000001700000004 R_X86_64_PLT32         0000000000000000 ffi_call_unix64 + fffffffffffffffc
00000000000005c8  0000001800000009 R_X86_64_GOTPCREL      0000000000000000 ffi_closure_unix64 + fffffffffffffffc

unix64.o has
Symbol table '.symtab' contains 9 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
...
     5: 0000000000000000   366 FUNC    GLOBAL DEFAULT    1 ffi_call_unix64
     6: 000000000000016e   314 FUNC    GLOBAL DEFAULT    1 ffi_closure_unix64


If i try to 'manually' build ffi64.o and unix64.o (ie make .o, so no libtool invocation and no -fPIC -DPIC) libffi.a links but the js shell fails to link against it differently (as if it lost all other libffi symbols)

../libjs_static.a(CTypes.o)(.text._ZN2js6ctypes12FunctionTypeL4CallEP9JSContextjPN2JS5ValueE+0x57d): In function `_ZN2js6ctypes
12FunctionTypeL4CallEP9JSContextjPN2JS5ValueE':
./../../dist/include/js/Vector.h:679: undefined reference to `ffi_call'
../libjs_static.a(CTypes.o)(.text._ZN2js6ctypes8CClosureL8FinalizeEP8JSFreeOpP8JSObject+0x35): In function `_ZN2js6ctypes8CClos
ureL8FinalizeEP8JSFreeOpP8JSObject':
./../../dist/include/js/Vector.h:581: undefined reference to `ffi_closure_free'
../libjs_static.a(CTypes.o)(.text._ZN2js6ctypes15InitTypeClassesEP9JSContextN2JS6HandleIP8JSObjectEE+0x8f7): In function `js::c
types::InitTypeClasses(JSContext*, JS::Handle<JSObject*>)':
/home/landry/src/mozilla-central/js/src/ctypes/typedefs.h:72: undefined reference to `ffi_type_uint8'
../libjs_static.a(CTypes.o)(.text._ZN2js6ctypes15InitTypeClassesEP9JSContextN2JS6HandleIP8JSObjectEE+0x941): In function `js::c
types::InitTypeClasses(JSContext*, JS::Handle<JSObject*>)':
/home/landry/src/mozilla-central/js/src/jsval.h:647: undefined reference to `ffi_type_uint8'

etc etc..


Now, when building with gcc 4.2.1, libtool still adds -fPIC -DPIC :

/bin/sh ./libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi  -I. -I/home/l
andry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src  -Wall -g -fexceptions  -O2  -MT src/x86/ffi64.lo -MD -MP -MF $depbase.Tpo -c -o src/x86/ffi64.lo /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src -Wall -g -fexceptions -O2 -MT src/x86/ffi64.lo -MD -MP -MF src/x86/.deps/ffi64.Tpo -c /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c  -fPIC -DPIC -o src/x86/ffi64.o
depbase=`echo src/x86/unix64.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ./libtool   --mode=compile gcc -DHAVE_CONFIG_H -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi  -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src  -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src  -MT src/x86/unix64.lo -MD -MP -MF $depbase.Tpo -c -o src/x86/unix64.lo /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi -I. -I/home/landry/src/mozilla-central/
js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src -MT src/x86/unix64.lo -MD -MP -MF src/x86/.deps/unix64.Tpo -c /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S  -fPIC -DPIC -o src/x86/unix64.o

ffi64.o has:

    Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
...
0000000000000393  0000001400000009 R_X86_64_GOTPCREL      0000000000000000 ffi_closure_unix64 + fffffffffffffffc
0000000000000c50  0000001c00000004 R_X86_64_PLT32         0000000000000000 ffi_call_unix64 + fffffffffffffffc

unix64.o has:
Symbol table '.symtab' contains 19 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
..
    15: 0000000000000000   366 FUNC    GLOBAL DEFAULT    1 ffi_call_unix64
    16: 000000000000016e   314 FUNC    GLOBAL DEFAULT    1 ffi_closure_unix64

And maybe it has its importance, but the symtab in unix64.o contains several entries like this:
     7: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S

which were not present when building with clang++.

Note that if i manually build ffi64.o/unix64.o with gcc (ie _without_ the libtool invocation, only gmake src/x86/ffi64.o) -fPIC -DPIC is not added either with gcc, and the relocation section for ffi64.o contains :

0000000000000b4d  0000001b00000002 R_X86_64_PC32          0000000000000000 ffi_call_unix64 + fffffffffffffffc
0000000000000349  000000130000000b R_X86_64_32S           0000000000000000 ffi_closure_unix64 + 0
Assignee

Comment 17

7 years ago
And to compare against the clang 3.1 log, here's the readelf -aW .libs/libffi.a when building with gcc 4.2.1.
I currently have no idea why the linker in giving you an undefined error :-(

If you run the linker with -t, does it say anything about using unix64.o?
Assignee

Comment 19

7 years ago
(In reply to Rafael Ávila de Espíndola (:espindola) from comment #18)
> I currently have no idea why the linker in giving you an undefined error :-(
> 
> If you run the linker with -t, does it say anything about using unix64.o?

You mean when linking libffi.a itself, or the js shell against libjs_static.a and libffi.a ?
> You mean when linking libffi.a itself, or the js shell against
> libjs_static.a and libffi.a ?

The one that gives you the error in comment 2.
Assignee

Comment 21

7 years ago
So, adding -v to the clang++ link line shows the ld call, and i've then added -Wl,-t so that -t gets passed to ld (full output) :

 "/usr/bin/ld" -e __start -export-dynamic --eh-frame-hdr -Bdynamic -dynamic-linker /usr/libexec/ld.so -o js /usr/lib/crt0.o /usr/lib/crtbegin.o -L/usr/lib/gcc-lib/amd64-unknown-openbsd5.2/4.2.1 -L../../../dist/bin -L../../../dist/lib -L/usr/obj/m-c/dist/lib /usr/obj/m-c/js/src/shell/tmpYe4vDb.list --no-keep-memory -rpath-link ../../../dist/bin -rpath-link /usr/obj/m-c/dist/lib -lplds4 -lplc4 -lnspr4 -lc ../libjs_static.a --whole-archive ../../../dist/lib/libmozglue.a --no-whole-archive -lstdc++ -lm -lgcc -lpthread -lc -lgcc /usr/lib/crtend.o

/usr/bin/ld: mode elf_x86_64_obsd
/usr/lib/crt0.o
/usr/lib/crtbegin.o
js.o
jsoptparse.o
jsheaptools.o
../editline/editline.o
../editline/sysunix.o
../../../modules/zlib/src/adler32.o
../../../modules/zlib/src/compress.o
../../../modules/zlib/src/crc32.o
../../../modules/zlib/src/deflate.o
../../../modules/zlib/src/gzclose.o
../../../modules/zlib/src/gzlib.o
../../../modules/zlib/src/gzread.o
../../../modules/zlib/src/gzwrite.o
../../../modules/zlib/src/infback.o
../../../modules/zlib/src/inffast.o
../../../modules/zlib/src/inflate.o
../../../modules/zlib/src/inftrees.o
../../../modules/zlib/src/trees.o
../../../modules/zlib/src/uncompr.o
../../../modules/zlib/src/zutil.o
-lplds4 (../../../dist/bin/libplds4.so.1.0)
-lplc4 (../../../dist/bin/libplc4.so.1.0)
-lnspr4 (../../../dist/bin/libnspr4.so.1.0)
-lc (/usr/lib/libc.so.65.0)
js.o(.text._ZL5SnarfP9JSContextjPN2JS5ValueE+0xe7): In function `_ZL7OptionsP9JSContextjPN2JS5ValueE':
/home/landry/src/mozilla-central/js/src/shell/js.cpp:1033: warning: strcpy() is almost always misused, please use strlcpy()
../../../modules/zlib/src/gzlib.o(.text.MOZ_Z_gz_error+0xac): In function `MOZ_Z_gz_error':
/home/landry/src/mozilla-central/modules/zlib/src/gzlib.c:87: warning: strcat() is almost always misused, please use strlcat()
../../../modules/zlib/src/gzlib.o(.text.MOZ_Z_gzdopen+0x2f): In function `MOZ_Z_gzdopen':
/home/landry/src/mozilla-central/modules/zlib/src/gzlib.c:322: warning: sprintf() is often misused, please use snprintf()
(../libjs_static.a)jsapi.o
(../libjs_static.a)jsarray.o
(../libjs_static.a)jsatom.o
(../libjs_static.a)jsbool.o
(../libjs_static.a)jsclone.o
(../libjs_static.a)jscntxt.o
(../libjs_static.a)jscompartment.o
(../libjs_static.a)jsdate.o
(../libjs_static.a)jsdbgapi.o
(../libjs_static.a)jsdtoa.o
(../libjs_static.a)jsexn.o
(../libjs_static.a)jsfriendapi.o
(../libjs_static.a)jsfun.o
(../libjs_static.a)jsgc.o
(../libjs_static.a)jsinfer.o
(../libjs_static.a)jsinterp.o
(../libjs_static.a)jsiter.o
(../libjs_static.a)jsmath.o
(../libjs_static.a)jsnativestack.o
(../libjs_static.a)jsnum.o
(../libjs_static.a)jsobj.o
(../libjs_static.a)json.o
(../libjs_static.a)jsonparser.o
(../libjs_static.a)jsopcode.o
(../libjs_static.a)jsproxy.o
(../libjs_static.a)jsprf.o
(../libjs_static.a)jsprobes.o
(../libjs_static.a)jspropertycache.o
(../libjs_static.a)jspropertytree.o
(../libjs_static.a)jsreflect.o
(../libjs_static.a)jsscope.o
(../libjs_static.a)jsscript.o
(../libjs_static.a)jsstr.o
(../libjs_static.a)jstypedarray.o
(../libjs_static.a)jswatchpoint.o
(../libjs_static.a)jsweakmap.o
(../libjs_static.a)jswrapper.o
(../libjs_static.a)jsxml.o
(../libjs_static.a)prmjtime.o
(../libjs_static.a)ArgumentsObject.o
(../libjs_static.a)ScopeObject.o
(../libjs_static.a)Debugger.o
(../libjs_static.a)GlobalObject.o
(../libjs_static.a)ObjectImpl.o
(../libjs_static.a)Stack.o
(../libjs_static.a)String.o
(../libjs_static.a)BytecodeCompiler.o
(../libjs_static.a)BytecodeEmitter.o
(../libjs_static.a)FoldConstants.o
(../libjs_static.a)ParseMaps.o
(../libjs_static.a)ParseNode.o
(../libjs_static.a)Parser.o
(../libjs_static.a)SemanticAnalysis.o
(../libjs_static.a)SPSProfiler.o
(../libjs_static.a)TokenStream.o
(../libjs_static.a)TreeContext.o
(../libjs_static.a)TestingFunctions.o
(../libjs_static.a)LifoAlloc.o
(../libjs_static.a)Eval.o
(../libjs_static.a)MapObject.o
(../libjs_static.a)RegExpObject.o
(../libjs_static.a)RegExpStatics.o
(../libjs_static.a)RegExp.o
(../libjs_static.a)Marking.o
(../libjs_static.a)Memory.o
(../libjs_static.a)Statistics.o
(../libjs_static.a)StringBuffer.o
(../libjs_static.a)Unicode.o
(../libjs_static.a)Xdr.o
(../libjs_static.a)MethodJIT.o
(../libjs_static.a)Compiler.o
(../libjs_static.a)FrameState.o
(../libjs_static.a)FastArithmetic.o
(../libjs_static.a)FastBuiltins.o
(../libjs_static.a)FastOps.o
(../libjs_static.a)LoopState.o
(../libjs_static.a)StubCompiler.o
(../libjs_static.a)MonoIC.o
(../libjs_static.a)PolyIC.o
(../libjs_static.a)InvokeHelpers.o
(../libjs_static.a)Retcon.o
(../libjs_static.a)TrampolineCompiler.o
(../libjs_static.a)ExecutableAllocator.o
(../libjs_static.a)YarrInterpreter.o
(../libjs_static.a)YarrPattern.o
(../libjs_static.a)YarrSyntaxChecker.o
(../libjs_static.a)ExecutableAllocatorPosix.o
(../libjs_static.a)OSAllocatorPosix.o
(../libjs_static.a)YarrJIT.o
(../libjs_static.a)CTypes.o
(../libjs_static.a)Library.o
(../libjs_static.a)jsperf.o
(../libjs_static.a)pm_stub.o
(../libjs_static.a)prep_cif.o
(../libjs_static.a)types.o
(../libjs_static.a)closures.o
(../libjs_static.a)ffi64.o
(../libjs_static.a)jsalloc.o
(../libjs_static.a)jsanalyze.o
(../libjs_static.a)jscrashreport.o
(../libjs_static.a)jshash.o
(../libjs_static.a)StubCalls.o
(../../../dist/lib/libmozglue.a)dummy.o
(../../../dist/lib/libmozglue.a)bignum-dtoa.o
(../../../dist/lib/libmozglue.a)bignum.o
(../../../dist/lib/libmozglue.a)cached-powers.o
(../../../dist/lib/libmozglue.a)diy-fp.o
(../../../dist/lib/libmozglue.a)double-conversion.o
(../../../dist/lib/libmozglue.a)fast-dtoa.o
(../../../dist/lib/libmozglue.a)fixed-dtoa.o
(../../../dist/lib/libmozglue.a)strtod.o
(../../../dist/lib/libmozglue.a)HashFunctions.o
-lstdc++ (/usr/lib/libstdc++.so.55.0)
-lm (/usr/lib/libm.so.7.0)
-lpthread (/usr/lib/libpthread.so.16.0)
-lc (/usr/lib/libc.so.65.0)
/usr/lib/crtend.o
../libjs_static.a(ffi64.o)(.text+0x561): In function `ffi_call':
/home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c:485: undefined reference to `ffi_call_unix64'
../libjs_static.a(ffi64.o)(.text+0x5c8): In function `ffi_prep_closure_loc':
/home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/ffi64.c:504: undefined reference to `ffi_closure_unix64'
/usr/bin/ld: link errors found, deleting executable `js'
clang-3: error: linker command failed with exit code 1 (use -v to see invocation)

ffi64.o is here.... but _not_ unix64.o. Interestingly, unix64.o _is_ in readelf -aW libjs_static.a :

$readelf -aW /usr/obj/m-c/js/src/libjs_static.a | grep unix64                                        
0000000000000561  0000001700000004 R_X86_64_PLT32         0000000000000000 ffi_call_unix64 + fffffffffffffffc
00000000000005c8  0000001800000009 R_X86_64_GOTPCREL      0000000000000000 ffi_closure_unix64 + fffffffffffffffc
    18: 0000000000000610   855 FUNC    GLOBAL DEFAULT    1 ffi_closure_unix64_inner
    23: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ffi_call_unix64
    24: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ffi_closure_unix64
File: /usr/obj/m-c/js/src/libjs_static.a(unix64.o)
00000000000001b0  0000000800000004 R_X86_64_PLT32         0000000000000000 ffi_closure_unix64_inner + fffffffffffffffc
     5: 0000000000000000   366 FUNC    GLOBAL DEFAULT    1 ffi_call_unix64
     6: 000000000000016e   314 FUNC    GLOBAL DEFAULT    1 ffi_closure_unix64
     8: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ffi_closure_unix64_inner
Interesting. Looks like a broken .a or linker. Can you try nm --print-armap on libjs_static.a to see if unix64.o is indexed correctly? On linux it shows:

...
ffi_call_unix64 in unix64.o
...
Assignee

Comment 23

7 years ago
And to compare the ld invocation from gcc, with -t :

/usr/bin/ld: mode elf_x86_64_obsd
/usr/lib/crt0.o
/usr/lib/crtbegin.o
js.o
jsoptparse.o
jsheaptools.o
../editline/editline.o
../editline/sysunix.o
../../../modules/zlib/src/adler32.o
../../../modules/zlib/src/compress.o
../../../modules/zlib/src/crc32.o
../../../modules/zlib/src/deflate.o
../../../modules/zlib/src/gzclose.o
../../../modules/zlib/src/gzlib.o
../../../modules/zlib/src/gzread.o
../../../modules/zlib/src/gzwrite.o
../../../modules/zlib/src/infback.o
../../../modules/zlib/src/inffast.o
../../../modules/zlib/src/inflate.o
../../../modules/zlib/src/inftrees.o
../../../modules/zlib/src/trees.o
../../../modules/zlib/src/uncompr.o
../../../modules/zlib/src/zutil.o
-lplds4 (../../../dist/bin/libplds4.so.1.0)
-lplc4 (../../../dist/bin/libplc4.so.1.0)
-lnspr4 (../../../dist/bin/libnspr4.so.1.0)
(../libjs_static.a)jsapi.o
(../libjs_static.a)jsarray.o
(../libjs_static.a)jsatom.o
(../libjs_static.a)jsbool.o
(../libjs_static.a)jsclone.o
(../libjs_static.a)jscntxt.o
(../libjs_static.a)jscompartment.o
(../libjs_static.a)jsdate.o
(../libjs_static.a)jsdbgapi.o
(../libjs_static.a)jsdtoa.o
(../libjs_static.a)jsexn.o
(../libjs_static.a)jsfriendapi.o
(../libjs_static.a)jsfun.o
(../libjs_static.a)jsgc.o
(../libjs_static.a)jsinfer.o
(../libjs_static.a)jsinterp.o
(../libjs_static.a)jsiter.o
(../libjs_static.a)jsmath.o
(../libjs_static.a)jsnativestack.o
(../libjs_static.a)jsnum.o
(../libjs_static.a)jsobj.o
(../libjs_static.a)json.o
(../libjs_static.a)jsonparser.o
(../libjs_static.a)jsopcode.o
(../libjs_static.a)jsproxy.o
(../libjs_static.a)jsprf.o
(../libjs_static.a)jsprobes.o
(../libjs_static.a)jspropertycache.o
(../libjs_static.a)jspropertytree.o
(../libjs_static.a)jsreflect.o
(../libjs_static.a)jsscope.o
(../libjs_static.a)jsscript.o
(../libjs_static.a)jsstr.o
(../libjs_static.a)jstypedarray.o
(../libjs_static.a)jswatchpoint.o
(../libjs_static.a)jsweakmap.o
(../libjs_static.a)jswrapper.o
(../libjs_static.a)jsxml.o
(../libjs_static.a)prmjtime.o
(../libjs_static.a)ArgumentsObject.o
(../libjs_static.a)ScopeObject.o
(../libjs_static.a)Debugger.o
(../libjs_static.a)GlobalObject.o
(../libjs_static.a)ObjectImpl.o
(../libjs_static.a)Stack.o
(../libjs_static.a)String.o
(../libjs_static.a)BytecodeCompiler.o
(../libjs_static.a)BytecodeEmitter.o
(../libjs_static.a)FoldConstants.o
(../libjs_static.a)ParseMaps.o
(../libjs_static.a)ParseNode.o
(../libjs_static.a)Parser.o
(../libjs_static.a)SemanticAnalysis.o
(../libjs_static.a)SPSProfiler.o
(../libjs_static.a)TokenStream.o
(../libjs_static.a)TreeContext.o
(../libjs_static.a)TestingFunctions.o
(../libjs_static.a)LifoAlloc.o
(../libjs_static.a)Eval.o
(../libjs_static.a)MapObject.o
(../libjs_static.a)RegExpObject.o
(../libjs_static.a)RegExpStatics.o
(../libjs_static.a)RegExp.o
(../libjs_static.a)Marking.o
(../libjs_static.a)Memory.o
(../libjs_static.a)Statistics.o
(../libjs_static.a)StringBuffer.o
(../libjs_static.a)Unicode.o
(../libjs_static.a)Xdr.o
(../libjs_static.a)MethodJIT.o
(../libjs_static.a)Compiler.o
(../libjs_static.a)FrameState.o
(../libjs_static.a)FastArithmetic.o
(../libjs_static.a)FastBuiltins.o
(../libjs_static.a)FastOps.o
(../libjs_static.a)LoopState.o
(../libjs_static.a)StubCompiler.o
(../libjs_static.a)MonoIC.o
(../libjs_static.a)PolyIC.o
(../libjs_static.a)InvokeHelpers.o
(../libjs_static.a)Retcon.o
(../libjs_static.a)TrampolineCompiler.o
(../libjs_static.a)ExecutableAllocator.o
(../libjs_static.a)YarrInterpreter.o
(../libjs_static.a)YarrPattern.o
(../libjs_static.a)YarrSyntaxChecker.o
(../libjs_static.a)ExecutableAllocatorPosix.o
(../libjs_static.a)OSAllocatorPosix.o
(../libjs_static.a)YarrJIT.o
(../libjs_static.a)CTypes.o
(../libjs_static.a)Library.o
(../libjs_static.a)jsperf.o
(../libjs_static.a)pm_stub.o
(../libjs_static.a)prep_cif.o
(../libjs_static.a)types.o
(../libjs_static.a)closures.o
(../libjs_static.a)ffi64.o
(../libjs_static.a)unix64.o
(../libjs_static.a)jsalloc.o
(../libjs_static.a)jsanalyze.o
(../libjs_static.a)jscrashreport.o
(../libjs_static.a)jshash.o
(../libjs_static.a)StubCalls.o
(../../../dist/lib/libmozglue.a)dummy.o
(../../../dist/lib/libmozglue.a)bignum-dtoa.o
(../../../dist/lib/libmozglue.a)bignum.o
(../../../dist/lib/libmozglue.a)cached-powers.o
(../../../dist/lib/libmozglue.a)diy-fp.o
(../../../dist/lib/libmozglue.a)double-conversion.o
(../../../dist/lib/libmozglue.a)fast-dtoa.o
(../../../dist/lib/libmozglue.a)fixed-dtoa.o
(../../../dist/lib/libmozglue.a)strtod.o
(../../../dist/lib/libmozglue.a)HashFunctions.o
-lstdc++ (/usr/lib/libstdc++.so.55.0)
-lm (/usr/lib/libm.so.7.0)
-lc (/usr/lib/libc.so.65.0)
-lpthread (/usr/lib/libpthread.so.16.0)
-lc (/usr/lib/libc.so.65.0)
/usr/lib/crtend.o
js.o(.text._Z7ProcessP9JSContextP8JSObjectPKcb+0x579): In function `Evaluate':
/home/landry/src/mozilla-central/js/src/shell/js.cpp:761: warning: strcpy() is almost always misused, please use strlcpy()
../editline/sysunix.o(.text.rl_add_slash+0x3b): In function `rl_ttyset':
/home/landry/src/mozilla-central/js/src/editline/sysunix.c:46: warning: strcat() is almost always misused, please use strlcat()
../../../modules/zlib/src/gzlib.o(.text.MOZ_Z_gzdopen+0x41): In function `MOZ_Z_gzdopen':
/home/landry/src/mozilla-central/modules/zlib/src/gzlib.c:86: warning: sprintf() is often misused, please use snprintf()

This time unix64.o is here. and the readelf -aW output :

$readelf -aW /usr/obj/m-c/js/src/libjs_static.a | grep unix64
0000000000000393  0000001400000009 R_X86_64_GOTPCREL      0000000000000000 ffi_closure_unix64 + fffffffffffffffc
0000000000000c50  0000001c00000004 R_X86_64_PLT32         0000000000000000 ffi_call_unix64 + fffffffffffffffc
    20: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ffi_closure_unix64
    22: 00000000000004d0   812 FUNC    GLOBAL DEFAULT    1 ffi_closure_unix64_inner
    28: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ffi_call_unix64
File: /usr/obj/m-c/js/src/libjs_static.a(unix64.o)
00000000000001b0  0000001200000004 R_X86_64_PLT32         0000000000000000 ffi_closure_unix64_inner + fffffffffffffffc
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
     5: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
     7: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
    10: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
    15: 0000000000000000   366 FUNC    GLOBAL DEFAULT    1 ffi_call_unix64
    16: 000000000000016e   314 FUNC    GLOBAL DEFAULT    1 ffi_closure_unix64
    18: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND ffi_closure_unix64_inner


Of course i can tar the two full objdirs and make them available online..
Assignee

Comment 24

7 years ago
(In reply to Rafael Ávila de Espíndola (:espindola) from comment #22)
> Interesting. Looks like a broken .a or linker. Can you try nm --print-armap
> on libjs_static.a to see if unix64.o is indexed correctly? On linux it shows:
> 
> ...
> ffi_call_unix64 in unix64.o
> ...

I think you have a point here. Just doing it on libffi.a is enough to show it's missing :

$nm -s js-gcc-ok/src/ctypes/libffi/.libs/libffi.a | grep unix64                                                   
ffi_closure_unix64_inner in ffi64.o
ffi_call_unix64 in unix64.o
ffi_closure_unix64 in unix64.o
         U ffi_call_unix64
         U ffi_closure_unix64
000004d0 T ffi_closure_unix64_inner
unix64.o:
00000000 F /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
00000000 F /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
00000000 F /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
00000000 F /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
00000000 T ffi_call_unix64
0000016e T ffi_closure_unix64
         U ffi_closure_unix64_inner

$nm -s m-c/js/src/ctypes/libffi/.libs/libffi.a | grep unix64    
ffi_closure_unix64_inner in ffi64.o
         U ffi_call_unix64
         U ffi_closure_unix64
00000610 T ffi_closure_unix64_inner
unix64.o:
00000000 T ffi_call_unix64
0000016e T ffi_closure_unix64
         U ffi_closure_unix64_inner

Note: we're still using GNU ld/binutils 2.15... yeah i know. No real choice, not my war.

Next step maybe, compare the ld invocations by libtool when creating libffi.a ?
> Next step maybe, compare the ld invocations by libtool when creating
> libffi.a ?

Probably. Also check if there is ranlib invocation. In other contexts I have seen ar create a valid armap just to have ranlib destroy it :-(
Assignee

Comment 26

7 years ago
Interestingly, nm can be invoked on clang-generated unix64.o, but objdump doesnt recognize it.

[10:22] dawn:/usr/obj/ $file js-clang-bork/src/ctypes/libffi/src/x86/unix64.o
js-clang-bork/src/ctypes/libffi/src/x86/unix64.o: ELF 64-bit LSB relocatable, x86-64, version 1, not stripped
[10:22] dawn:/usr/obj/ $file js-gcc-ok/src/ctypes/libffi/src/x86/unix64.o     
js-gcc-ok/src/ctypes/libffi/src/x86/unix64.o: ELF 64-bit LSB relocatable, x86-64, version 1, not stripped
[10:19] dawn:/usr/obj/ $objdump -D js-clang-bork/src/ctypes/libffi/src/x86/unix64.o 
objdump: js-clang-bork/src/ctypes/libffi/src/x86/unix64.o: File format not recognized
[10:19] dawn:/usr/obj/ $objdump -D js-gcc-ok/src/ctypes/libffi/src/x86/unix64.o     

js-gcc-ok/src/ctypes/libffi/src/x86/unix64.o:     file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <ffi_call_unix64>:
...
...


[10:36] dawn:/usr/obj/m-c/js/src/ctypes/libffi/ $gmake src/x86/unix64.lo 
depbase=`echo src/x86/unix64.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ./libtool   --mode=compile clang -DHAVE_CONFIG_H -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi  -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src  -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src  -MT src/x86/unix64.lo -MD -MP -MF $depbase.Tpo -c -o src/x86/unix64.lo /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  clang -DHAVE_CONFIG_H -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src -I. -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/include -Iinclude -I/home/landry/src/mozilla-central/js/src/ctypes/libffi/src -MT src/x86/unix64.lo -MD -MP -MF src/x86/.deps/unix64.Tpo -c /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S  -fPIC -DPIC -o src/x86/unix64.o
[10:37] dawn:/usr/obj/m-c/js/src/ctypes/libffi/ $objdump -D src/x86/unix64.o                                                             
objdump: src/x86/unix64.o: File format not recognized
[10:37] dawn:/usr/obj/m-c/js/src/ctypes/libffi/ $nm -s src/x86/unix64.o                                                                  
         U _GLOBAL_OFFSET_TABLE_
00000000 T ffi_call_unix64
0000016e T ffi_closure_unix64
         U ffi_closure_unix64_inner

And it changes nothing if i directly build the .o (not throught the libtool invocation).

ranlib is of course invoked, either with clang or gcc.

And finally, according to one of my fellow developers, it looks like it triggers a binutils 2.15 issue :
$readelf -aW src/x86/ffi64.o | grep eh_frame
  [20] .eh_frame         PROGBITS        0000000000000000 003590 0000c0 00   A  0   0  8
$readelf -aW src/x86/unix64.o | grep eh_frame
  [ 5] .eh_frame         LOPROC+1        0000000000000000 0002e8 000070 00   A  0   0  8


<mxxx>   [ 5] .eh_frame         LOPROC+1        0000000000000000 0002e8 000070 00   A  0   0  8
<mxxx> I think it's because of the LOPROC+1 there.
<mxxx> But I can't figure out how to reproduce it.
<mxxx> (As in, how to create a .o file that contains LOPROC+1 (aka X86_64_UNWIND) like that.)
<mxxx> binutils 2.17 objdump can handle it
<mxxx> Yeah, that's the bug.
<mxxx> If I patch binutils 2.15 to recognize SHT_X86_64_UNWIND, then 'ar cru foo.a unix64.o' creates an archive with an appropriate symbol table.
<mxxx> It's curious why clang is creating that section with SHT_X86_64_UNWIND instead of SHT_PROGBITS though.  I can't figure out how to do that.
<mxxx> It looks like mozilla must be explicitly using @unwind for the .section .eh_frame directive.

So now, to figure out what happens when clang assembles unix64.S...
Assignee

Comment 27

7 years ago
And for the verbose assembler call:

clang version 3.1 (branches/release_31)
Target: amd64-unknown-openbsd5.2
Thread model: posix
 "/usr/local/bin/clang-3.1" -cc1 -triple amd64-unknown-openbsd5.2 -E -disable-free -disable-llvm-verifier -main-file-name unix64.S -pic-level 2 -mdisable-fp-elim -masm-verbose -mconstructor-aliases -target-cpu x86-64 -momit-leaf-frame-pointer -v -coverage-file /tmp/unix64-ppG0DK.s -resource-dir /usr/local/bin/../lib/clang/3.1 -dependency-file src/x86/.deps/unix64.Tpo -sys-header-deps -MP -MT src/x86/unix64.lo -D HAVE_CONFIG_H -D PIC -I . -I /home/landry/src/mozilla-central/js/src/ctypes/libffi -I . -I /home/landry/src/mozilla-central/js/src/ctypes/libffi/include -I include -I /home/landry/src/mozilla-central/js/src/ctypes/libffi/src -I . -I /home/landry/src/mozilla-central/js/src/ctypes/libffi/include -I include -I /home/landry/src/mozilla-central/js/src/ctypes/libffi/src -fmodule-cache-path /var/tmp/clang-module-cache -ferror-limit 19 -fmessage-length 138 -mstackrealign -fgnu-runtime -fobjc-runtime-has-arc -fobjc-runtime-has-weak -fobjc-dispatch-method=non-legacy -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/unix64-ppG0DK.s -x assembler-with-cpp /home/landry/src/mozilla-central/js/src/ctypes/libffi/src/x86/unix64.S
clang -cc1 version 3.1 based upon LLVM 3.1svn default target amd64-unknown-openbsd5.2
ignoring duplicate directory "."
ignoring duplicate directory "."
ignoring duplicate directory "/home/landry/src/mozilla-central/js/src/ctypes/libffi/include"
ignoring duplicate directory "include"
ignoring duplicate directory "/home/landry/src/mozilla-central/js/src/ctypes/libffi/src"
#include "..." search starts here:
#include <...> search starts here:
 .
 /home/landry/src/mozilla-central/js/src/ctypes/libffi
 /home/landry/src/mozilla-central/js/src/ctypes/libffi/include
 include
 /home/landry/src/mozilla-central/js/src/ctypes/libffi/src
 /usr/local/include
 /usr/local/bin/../lib/clang/3.1/include
 /usr/include
End of search list.
 "/usr/local/bin/clang-3.1" -cc1as -triple amd64-unknown-openbsd5.2 -filetype obj -o src/x86/unix64.o /tmp/unix64-ppG0DK.s


Looking at the generated assembly when passing -S, i see :

 .section .eh_frame,"a",@unwind

And in unix64.S :

327 #ifdef HAVE_AS_X86_64_UNWIND_SECTION_TYPE
328         .section        .eh_frame,"a",@unwind
329 #else
330         .section        .eh_frame,"a",@progbits
331 #endif

So _there_ might be the issue. 
/usr/obj/m-c/js/src/ctypes/libffi/fficonfig.h:#define HAVE_AS_X86_64_UNWIND_SECTION_TYPE 1

Finally, it might be the little brother of bug 631928. Checking with a similar fix.
Can't that simply be related to bug 748138?
Assignee

Comment 29

7 years ago
If i compare a config.log between gcc and clang :
gcc:
configure:12431: checking assembler supports unwind section type
configure:12444: result: no
clang:
configure:12431: checking assembler supports unwind section type
configure:12444: result: yes

And it turns out it's a little more complicated than bug 631928. If i try with a similar fix :

-       libffi_cv_as_x86_64_unwind_section_type=yes
+       libffi_cv_as_x86_64_unwind_section_type=no
        echo '.section .eh_frame,"a",@unwind' > conftest.s
-       if $CC $CFLAGS -c conftest.s 2>&1 | grep -i warning > /dev/null; then
-           libffi_cv_as_x86_64_unwind_section_type=no
+       if $CC $CFLAGS -c conftest.s > /dev/null; then
+           libffi_cv_as_x86_64_unwind_section_type=yes

The test still succeeds, because clang itself supports the unwind section type, but the binutils used to link the resulting objfiles dont.

dawn:/tmp/ $echo '.section .eh_frame,"a",@unwind' > conftest.s
dawn:/tmp/ $clang -c conftest.s
dawn:/tmp/ $gcc -c conftest.s    
conftest.s: Assembler messages:
conftest.s:1: Warning: unrecognized section type

For the test to be complete, it should also try to link the resulting obj..

So from there, different options :
- hack unix64.S to not use @unwind when on OpenBSD (or detect an older binutils..)
- backport the @unwind support to OpenBSD's binutils
- hack the configure test for @unwind so that it also tries to link

using that small chunk:
-#ifdef HAVE_AS_X86_64_UNWIND_SECTION_TYPE
+#if (defined(HAVE_AS_X86_64_UNWIND_SECTION_TYPE) && !defined(__OpenBSD__))
        .section        .eh_frame,"a",@unwind

the js shell finally links correctly with libjs_static.a/libffi.a using clang! But of course i'm not sure at all it's the correct fix.
Assignee

Comment 30

7 years ago
(In reply to Mike Hommey [:glandium] from comment #28)
> Can't that simply be related to bug 748138?

Doesn't seem so, because clang is cached when it comes to libffi's configure:

configuring in ctypes/libffi
running /bin/sh /home/landry/src/mozilla-central/js/src/ctypes/libffi/configure  --disable-shared --enable-static --disable-raw-api --with
-pic --cache-file=/usr/obj/m-c/js/src/ctypes/libffi/config.cache --srcdir=/home/landry/src/mozilla-central/js/src/ctypes/libffi
configure: loading cache /usr/obj/m-c/js/src/ctypes/libffi/config.cache
...
checking for gcc... (cached) clang

$grep clang //usr/obj/m-c/js/src/ctypes/libffi/config.cache
ac_cv_prog_CPP=${ac_cv_prog_CPP='clang -E'}
ac_cv_prog_ac_ct_CC=${ac_cv_prog_ac_ct_CC=clang}

Maybe because i use export in .mozconfig ?

export CC=clang
export CXX=clang++
> the js shell finally links correctly with libjs_static.a/libffi.a using
> clang! But of course i'm not sure at all it's the correct fix.

I think the correct fix is to turn the check for HAVE_AS_X86_64_UNWIND_SECTION_TYPE into a link test. Would something like

clang -shared test.s -o test.so

detect that the linker doesn't support it?
Assignee

Comment 32

7 years ago
[21:47] dawn:/tmp/ $cat conftest.s                                                                                            
.section .eh_frame,"a",@unwind
[21:47] dawn:/tmp/ $clang -v -shared conftest.s -o conftest.so
clang version 3.1 (branches/release_31)
Target: amd64-unknown-openbsd5.2
Thread model: posix
 "/usr/local/bin/clang-3.1" -cc1as -triple amd64-unknown-openbsd5.2 -filetype obj -o /tmp/conftest-S6b86S.o conftest.s
 "/usr/bin/ld" --eh-frame-hdr -Bdynamic -shared -o conftest.so /usr/lib/crtbeginS.o -L/usr/lib/gcc-lib/amd64-unknown-openbsd5.2/4.2.1 /tmp/conftest-S6b86S.o -lgcc -lgcc /usr/lib/crtendS.o

So, doesnt seem so. It's properly linked... without errors.

Another option would be to use readelf -u on the resulting objfile, but atm configure only uses readelf on android, so i'm not sure we can assume it's here.

dawn:/tmp/ $clang -c conftest.s                                                                                       
dawn:/tmp/ $readelf -u conftest.o                                                                                     

There are no unwind sections in this file.

Or objdump -h ,or objdump -j unwind....
It would be better to avoid readelf and objdump if we can. They would fail when doing LTO for example.

How about putting the .o in an archive and trying to use it? That should work on any unix system, with and without lto.
Assignee

Comment 34

7 years ago
I have no friggin' idea what an @unwind section is, and am definitely not an asm person, but the following steps fails here :

$cat conftest.s
.text
        .globl  foo
        .type   foo,@function
foo:
        jmp .bar
        ret
        .size    foo,.-foo
.section .eh_frame,"a",@unwind
.bar:
        ret
$clang -c conftest.s
$ar cru conftest.a conftest.o
$cat main.c
extern void foo();
int main(){foo();}
$ clang main.c conftest.a
clang-3: error: unable to execute command: Segmentation fault (core dumped)
clang-3: error: linker command failed due to signal (use -v to see invocation)

If i do the exact same test with @progbits instead of @unwind it manages to link the file (ok, the resulting binary segfaults..)

Dunno how to shoehorn properly that in a configure test, nor how to properly push that upstream, which added the test only recently : http://gcc.gnu.org/ml/java-patches/2010-q1/msg00058.html
http://llvm.org/pr13502 was fixed, you might want to add a FIXME to configure.in to remove extra logic once openbsd upgrades to a clang that includes the fix.

I can try to patch libffi's configure to fix the HAVE_AS_X86_64_UNWIND_SECTION_TYPE detection, but we have to be OK with keeping local changes since the last time (bug 631928) I was not able to get a single reply from the libffi list.
Assignee

Comment 36

7 years ago
(In reply to Rafael Ávila de Espíndola (:espindola) from comment #35)
> http://llvm.org/pr13502 was fixed, you might want to add a FIXME to
> configure.in to remove extra logic once openbsd upgrades to a clang that
> includes the fix.

Sure, i'll remove it once we have llvm 3.2 (not yet released).

> I can try to patch libffi's configure to fix the
> HAVE_AS_X86_64_UNWIND_SECTION_TYPE detection, but we have to be OK with
> keeping local changes since the last time (bug 631928) I was not able to get
> a single reply from the libffi list.

Why not using a patchset applied on top of upstream releases, like it's done for skia or libvpx ? Did you try directly the upstream declared author (https://github.com/atgreen/libffi/ .....@moxielogic.com) ?
> Sure, i'll remove it once we have llvm 3.2 (not yet released).

Thanks!

> Why not using a patchset applied on top of upstream releases, like it's done
> for skia or libvpx ? Did you try directly the upstream declared author
> (https://github.com/atgreen/libffi/ .....@moxielogic.com) ?

I guess that is what we have, see js/src/ctypes/patches-libffi. I tried the mailing list: http://sourceware.org/ml/libffi-discuss/2011/.

In any case, I will try to write a patch today.
Posted patch preliminary patch (obsolete) — Splinter Review
Landry, can you try the attached patch? On linux I correctly get

$ grep UNWIND  fficonfig.h 
#define HAVE_AS_X86_64_UNWIND_SECTION_TYPE 1

You should get a 0.
Attachment #648704 - Flags: feedback?(landry)
Assignee

Comment 39

7 years ago
(In reply to Rafael Ávila de Espíndola (:espindola) from comment #37)
> > Why not using a patchset applied on top of upstream releases, like it's done
> > for skia or libvpx ? Did you try directly the upstream declared author
> > (https://github.com/atgreen/libffi/ .....@moxielogic.com) ?
> 
> I guess that is what we have, see js/src/ctypes/patches-libffi. I tried the
> mailing list: http://sourceware.org/ml/libffi-discuss/2011/.

Doh, of course! That made me realize that https://bugzilla.mozilla.org/attachment.cgi?id=545222 which landed in http://hg.mozilla.org/mozilla-central/rev/50bc416e57c4 was :
- landed with the wrong # (it was #670719)
- didnt end up as a patch
- didnt fix configure.ac 

I can wrap a patch to properly fix that.

As for the configure patch, it doesn't do the right thing :
- with gcc:
checking system supports unwind section type... conftest1.s: Assembler messages:
conftest1.s:4: Warning: unrecognized section type
yes

$grep UNWI /usr/obj/m-c/js/src/ctypes/libffi/fficonfig.h  
#define HAVE_AS_X86_64_UNWIND_SECTION_TYPE 1

with clang:
checking system supports unwind section type... yes

If i add a bar: label in the unwind section and jump to bar, it crashes clang when building (and the test returns no), but it still compiles and returns yes for gcc.

.text
.globl foo
foo:
jmp bar
.section .eh_frame,"a",@unwind
bar:

Of course dunno if that's valid asm...

Not easy to wrap a proper test... too bad we cant rely on readelf -u
> Doh, of course! That made me realize that
> https://bugzilla.mozilla.org/attachment.cgi?id=545222 which landed in
> http://hg.mozilla.org/mozilla-central/rev/50bc416e57c4 was :
> - landed with the wrong # (it was #670719)
> - didnt end up as a patch
> - didnt fix configure.ac 
> 
> I can wrap a patch to properly fix that.

Please do :-)
Posted patch preliminary patch 2 (obsolete) — Splinter Review
Same as before, but --fatal-warnings and the jmp to cause a linker crash (please check that it is the linker that is crashing :-) ).
Attachment #648704 - Attachment is obsolete: true
Attachment #648704 - Attachment is obsolete: true
Attachment #648704 - Flags: feedback?(landry)
Attachment #649742 - Flags: feedback?(landry)
Attachment #648704 - Flags: feedback?(landry)
Attachment #649743 - Flags: feedback?(landry)
Posted patch preliminary patch 2 (obsolete) — Splinter Review
Same as before, but --fatal-warnings and the jmp to cause a linker crash (please check that it is the linker that is crashing :-) ).
Attachment #649743 - Attachment is obsolete: true
Attachment #649743 - Flags: feedback?(landry)
Assignee

Comment 43

7 years ago
With gcc :

checking assembler supports unwind section type... conftest1.s: Assembler messages:
conftest1.s:5: Warning: unrecognized section type
conftest1.s:6: Error: 1 warnings, treating warnings as errors
no

with clang :

checking assembler supports unwind section type... clang-3: error: unable to execute command: Segmentation fault (core dumped)
clang-3: error: linker command failed due to signal (use -v to see invocation)
no

So that version works for me..if it says yes with a recent gcc/ld/clang then it should be okay. We might redirect stdout/err to dev/null for pretty-printing :)
Assignee

Updated

7 years ago
Attachment #649742 - Flags: feedback?(landry) → feedback+
Assignee

Comment 44

7 years ago
Let's try this again cleaned up after bug 783950, this time with:
- cc/as/ld redirection to dev/null to avoid vomit in the output
- configure/configure.ac change
- 06-bug-778414.patch added

this patch is generated on top of patch from bug-783950 in my mq. I'm not sure the commit msg is 100% correct, and i dont really claim authorship on the patch, rafael can take it :)

If we can have this commited before the next uplift i'll switch all my openbsd builders to clang...
Attachment #649742 - Attachment is obsolete: true
Attachment #653903 - Flags: review?(mh+mozilla)
Comment on attachment 653903 [details] [diff] [review]
detect if linker supports @unwind section

Review of attachment 653903 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/ctypes/libffi/configure.ac
@@ +331,5 @@
>       ;;
>  esac
>  
>  if test x$TARGET = xX86_64; then
> +    AC_CACHE_CHECK([system supports unwind section type],

s/system/toolchain/

@@ +349,5 @@
> +EOF
> +
> +	libffi_cv_as_x86_64_unwind_section_type=no
> +	if $CC -Wa,--fatal-warnings $CFLAGS -c conftest1.s > /dev/null 2>&1 && \
> +           $AR cru conftest1.a conftest1.o > /dev/null 2>&1 &&

You shouldn't need to create a .a, just use the .o in the second CC call.

Please add comments (starting with dnl) as to why you're doing the second CC, which, aiui, is that your toolchain supports unwind section, but can't link correctly when it's used.
Attachment #653903 - Flags: review?(mh+mozilla) → review+
Assignee

Comment 46

7 years ago
New patch addressing glandium's comments, carrying r+ & ready to push once bug 783950 is fixed.

# we ensure that we can compile _and_ link an assembly file containing an @unwind section
# since the compiler can support it and not the linker (ie old binutils)

added that comment (with #, only two occurences of dnl in there...), can someone with a recent binutils confirm that this still yields yes without the ar cru line ?

back on sunday, hopefully this'll make next uplift..
Comment on attachment 654139 [details] [diff] [review]
detect if linker supports @unwind section

Ah, forgot to mention, if you want to change authorship, use hg qref -u 'User name <email@address>' (iirc)
> added that comment (with #, only two occurences of dnl in there...), can
> someone with a recent binutils confirm that this still yields yes without
> the ar cru line ?

I was under the impression the ar was there to get an error when using clang and an old binutils. Do you still get a no without it when using clang?
And yes, the current patch prints a yes with a new linux toolchain.
Assignee

Comment 50

7 years ago
(In reply to Rafael Ávila de Espíndola (:espindola) from comment #48)
> > added that comment (with #, only two occurences of dnl in there...), can
> > someone with a recent binutils confirm that this still yields yes without
> > the ar cru line ?
> 
> I was under the impression the ar was there to get an error when using clang
> and an old binutils. Do you still get a no without it when using clang?

Yes, binutils 2.15 & clang 3.1 yields a no, as does gcc 4.2.1
Assignee

Comment 51

7 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/b0d9390b60c6

with that, amd64 & i386 openbsd clang trunk builds are fine. Removing [leave open].
Whiteboard: [leave open]
Assignee

Comment 52

7 years ago
Hm, scary win64 red on my push: https://tbpl.mozilla.org/php/getParsedLog.php?id=14785073&tree=Mozilla-Inbound

I cant make any sense of it.. as win64 doesnt seem to run the @unwind test at all :

checking whether byte ordering is bigendian... no
checking assembler .cfi pseudo-op support... no
checking for _ prefix in compiled symbols... no
checking whether .eh_frame section should be read-only... no
checking for __attribute__((visibility("hidden")))... no
cl : Command line warning D9002 : ignoring unknown option '-print-multi-os-directory'
cl : Command line error D8003 : missing source filename

the @unwind test should be before the .eh_frame one..
Assignee

Comment 53

7 years ago
Backed out in http://hg.mozilla.org/integration/mozilla-inbound/rev/a7c3131505d7 due to the bustage from bug 783950. Will reland as soon as tree reopens.
Back out was premature, the bustages just needed a clobber. Relanded in https://hg.mozilla.org/integration/mozilla-inbound/rev/94dc6f0b5f8b

Sorry for the noise!
https://hg.mozilla.org/mozilla-central/rev/94dc6f0b5f8b
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla18

Updated

Last year
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.