293438 - nspr (trunk) does not build anymore on x86-64

Status: RESOLVED FIXED

(NSPR :: NSPR, defect, P1)

Description

[Wolfgang Rosenauer [:wolfiR]]

please note that this may be a compiler bug.
NSPR trunk does build with gcc 3.3.5 but not with 4.0.1.
It worked with a trunk build from 4/29/05 though and stopped working with a
cvs checkout with -D 2005-04-30.
So anything here must cause the problem:
http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=NSPR&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2005-04-29+00%3A00%3A00&maxdate=2005-05-01+00%3A00%3A00&cvsroot=%2Fcvsroot

The following error occurs:

gcc -shared -Wl,-soname -Wl,libnspr4.so -o libnspr4.so ./prvrsion.o io/./prfdcac
h.o io/./prmwait.o io/./prmapopt.o io/./priometh.o io/./pripv6.o io/./prlayer.o 
io/./prlog.o io/./prmmap.o io/./prpolevt.o io/./prprf.o io/./prscanf.o io/./prst
dio.o threads/./prcmon.o threads/./prrwlock.o threads/./prtpd.o linking/./prlink
.o malloc/./prmem.o md/./prosdep.o memory/./prshm.o memory/./prshma.o memory/./p
rseg.o misc/./pralarm.o misc/./pratom.o misc/./prcountr.o misc/./prdtoa.o misc/.
/prenv.o misc/./prerr.o misc/./prerror.o misc/./prerrortable.o misc/./prinit.o m
isc/./prinrval.o misc/./pripc.o misc/./prlog2.o misc/./prlong.o misc/./prnetdb.o
 misc/./prolock.o misc/./prrng.o misc/./prsystem.o misc/./prthinfo.o misc/./prtp
ool.o misc/./prtrace.o misc/./prtime.o malloc/./prmalloc.o pthreads/./ptsynch.o 
pthreads/./ptio.o pthreads/./ptthread.o pthreads/./ptmisc.o md/unix/./unix.o md/
unix/./unix_errors.o md/unix/./uxproces.o md/unix/./uxrng.o md/unix/./uxshm.o md
/unix/./uxwrap.o md/unix/./linux.o md/unix/./os_Linux_x86_64.o  -lpthread -ldl
/usr/lib64/gcc/x86_64-suse-linux/4.0.1/../../../../x86_64-suse-linux/bin/ld: io/
./priometh.o: relocation R_X86_64_PC32 against `memcpy@@GLIBC_2.2.5' can not be 
used when making a shared object; recompile with -fPIC
/usr/lib64/gcc/x86_64-suse-linux/4.0.1/../../../../x86_64-suse-linux/bin/ld: fin
al link failed: Bad value

(I can attach a full build log if needed)

Comment 1

[Wan-Teh Chang]

This build failure may have to do with the new symbol
visibility feature in GCC 4.0.  But we have been using
that GCC 4.0 feature on the NSPRPUB_PRE_4_2_CLIENT_BRANCH
for a few months.

Wolfgang: could you build NSPRPUB_PRE_4_2_CLIENT_BRANCH
and see if it has the same problem?

Please attach your build log and nsprpub/config/autoconf.mk
file.

Comment 2

[Wolfgang Rosenauer [:wolfiR]]

(In reply to comment #1)
> This build failure may have to do with the new symbol
> visibility feature in GCC 4.0.  But we have been using
> that GCC 4.0 feature on the NSPRPUB_PRE_4_2_CLIENT_BRANCH
> for a few months.
> 
> Wolfgang: could you build NSPRPUB_PRE_4_2_CLIENT_BRANCH
> and see if it has the same problem?

yes, I have the same problem with that codebase.
I use gcc 4 since two or three weeks and never built
NSPRPUB_PRE_4_2_CLIENT_BRANCH before with it.

Logs will follow.

Comment 3

[Wolfgang Rosenauer [:wolfiR]]

Attachment: build log

Comment 4

[Wolfgang Rosenauer [:wolfiR]]

Attachment: autoconf.mk

Comment 5

[Wan-Teh Chang]

I did a web search for "relocation R_X86_64_PC32 against" and
found many hits, but I didn't find an answer.

The closest thing I found is
http://sources.redhat.com/bugzilla/show_bug.cgi?id=584 and
the person who may be able to shed some light is H.J. Lu,
who works on binutils.

Comment 6

[Wan-Teh Chang]

This thread on the binutils mailing list may be relevant:
http://sources.redhat.com/ml/binutils/2005-01/msg00263.html
The last message on the thread
(http://sources.redhat.com/ml/binutils/2005-01/msg00511.html)
cited a GCC bug report:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19664

Comment 7

[Wolfgang Rosenauer [:wolfiR]]

thanks, I will contact some GCC guys. I will report back what they tell me.

Comment 8

[Wan-Teh Chang]

I reproduced this bug in Fedora Core 4 Test 3.  I don't know
what caused it and can't come up with a small test case with
the same problem, so I can't even write a configure test for
this problem.

Comment 9

[Brian Ryner (not reading)]

Just to make sure it's not something wrong on our end, the preprocessed source
for priometh.c would be helpful.

Comment 10

[Wan-Teh Chang]

Attachment: Preprocessor output (gcc -E) of pripv6.c

Brian,

In my build the linker complains about pripv6.o instead of priometh.o:

gcc -shared -Wl,-soname -Wl,libnspr4.so -o libnspr4.so ./prvrsion.o
io/./prfdcach.o io/./prmwait.o io/./prmapopt.o io/./priometh.o io/./pripv6.o
io/./prlayer.o io/./prlog.o io/./prmmap.o io/./prpolevt.o io/./prprf.o
io/./prscanf.o io/./prstdio.o threads/./prcmon.o threads/./prrwlock.o
threads/./prtpd.o linking/./prlink.o malloc/./prmem.o md/./prosdep.o
memory/./prshm.o memory/./prshma.o memory/./prseg.o misc/./pralarm.o
misc/./pratom.o misc/./prcountr.o misc/./prdtoa.o misc/./prenv.o misc/./prerr.o
misc/./prerror.o misc/./prerrortable.o misc/./prinit.o misc/./prinrval.o
misc/./pripc.o misc/./prlog2.o misc/./prlong.o misc/./prnetdb.o
misc/./prolock.o misc/./prrng.o misc/./prsystem.o misc/./prthinfo.o
misc/./prtpool.o misc/./prtrace.o misc/./prtime.o malloc/./prmalloc.o
pthreads/./ptsynch.o pthreads/./ptio.o pthreads/./ptthread.o
pthreads/./ptmisc.o md/unix/./unix.o md/unix/./unix_errors.o
md/unix/./uxproces.o md/unix/./uxrng.o md/unix/./uxshm.o md/unix/./uxwrap.o
md/unix/./linux.o md/unix/./os_Linux_x86_64.o  -lpthread -ldl
/usr/bin/ld: io/./pripv6.o: relocation R_X86_64_PC32 against
`memcpy@@GLIBC_2.2.5' can not be used when making a shared object; recompile
with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status

So I'm attaching the preprocessor output of pripv6.c here.  I added the -E
flag to the gcc compile command line:

gcc -o pripv6.o -c -I../../../dist/include/nspr/system_wrappers -include
../../../../mozilla/nsprpub/config/gcc_hidden.h   -E  -pipe -ansi -Wall
-pthread -g -fno-inline -fPIC  -UNDEBUG -DDEBUG_wtc  -DDEBUG=1
-DHAVE_VISIBILITY_HIDDEN_ATTRIBUTE=1 -DHAVE_VISIBILITY_PRAGMA=1 -DXP_UNIX=1
-D_POSIX_SOURCE=1 -D_BSD_SOURCE=1 -D_SVID_SOURCE=1 -D_LARGEFILE64_SOURCE=1
-DHAVE_FCNTL_FILE_LOCKING=1 -DLINUX=1 -DHAVE_LCHOWN=1 -DHAVE_STRERROR=1
-D_REENTRANT=1	-DFORCE_PR_LOG -D_PR_PTHREADS -UHAVE_CVAR_BUILT_ON_SEM
-D_NSPR_BUILD_ -I../../../dist/include/nspr
-I../../../../mozilla/nsprpub/pr/include
-I../../../../mozilla/nsprpub/pr/include/private 
../../../../mozilla/nsprpub/pr/src/io/pripv6.c

Comment 11

[Wan-Teh Chang]

Attachment: Preprocessor output (gcc -E) of priometh.c

I'm attaching the preprocessor output of priometh.c as well.
The command I used was:

gcc -o priometh.o -c -I../../../dist/include/nspr/system_wrappers -include
../../../../mozilla/nsprpub/config/gcc_hidden.h   -E  -pipe -ansi -Wall
-pthread -g -fno-inline -fPIC  -UNDEBUG -DDEBUG_wtc  -DDEBUG=1
-DHAVE_VISIBILITY_HIDDEN_ATTRIBUTE=1 -DHAVE_VISIBILITY_PRAGMA=1 -DXP_UNIX=1
-D_POSIX_SOURCE=1 -D_BSD_SOURCE=1 -D_SVID_SOURCE=1 -D_LARGEFILE64_SOURCE=1
-DHAVE_FCNTL_FILE_LOCKING=1 -DLINUX=1 -DHAVE_LCHOWN=1 -DHAVE_STRERROR=1
-D_REENTRANT=1	-DFORCE_PR_LOG -D_PR_PTHREADS -UHAVE_CVAR_BUILT_ON_SEM
-D_NSPR_BUILD_ -I../../../dist/include/nspr
-I../../../../mozilla/nsprpub/pr/include
-I../../../../mozilla/nsprpub/pr/include/private 
../../../../mozilla/nsprpub/pr/src/io/priometh.c

Comment 12

[Wan-Teh Chang]

After a lot of web searching, I finally found that this is a
known GCC bug:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20297

This is the web search that led to the GCC bug report above:

- Enter "R_X86_64_PC32 gcc"
- Click on the second result from sources.redhat.com:
  http://sources.redhat.com/ml/binutils/2005-04/msg00653.html
- Check out the three GCC bugs H.J. Lu suspected.  This is the
  third one.

Here is the description of the bug:

  When "#pragma GCC visibility push" is used on builtin functions, it may
  not be properly handled if "#pragma GCC visibility pop" is missing.

Apparently memcpy is a builtin function.

  

Comment 13

[Wan-Teh Chang]

I found that I can compile and run NSPR with just
-fvisibility=hidden, without -I$(dist_includedir)/system_wrappers
and without -include $(topsrcdir)/config/gcc_hidden.h,
in the following environments:
- Red Hat Enterprise Linux 4, x86, gcc (version 3.4.3-9.EL4)
- Red Hat Enterprise Linux 4, x86, gcc4 (version 4.0.0-0.14.EL4)
- Fedora Core 4 Test 3, x86-64, gcc (version 4.0.0-2)

(Red Hat Enterprise Linux 4 is very close to Fedora Core 3.)

I don't know why I don't need to wrap the system headers.

I verified that -fvisibility=hidden has effects -- NSPR
symbols that are not decorated with
__attribute__((visibility("default")))
are not exported from libnspr4.so.

Can we just use -fvisibility=hidden in NSPR?

Comment 14

[Wan-Teh Chang]

Attachment: Proposed patch: use -fvisibility=hidden, don'wrapped system headers and gcc_hidden.h

Comment 15

[Wan-Teh Chang]

Comment on attachment 183869 [details] [diff] [review]
Proposed patch: use -fvisibility=hidden, don'wrapped system headers and gcc_hidden.h

Brian, please review this patch or suggest a fix.  We need to do something
about this.

Wolfgang, please try this patch against NSPRPUB_PRE_4_2_CLIENT_BRANCH.
It should also apply to the NSPR trunk.

Comment 16

[Brian Ryner (not reading)]

Comment on attachment 183869 [details] [diff] [review]
Proposed patch: use -fvisibility=hidden, don'wrapped system headers and gcc_hidden.h

The generated code will be less optimal, but sure, let's get this fixed.

Comment 17

[Wolfgang Rosenauer [:wolfiR]]

It builds now for me with this patch.

Comment 18

[Wan-Teh Chang]

Brian, is this an accurate description of the effects of
my patch?

       # To work around a build problem on Linux x86-64 (Bugzilla bug
       # 293438), we use the -fvisibility=hidden flag.  This flag is less
       # optimal than #pragma GCC visibility push(hidden) because the flag
       # assumes that symbols defined outside the current source file have
       # the default visibility.  This has the advantage that we don't need
       # to wrap system header files, but has the disadvantage that calls
       # to hidden symbols defined in other source files cannot be
       # optimized by the compiler.  The -fvisibility=hidden flag does
       # hide and export symbols correctly.

Comment 19

[Wan-Teh Chang]

Comment on attachment 183869 [details] [diff] [review]
Proposed patch: use -fvisibility=hidden, don'wrapped system headers and gcc_hidden.h

This patch fixes a build problem on Linux x86-64.

It has been reviewed by Brian Ryner and tested by me
and the bug reporter.

Comment 20

[Asa Dotzler [:asa]]

Comment on attachment 183869 [details] [diff] [review]
Proposed patch: use -fvisibility=hidden, don'wrapped system headers and gcc_hidden.h

a=asa for landing into 1.8b2/1.1a1

Comment 21

[Wan-Teh Chang]

Patch checked in on the NSPR trunk (NSPR 4.6) and the
NSPRPUB_PRE_4_2_CLIENT_BRANCH (Mozilla 1.8 Beta 2,
Aviary 1.1 Alpha 1).

Comment 22

[Randell Jesup [:jesup] (needinfo me)]

This needs to be fixed for the root configure(.in) as well.  See bug 307168.

Also, it's easy to make this fix not hurt performance/optimization when we're
not compiling for x86-64.  Something like this:

        if test "x${target_cpu}" != xx86_64; then
          VISIBILITY_FLAGS='-I$(DIST)/include/system_wrappers -include
$(topsrcdir)/config/gcc_hidden.h'
          WRAP_SYSTEM_INCLUDES=1
        else
#this is the workaround for the GCC bug mentioned above
          VISIBILITY_FLAGS="-fvisibility=hidden"
          WRAP_SYSTEM_INCLUDES=
        fi

Comment 23

[Randell Jesup [:jesup] (needinfo me)]

A patch that removes the perf hit for non-x86-64 processors was posted to bug
307168.