352822 - apps since MOZILLA_1_8_BRANCH don't build anymore on FreeBSD

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Jürgen Herz]

User-Agent:       Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.6) Gecko/20060904 Firefox/1.5.0.6
Build Identifier: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.6) Gecko/20060904 Firefox/1.5.0.6

I'm unable to build Firefox or Seamonkey from HEAD or MOZILLA_1_8_BRANCH on a FreeBSD 6.1 system: undefined references to strncmp, malloc and other standard functions.

Build failure looks as follows:
gcc -I/usr/X11R6/include  -I/usr/X11R6/include -Wall -W -Wno-unused
-Wpointer-arith -Wcast-align -Wno-long-long -pedantic -pipe  -DDEBUG -D_DEBUG -DDEBUG_jrg -DTRACING -g -fPIC -shared -Wl,-z,defs -Wl,-h,libmozjs.so -o libmozjs.so  jsapi.o jsarena.o jsarray.o jsatom.o jsbool.o jscntxt.o jsdate.o jsdbgapi.o jsdhash.o jsdtoa.o jsemit.o jsexn.o jsfun.o jsgc.o jshash.o jsinterp.o jsiter.o jslock.o jslog2.o jslong.o jsmath.o jsnum.o jsobj.o jsopcode.o jsparse.o
jsprf.o jsregexp.o jsscan.o jsscope.o jsscript.o jsstr.o jsutil.o jsxdrapi.o jsxml.o prmjtime.o    -pthread -lm -pthread -L../../dist/lib -lplds4 -lplc4 -lnspr4 -pthread -lm
jsapi.o(.text+0xaf): In function `TryArgumentFormatter':
/usr/home/jrg/mozilla/js/src/jsapi.c:144: undefined reference to `strncmp'
jsapi.o(.text+0x5db): In function `__maskrune':
/usr/include/_ctype.h:101: undefined reference to `___runetype'
jsapi.o(.text+0x5ef):/usr/include/_ctype.h:101: undefined reference to
`_CurrentRuneLocale'
jsapi.o(.text+0xa32): In function `JS_AddArgumentFormatter':
/usr/home/jrg/mozilla/js/src/jsapi.c:414: undefined reference to `strlen'
jsapi.o(.text+0xa77):/usr/home/jrg/mozilla/js/src/jsapi.c:420: undefined
reference to `strcmp'
jsapi.o(.text+0xb0a): In function `JS_RemoveArgumentFormatter':
a.s.o

Apps from MOZILLA_1_8_0_BRANCH build ok. Comparing both linker statements, it shows "-Wl,-z,defs" is new in the not working builds.

After just adding -lc to the the LDFLAGS for FreeBSD, compiling worked for MOZILLA_1_8_BRANCH branch. Alternatively removing "-Wl,-z,defs" from DSO_LDOPTS led to the same result.

So from my point of view "-Wl,-z,defs" causes (gcc/ld on) FreeBSD not to find libc anymore. I'm using GCC 3.4.4, gmake 3.80 and autoconf 2.13.

Reproducible: Always

Comment 1

[Marco Perez [:mpk]]

Confirming for FreeBSD-STABLE. I remember having seen some
bug dealing with the same issue on Solaris some time ago.

Juergen, the options you mentioned are actually passed to
the linker and cause it to not allow undefined symbols in
objects (though they may still be allowed in shared libs).

Comment 2

[Jürgen Herz]

(In reply to comment #1)
> Juergen, the options you mentioned are actually passed to
> the linker and cause it to not allow undefined symbols in
> objects (though they may still be allowed in shared libs).

That's right, I've found that out afterwards.
Those symbols aren't found, but it seems that doesn't matter. In contrast to Linux binaries, ldd doesn't report libc or libpthread for libmozjs.so (neither for my build without -z defs, nor for the official FreeBSD packages of SM 1.0x or FF 1.5), but works nevertheless.
I don't know what all that means, I only know that the current build code makes it impossible to build on FreeBSD out of the box.

Comment 3

[Mats Palmgren (inactive)]

Attachment: wip

FWIW, the changes I did to compile Firefox trunk on FreeBSD 6.2-RC1/i386.

Comment 4

[Gopal Venkatesan]

Attachment: Patch for cairo (pixman.h) for building on FreeBSD 4.x

The file "stdint.h" is not available on FreeBSD 4.x, only "inttypes.h" is available on certain systems including BeOS, OpenBSD.

Comment 5

[Gopal Venkatesan]

On FreeBSD (at least 4.x), using "-lc" in LDFLAGS results in "rcmdsh: unknown user".

I've had successful builds for Firefox and Thunderbird with using "-lc_r".

Comment 6

[Gopal Venkatesan]

Post from my weblog on building Firefox/Thunderbird on FreeBSD 4.x:
http://gopalarathnam.com/weblog/2007/02/07/building-firefoxthunderbird-20-on-freebsd-4x.html

Comment 7

[Mats Palmgren (inactive)]

(In reply to comment #5)
> On FreeBSD (at least 4.x), using "-lc" in LDFLAGS results in "rcmdsh: unknown
> user".
> I've had successful builds for Firefox and Thunderbird with using "-lc_r".

On FreeBSD 4.x I presume.  This bug is about build problems on FreeBSD 6.2
or newer.  FreeBSD 4.11 reached end-of-life on February 1, 2007.
http://groups.google.com/group/mailing.freebsd.announce/browse_thread/thread/eaffc8c58bd9cc6a/c0585d8c62fabcbf?lnk=st&rnum=1&hl=en#c0585d8c62fabcbf

Comment 8

[Justin Grant]

Attachment: Patches to build firefox 3.0b2 on FreeBSD 6.2

There are a few patches required to get this to build. Been using 3.0b2 on FreeBSD 6.2 today(submitted the patches using it) and not seeing any bugs yet. There are definite performance improvements over Firefox 2.0

I have static i386 builds if anyone is interested in testing...

Comment 9

[Reed Loden [:reed]]

(In reply to comment #8)
> Created an attachment (id=295549) [details]
> Patches to build firefox 3.0b2 on FreeBSD 6.2
> 
> There are a few patches required to get this to build. Been using 3.0b2 on
> FreeBSD 6.2 today(submitted the patches using it) and not seeing any bugs yet.
> There are definite performance improvements over Firefox 2.0

Please make all patches against trunk CVS and submit them/it for review (not in a tarball, please).

Comment 10

[Justin Grant]

Attachment: config_autoconf.mk.in.patch

Comment 11

[Justin Grant]

Attachment: content_xslt_public_txDouble.h.patch

Comment 12

[Justin Grant]

Attachment: security_nss_cmd_shlibsign_sign.sh.patch

Comment 13

[Justin Grant]

> Please make all patches against trunk CVS and submit them/it for review (not in
> a tarball, please).
> 

No problem. Added the patch files separately.
More info on the build steps are here --> http://jng.imagine27.com/pages/buildfirefox30b2

Comment 14

[Justin Grant]

When running 'gmake package' it fails:

... SNIP ...

Linking .xpt files...
[browser]
Linking .xpt files completed.
Stripping package directory...
signing nss libraries
Fatal error 'Spinlock called when not threaded.' at line 87 in file /usr/src/lib/libpthread/thread/thr_spinlock.c (errno = 0)
Abort trap (core dumped)
Fatal error 'Spinlock called when not threaded.' at line 87 in file /usr/src/lib/libpthread/thread/thr_spinlock.c (errno = 0)
Abort trap (core dumped)
Removing unpackaged files...
cd ../../dist/firefox; rm -rf firefox-config firefox-bin.elf regchrome* regxpcom* xpcshell* xpidl* xpt_dump* xpt_link*  core bsdecho gtscc jscpucfg nsinstall viewer TestGtkEmbed bloaturls.txt codesighs* elf-dynstr-gc mangle* maptsv* mfc* mkdepend* msdump* msmap* nm2tsv* nsinstall* rebasedlls* res/samples res/throbber shlibsign* winEmbed.exe os2Embed.exe chrome/chrome.rdf chrome/app-chrome.manifest chrome/overlayinfo components/compreg.dat components/xpti.dat content_unit_tests necko_unit_tests 
/usr/home/jgrant/build/mozilla/firefox-3.0b2-en-US.freebsd62-i386-static/config/nsinstall -t removed-files ../../dist/firefox
Compressing...
cd ../../dist && gtar -c --owner=0 --group=0 --numeric-owner --mode="go-w" -f - firefox | bzip2 -vf > firefox-3.0b2.en-US.freebsd6.2-i386.tar.bz2
  (stdin): 

I had a similar bug which also related to signing the libs which I fixed to get the source to build. This seems like it may be the same issue.

Comment 15

[Marco Perez [:mpk]]

Comment on attachment 295563 [details] [diff] [review]
config_autoconf.mk.in.patch

I'm not sure it's a good idea to specify the libs directly in autoconf.mk.in (especially when those entries are valid for all platforms).
Shouldn't we specify those things in config/rules.mk instead?
(see Mats' approach in the first patch)

Comment 16

[Marco Perez [:mpk]]

(In reply to comment #14)
This seems to be caused by a conflict of multiple threading libs being used
simultaneously. I was able to work around the problem by adding the following
two lines to etc/libmap.conf (but it's more of an ugly hack):
 libc_r.so.6       libthr.so
 libpthread.so.2   libthr.so

I'm not sure how to solve the problem from within our codebase. Justin, was
security_nss_cmd_shlibsign_sign.sh.patch meant to tackle this specific problem?

Comment 17

[Wan-Teh Chang]

Comment on attachment 295566 [details] [diff] [review]
security_nss_cmd_shlibsign_sign.sh.patch

This change to sign.sh is a workaround that masks the real problem.
This patch causes us to run shlibsign with the system NSS libraries.
But we want to run shlibsign with the NSS libraries we just built.
Why can't we run shlibsign with LD_LIBRARY_PATH set?

Comment 18

[Justin Grant]

(In reply to comment #15)
> (From update of attachment 295563 [details] [diff] [review])
> I'm not sure it's a good idea to specify the libs directly in autoconf.mk.in
> (especially when those entries are valid for all platforms).
> Shouldn't we specify those things in config/rules.mk instead?
> (see Mats' approach in the first patch)
> 

Marco, you're right please go with the previous patch instead.

Comment 19

[Justin Grant]

Using the FreeBSD official port patches might help.
Take a look at /usr/ports/www/firefox/files on your FreeBSD system.

Comment 20

[Justin Grant]

(In reply to comment #17)
> (From update of attachment 295566 [details] [diff] [review])
> This change to sign.sh is a workaround that masks the real problem.
> This patch causes us to run shlibsign with the system NSS libraries.
> But we want to run shlibsign with the NSS libraries we just built.
> Why can't we run shlibsign with LD_LIBRARY_PATH set?
> 

Take a look at /usr/ports/www/firefox/files/patch-sysnss

Comment 21

[Wan-Teh Chang]

(In reply to comment #20)

Justin, I don't have a FreeBSD system, but I was able to find
the source of /usr/ports/www/firefox/files/patch-sysnss at
http://www.freebsd.org/cgi/cvsweb.cgi/ports/www/firefox/files/patch-sysnss

It is a patch for building Firefox using system NSS.  It doesn't
contain any patch for NSS itself.  So it doesn't answer my question
in comment 17 regarding your NSS patch (attachment 295566 [details] [diff] [review]).

Comment 22

[Wan-Teh Chang]

I found the patches for NSS itself in
http://www.freebsd.org/cgi/cvsweb.cgi/ports/security/nss/files/

but I don't see any patch related to your NSS patch (attachment 295566 [details] [diff] [review]).

Comment 23

[Justin Grant]

(In reply to comment #17)
> (From update of attachment 295566 [details] [diff] [review])
> This change to sign.sh is a workaround that masks the real problem.
> This patch causes us to run shlibsign with the system NSS libraries.
> But we want to run shlibsign with the NSS libraries we just built.
> Why can't we run shlibsign with LD_LIBRARY_PATH set?
> 

I don't see any reason why we can't run shlibsign the way you suggest using LD_LIBRARY_PATH set. I will play with it some more...

Comment 24

[Justin Grant]

(In reply to comment #21)
> (In reply to comment #20)
> 
> Justin, I don't have a FreeBSD system, but I was able to find
> the source of /usr/ports/www/firefox/files/patch-sysnss at
> http://www.freebsd.org/cgi/cvsweb.cgi/ports/www/firefox/files/patch-sysnss
> 
> It is a patch for building Firefox using system NSS.  It doesn't
> contain any patch for NSS itself.  So it doesn't answer my question
> in comment 17 regarding your NSS patch (attachment 295566 [details] [diff] [review]).
> 

Actually patch-sysnss looks like it does contain the patch to use the supplied NSS. Although with the 3.0b2 source it looks like you are right and LD_LIBRARY_PATH would still need to be set correctly.

I just confirmed this. Using patch-sysnss causes the signing to fail and libssl.so.3, libssl3.so and libsmime3.so etc. are reported missing. This means the patched version is trying to use the mozilla NSS libraries as these versions are not part of the FreeBSD nss system install.

Comment 25

[Reed Loden [:reed]]

Comment on attachment 295564 [details] [diff] [review]
content_xslt_public_txDouble.h.patch

Checking in content/xslt/public/txDouble.h;
/cvsroot/mozilla/content/xslt/public/txDouble.h,v  <--  txDouble.h
new revision: 1.2; previous revision: 1.1
done

Comment 26

[Greg Hamamjian]

To all that aren't able to compile Firefox 3.0 on FreeBSD,

I downloaded the Firefox 3.0.1 port from the ports tree here
http://cvsweb.freebsd.org/ports/www/firefox3/firefox3.tar.gz?tarball=1
and then created the .mozconfig file (contents of which are below) in my home directory and compiled it with only one problem.

It complained "can't cd to" the directory where it extracts the downloaded firefox 3 source code.

1) I then cd'd into the work directory and ran:
2) mv mozilla firefox-3.0.1-source
3) After which I cd'd back into its parent directory and ran make install.

.mozconfig contents
ac_add_options --enable-extensions=default
ac_add_options --enable-optimize="-O2"
ac_add_options --enable-default-toolkit=cairo-gtk2
ac_add_options --disable-debug
ac_add_options --enable-official-branding
ac_add_options --prefix=/usr/local/share/firefox-3.0.1
ac_add_options --disable-tests
mk_add_options MOZ_CURRENT_PROJECT=browser
ac_add_options --enable-application=browser

Regards
Greg

Comment 27

[Marco Perez [:mpk]]

On FreeBSD 7 (at least for i386) the Mozilla apps (tested on trunk) seem to
build quite nicely when disabling (or patching) the following parts:

* <audio>/<video> support (see bug 449066)
  (--disable-ogg and --disable-wave)

* ldap support (see bug 449453)
  (--disable-ldap)

* tests (probably several scattered bugs)
  (--disable-tests)

Comment 28

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Comment on attachment 254655 [details] [diff] [review]
Patch for cairo (pixman.h) for building on FreeBSD 4.x

Dunno if this is still needed, but it should go into upstream libpixman

Comment 29

[Marco Perez [:mpk]]

Current trunk code may require the following lines in the mozconfig file
in order to build on FreeBSD:

ac_add_options --disable-necko-wifi
ac_add_options --disable-updater

Comment 30

[Jan Beich]

Is this still an issue with supported FreeBSD releases (8.x and 9.x currently)?

Try building mozilla-central checkout with bug 753046 applied

  # my .mozconfig on 10.0-CURRENT amd64
  ac_add_options --with-system-libevent=/usr/local
  ac_add_options --disable-necko-wifi
  ac_add_options --disable-libnotify
  ac_add_options --disable-dbus

Regarding bundled libevent, fixing it is a waste of time:
- based on obsolete libevent14 branch
- likely needs FreeBSD patches from devel/libevent port

Comment 31

[Marco Perez [:mpk]]

It is, but attachment 647924 [details] [diff] [review] seems to fix it, at least on FreeBSD 9-Stable (amd64).

Comment 32

[Marco Perez [:mpk]]

Trunk builds again on FreeBSD, at least with clang (instead of gcc).
Jan's options from comment 30 should be used in mozconfig though.
Another working mozconfig for FreeBSD 9-stable (amd64):

CC=clang;      export CC
CXX=clang++;   export CXX
CPP=clang-cpp; export CPP
ac_add_options --disable-necko-wifi
ac_add_options --with-system-libevent=/usr/local
#ac_add_options --disable-tests