Closed Bug 514393 Opened 12 years ago Closed 4 years ago

mozilla-central fails to build in js/src

Categories

(Firefox Build System :: General, defect)

x86_64
Linux
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED INCOMPLETE

People

(Reporter: ogg.k.ogg.k, Unassigned)

References

Details

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20060214 Firefox/1.0.7
Build Identifier: Dunno, it does not build :D

I'm cloning git://bluishcoder.co.nz/git/mozilla-central
This is a tracker of the hg repo.

I run:
make -f client.mk

This stops at the end of js/src, with:
ar cr libjs_static.a jsapi.o jsarena.o jsarray.o jsatom.o jsbool.o jscntxt.o jsdate.o jsdbgapi.o jsdhash.o jsdtoa.o jsemit.o jsexn.o jsfun.o jsgc.o jshash.o jsinterp.o jsinvoke.o jsiter.o jslock.o jslog2.o jsmath.o jsnum.o jsobj.o json.o jsopcode.o jsparse.o jsprf.o jsregexp.o jsscan.o jsscope.o jsscript.o jsstr.o jstask.o jsutil.o jsxdrapi.o jsxml.o prmjtime.o  
ranlib libjs_static.a
make[4]: *** No rule to make target `-lpthread', needed by `libmozjs.so'.  Stop.

This is my .mozconfig:
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@
ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options AUTOCONF=autoconf-2.13

The faulty commit is:
commit 1ca3f5629c0dcea222d43cd28173a0b80f4cd2ff
Author: Karl Tomlinson <karlt+@karlt.net>
Date:   Mon Aug 17 09:54:45 2009 +1200

    bug 506845 targets are not rebuilt when archives from EXPAND_LIBNAME change r=bsmedberg

Reproducible: Always

Steps to Reproduce:
1. Get a fresh mozilla-central repo
2. Build on x86_64 Linux
3. 
Actual Results:  
Build fails with above error.

Expected Results:  
Build succeeds.

Setting as critical, as it doesn't even build, though FF doesn't crash.
FWIW, that change didn't break the linux64 build at Mozilla.
Blocks: 506845
Product: Firefox → Core
QA Contact: build.config → build-config
Version: unspecified → Trunk
Same happens with a fresh hg clone http://hg.mozilla.org/mozilla-central
Apparently make doesn't know where to find libpthread.so on your system.
Where is libpthread.so on your system?
Do you have LIBRARY_PATH set in the environment?

This looks similar to one of the issues in bug 511945: -lpthread is normally only added to OS_LDFLAGS, which would not be checked for dependencies, but in js/src/Makefile.in, it gets included through NSPR_LIBS.
Ah, it does look awfully similar indeed.

I do not seem to have a libpthread.so per se, but:

 file /lib64/libpthread*
/lib64/libpthread-2.7.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, stripped
/lib64/libpthread.so.0:   symbolic link to `libpthread-2.7.so'

I do have LIBRARY_PATH set to my $(HOME)/lib, but there is no libpthread/glibc related libs in there.

Other pthread using programs build and run just fine. ldd on one gives:
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1883b0b000)

FWIW, I tried:
  cd ~/lib
  ln -s /lib64/libpthread.so.0 libpthread.so
since that's in my LIBRARY_PATH, but still the same problem.
Thanks for the info.

The build system does not yet have support for LIBRARY_PATH, so that's why the link in ~/lib doesn't work yet.  We probably should support LIBRARY_PATH, though.

But I'm confused about how the link succeeded even before bug 506845 landed if there is no libpthread.so link.

Does "locate libpthread." find a libpthread.a?
I do not know, I don't have a database for locate, and starting updatedb elicited such cries of pain from the thrashing hard drive I had to kill it after a couple minutes :S

That said, I'd be surprised if the Mozilla build used the updatedb database to look for its files... ?

I don't seem to have a libpthread.a though, just a shared object in /lib64.

Also, gcc -print-search-dirs has /lib64 (well, /lib/../lib64/) for libraries.

That probably means gcc does find libpthread (after all, I can build stuff with pthread), but the Mozilla build system does not like it somehow.
The issue here is that the path that make is using differs from the path that gcc is using.  We can probably persuade make to use the same path as gcc, but it would be easiest to just consider LIBRARY_PATH if that works.

It could be helpful to understand where gcc is finding libpthread.so (assuming it is) so as to know the best way to fix this.

Does "gcc --print-file-name=libpthread.so" find that file?
Quite interesting:
gcc --print-file-name=libpthread.so
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.3/../../../../lib64/libpthread.so
(eg, /usr/lib64/libpthread.so).

So I do have a libpthread.so too after all. Both seem to have come from glibc.

But... looking at it more closely, this one is a linker script that points back to /lib64/libpthread.so.0, so it all looks OK.

There's a libpthread.a here as well, and a libpthread_nonshared_a, both normal archives.
FWIW, creating a libpthread.so in /lib "fixes" it, and I have then to do the same for libdl.so.
I'd rather not keep these custom links in /lib though.
Same problem in another place, that will presumably need the same fix, in case it's a per-makefile fix:

in toolkit/system/gnome:
make[4]: *** No rule to make target `-lrt', needed by `libmozgnome.so'.  Stop.

That's the last one, build succeeded after this third symlink.
I have just done some work tracking this down, and found out that on slackware64 (which exhibits this problem), make was compiled without a proper --libdir (if you install make into /usr and your system libs are in /usr/lib64 then you need to pass --libdir=/usr/lib64 to the configure script for make). While make does not install libraries, it does have a function to search for libraries, mozilla-central relies on this. libpthread, libdl, and other system libraries need to be in the "standard library search path" of make to build mozilla-central, this search path is derived at least in part from the libdir that make was compiled with. Make is also dumb when it comes to finding libraries, and it doesn't recognize the naming scheme supported by ld, libXXX.so is the only thing make will find with a default config, if your system only has a libXXX.so.0 or a libXXX.so.1.2.3 or libXXX-1.2.so then ld will find it and make won't which gives you the no rule to make target error even when you have the library in question.
Sounds like a bug in your distro, then.
Status: UNCONFIRMED → RESOLVED
Closed: 11 years ago
Resolution: --- → INVALID
Would you mind giving me the relevant links to information you found about this when working out the problem?
I'm not sure whether the bug is not having --libdir for make, or having glibc install libpthread in lib64 rather than lib, and I'd like to read up on this before moaning to the distro maintainer.
Thanks
The error happens because there is a makefile that makes make search for libpthread (not gcc/ld, their configs are irrelevant), a quick method to check for the problem is with a command like this

echo -e 'js: -lpthread\n\techo ABC' | make -f -

On your distro you should see it run 'echo ABC' after it confirms that libpthread exists, and firefox relies on this to build. If that is failing with no rule to make target '-lpthread' then make can't find libpthread.

Make does library searches as defined here http://www.gnu.org/software/make/manual/html_node/Libraries_002fSearch.html#Libraries_002fSearch if you read carefully, it says it searches "/lib, /usr/lib, and prefix/lib" as well as the VPATH, the VPATH is defined by the mozilla build system for it's own libraries, and the other libraries (like libpthread) need to be in one of the locations that is also searched. Those are defined in library_search() in http://cvs.savannah.gnu.org/viewvc/make/remake.c?root=make&view=markup if you read it, one of those locations is LIBDIR, go through the configure script for make and you will find LIBDIR is defined to be whatever you pass to --libdir (defaults to PREFIX/lib which is why the make docs say the third location is prefix/lib), you may want to patch make and put in all lib64 directories for your distro in there (you might even consider those hard coded paths to be bugs in make itself).

The other thing that can cause it on your distro is you have libpthread and make can't find it because makes library searching is no more advanced than the regex .LIBPATTERNS which defaults to ‘lib%.so lib%.a’, thus if for some reason you don't have a file named [exactly] libpthread.so in one of the directories searched by make then make won't find it and you would get the same error as above.
Thanks.
What I meant by "or having glibc install libpthread in lib64 rather than lib" is that I'm on x86_64, and it'd seem more correct to install libpthread in /usr/lib, where mine is currently in /usr/lib64. If it was in /usr/lib, then if I understand your point, it'd "all work" as well. So it's either a problem in not specifying a /usr/lib64 libdir to make's build, or getting glibc to install libthread in /usr/lib64 instead of /usr/lib. Hence my question.
In any case, thanks for working out the problem, and my apologies for reporting something which wasn't a mozilla bug after all.
I think it is pretty standard to have libpthread in /usr/lib64 on a 64-bit distro (works better in a multilib distro where you have 32-bit things in /usr/lib and things with the same name but 64-bit in /usr/lib64). But wherever it is installed, glibc should install libpthread in a standard location and that standard location needs to be known to make (it should be in makes list of standard library locations), the actual location of the library shouldn't be important, but everything needs to agree on where to look.
I tend to think that the Mozilla build system should either
a) use gcc -print-search-dirs to set make's library search path, or
b) not include system libraries in dependencies

It sounds like make can't be configured (without patching) to search more than one system directory, but system libraries are often in multiple directories.
Relying on make would also require a different make executable for each cross-compile setup.
option a) can easily be done with autoconf/automake (would need to just add everything to the VPATH(, and option b) seems to be how FF 3.6 and earlier worked (because I never had that problem with that version)
FWIW, I'm hitting what looks like the same bug in a 64-bit system running up-to-date Ubuntu 11.04 (installed at alpha2, though most recent release is alpha3):
> No rule to make target `-lpthread' needed by js

Running the "echo...make" command from comment 14 shows that my system is indeed broken.

I was able to build successfully on this system about 3 weeks ago, so a recent-ish system-update must have changed the configuration.

(I know this isn't a Firefox bug, but I'm mentioning it here just as a heads-up, since new Ubuntu 11.04 64-bit users may start hitting this, as we approach the beta for that Ubuntu release.)
I believe it is a Mozilla build system bug.  (Comment 17)
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: INVALID → ---
(This machine might also have been busted by a broken alpha-quality package upgrade at some point -- my /lib and /lib64 dirs have no files with 'pthread' in their name, while my /lib32 dir _does_ have libpthread-2.13.so.  For comparison, on a different working 64-bit system running stable Ubuntu 10.10, I've got libpthread files in /lib, /lib64, and /lib32)
What does "gcc -print-file-name=libpthread.so" say?
I think I mentioned this above, but the issue is make, not gcc, can search for libraries, the search path is determined at make's compile, and it is non-configurable (other than it's --prefix and --libdir options), the gcc configuration is irrelevant if make it looking for the file (make is run before gcc in the build process). It is my opinion that due to the lack of configuration of this "feature" of make shouldn't be used by the firefox build system (reduces flexibility, call it a feature request if you want).

If you see the error you need to locate the library (libpthread in this case) and recompile make with --libdir=/path/to/parent/of/libdir/ (usually --libdir=/usr/lib64 if you have the issue), make itself doesn't actually install anything into the libdir so it is easy for a distro matainer to skip the --libdir when building make. I have reported this as a bug/feature request to gnu make and I have not received a response [ http://savannah.gnu.org/bugs/?30914 ].
(In reply to comment #22)
> What does "gcc -print-file-name=libpthread.so" say?
Ubuntu 11.04 system:
  /usr/lib/x86_64-linux-gnu/libpthread.so
10.10 stable system: 
  /usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../../lib/libpthread.so
  (looks like that translates to /usr/lib/libpthread.so)
blocking-b2g: --- → 2.2r?
blocking-b2g: 2.2r? → 2.6?
blocking-b2g: 2.6? → ---
Bug against ancient toolchain and source revision.
Status: REOPENED → RESOLVED
Closed: 11 years ago4 years ago
Resolution: --- → INCOMPLETE
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.