'make install' changes needed on Linux platforms : re-usable system NSPR + NSS + XUL "sub-packages" + no links into source tree

RESOLVED WONTFIX

Status

RESOLVED WONTFIX
8 years ago
7 months ago

People

(Reporter: jason.vas.dias, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 7 obsolete attachments)

(Reporter)

Description

8 years ago
User-Agent:       Mozilla/5.0 (X11; Linux x86_64; rv:2.0b13pre) Gecko/20110415 Firefox/4.0b13pre
Build Identifier: 

Frustration in dealing with the quirks of a 10+ years old "make install" 
Mozilla installation process led me to submit this email on the "Nightly Build" 
landing page , which was thankfully responded to :

 > E-mail: jason.vas.dias@gmail.com
> > Area of Interest: Coding
> > Comment: Hi -
> > Back in the mists of time, I seem to remember submitting a bug report very similar to this issue about  building & installing netscape navigator - your build and install process has not changed all that much since then :-)
> >
> > Having done:
> > $ hg clone http://hg.mozilla.org/releases/mozilla-2.1
> >
> > thinking I would get "the latest 2.1 stable release",
> >
> > But instead I got "minefield v2.0.b13" .
> >
> > I thought /releases/* meant "NOT the trunk" / beta release ?
> >
> > You know, it would be really, really nice if on your "hg.mozilla.org"  homepage you had a link in a nice big font to "THE LATEST STABLE FIREFOX AND XULRUNNER  SOURCE WITH LATEST BUG FIXES" .
> >
> > Anyway, it appears that such a beast does not exist.
> >
> >
> > So with my  .mozconfig :
> > $ cat .mozconfig
> > mk_add_options AUTOCONF=autoconf-2.13
> > mk_add_options MOZ_OBJDIR=/home/firefox/x86_64
> > ac_add_options --prefix=/usr
> > ac_add_options --libdir=/usr/lib64
> > ac_add_options --disable-static
> > ac_add_options --enable-shared
> > ac_add_options --with-pic
> > ac_add_options --enable-application=
> > # FIRST xulrunner ; make & install
> > # THEN browser
> > ac_add_options --enable-debug
> > ac_add_options --enable-default-toolkit=cairo-gtk2
> > ac_add_options --enable-plugins
> > ac_add_options --with-pthreads
> > ac_add_options --with-libxul-sdk=/usr/lib64/xulrunner-devel-2.0b13pre
> > ac_add_options --with-system-libxul
> > ac_add_options --with-system-nspr
> > ac_add_options --with-nspr-prefix=/usr
> > ac_add_options --with-system-libevent=/usr
> > ac_add_options --with-system-jpeg=/usr
> > ac_add_options --with-system-zlib=/usr
> > ac_add_options --with-system-bz2=/usr
> > ac_add_options --with-java-bin-path=/usr/java/bin
> > ac_add_options --with-java-include-path=/usr/java/include
> > ac_add_options --disable-strip
> > ac_add_options --disable-install-strip
> >
> > Having first done:
> > $ cd src;   make -f client.mk
> >
> > with --enable-application=xulrunner
> > in .mozconfig
> > and then done
> > $ cd ~/x86_64 ; make -j2 && make install
> >
> > and then edited .mozconfig, changing only
> > --enable-application=browser and
> > adding :
> >
> > ac_add_options --with-libxul-sdk=/usr/lib64/xulrunner-devel-2.0b13pre
> > ac_add_options --with-system-libxul
> > ac_add_options --with-system-nspr
> >
> > to .mozconfig, to , and then successfully done
> > $ cd ~/src; make -f client.mk &&
> > cd ~/x86_64 &&
> > make -j2 && make install
> >
> > Then xulrunner is correctly installed under
> >
> > /usr/lib64/xulrunner-2.0b13/
> >
> > and firefox does get installed under
> >
> > /usr/lib64/firefox-4.0b13/
> >
> > but the firefox directory is full of links to /home/firefox/src , via an extra
> >  '../../../' - should be '../..'
> > and particulary the /usr/lib64/firefox-4.0b13/xulrunner directory in firefox is a link to ../xulrunner, which doesn't exist - having fixed such simple
> > link errors, I had to dereference the faulty '../../..' links by doing:
> > $ mkdir -p ~firefox/x86_64/dist/d/d/d;
> > find . -type l -ls | grep '\.\./\.\./\.\./' | sed -n '/\.\.\/\.\.\/\.\.\//{s,^.*[\ \     ]\([^\ \        ][^\ \  ]*\)[\ \        ]*->[\ \        ]*\(\.\./\.\./\.\./.*\)$,\1 \2,;p}' | while read f l; do cd ~firefox/x86_64/dist/d/d/d; if [ -e $l ]; then d=${l%/[^/]*}; cd $d && echo $f $(pwd)/${l##*/}; else echo $f '->' $l not found >>/dev/fd/2; fi; done | while read f l; do echo $f $l; rm -f $f; cp -fp $l $f; echo '$f COPIED OK' >>/dev/fd/2; done
> >
> > that cleaned up the links in my /usr/lib64/firefox-4.0b13 directory OK and the great new browser that is 4.0x+ started for the first time on my box -
> > really nice browser folks - THANKS !
> > But please,  can't you clean up the Linux build and installation process ?
> >  I am a software engineer and would be happy to assist - I could send patches , but they would probably involve doing the whole installation in a radically different way (using only make(1), no tar(1) ).
> >


Re: [Contribute] Inquiry about Mozilla Coding
From: 
Paul Biggar <pbiggar@mozilla.com>
  To: 
jason.vas.dias@gmail.com
  CC: 
contribute@mozilla.org, Ted Mielczarek <ted@mielczarek.org>
  Date: 
2011-04-16 19:04
   
Hi Jason,

On Sat, Apr 16, 2011 at 17:54,  <contribute-form@mozilla.org> wrote:
> Back in the mists of time, I seem to remember submitting a bug report very similar to this issue about  building & installing netscape navigator - your build and install process has not changed all that much since then :-)

You're 100% right. However, it's so legacy nearly everyone is afraid
to touch it!


> Having done  $ hg clone http://hg.mozilla.org/releases/mozilla-2.1 thinking I would get "the latest 2.1 stable release",

You know, that's how I would have done it too. There is a problem that
few mozilla developers fully understand how the build is supposed to
work, so how can we expect new contributors to?

We're working on related things, and would love to fix that.


> You know, it would be really, really nice if on your "hg.mozilla.org"  homepage you had a link in a nice big font to "THE LATEST STABLE FIREFOX AND XULRUNNER  SOURCE WITH LATEST BUG FIXES" .

I just added https://bugzilla.mozilla.org/show_bug.cgi?id=650518,
which is similar to, but not what you want. I'm trying to figure out
what you're asking for. Do you mean the source from Firefox 4? What do
you mean by "latest bug fixes"?


> but the firefox directory is full of links to /home/firefox/src , via an extra
>  '../../../' - should be '../..'

I suspect you might have needed a `make package` step? I'm honestly not sure.


>  I am a software engineer and would be happy to assist - I could send patches , but they would probably involve doing the whole installation in a radically different way (using only make(1), no tar(1) ).

Awesome. I'd personally love it if you fixed:

https://bugzilla.mozilla.org/show_bug.cgi?id=648701
https://bugzilla.mozilla.org/show_bug.cgi?id=648681

I've CCed Ted, our build system maintainer. While I suspect that a
radical rewrite of the build system is infeasible, we'd certainly love
help in bringing it up to date. You can find the build system bugs at
https://bugzilla.mozilla.org/buglist.cgi?product=Core&amp;component=Build%20Config&amp;resolution=---.


Thanks,
Paul


Re: [Contribute] Inquiry about Mozilla Coding
From: 
Jason Vas Dias <jason.vas.dias@gmail.com>
  To: 
Paul Biggar <pbiggar@mozilla.com>
  CC: 
contribute@mozilla.org, Ted Mielczarek <ted@mielczarek.org>
  Date: 
2011-04-16 19:49
   
On Saturday 16 April 2011 19:04:36 Paul Biggar wrote:
> Hi Jason,
> 
> On Sat, Apr 16, 2011 at 17:54,  <contribute-form@mozilla.org> wrote:
> > Back in the mists of time, I seem to remember submitting a bug report very similar to this issue about  building & installing netscape navigator - your build and install process has not changed all that much since then :-)
> 
> You're 100% right. However, it's so legacy nearly everyone is afraid
> to touch it!
> 
But one shouldn't be - it is doing everything OK, as far as I can see, up to the point where the $OBJDIR/dist directories are created ;
then it does some horrible tar(1) pipeline thing that creates alot of redundant copies of the built objects; it looks to me to be quite
straightforward to remove the invocation of the "installer" and replace it with a generated make(1) file , that is then "include"-d in the
main make file - ie in make syntax:

"
    install:  $(MOZ_APP).install.mk
    
    %.install.mk:$(top_srcdir)/$(MOZ_APP)/make.install.sh
        $< > $@

    include $(MOZ_APP).install.mk
"

Now, the "make.install.sh" shell script could do anything in order to generate the "install make script", which it prints to standard output
and would consist of statements like:

 '  $(DESTDIR)/$(PREFIX)/$(MOZ_APP_INSTALL_DIR)/bin/$(MOZ_APP) :  $(MOZ_OBJDIR)/$(MOZ_APP)/$(MOZ_APP)
               $(INSTALL) ... $< $@
 '

So there is no need for any "installer" run on the Linux platform.
 
> 
> > Having done  $ hg clone http://hg.mozilla.org/releases/mozilla-2.1 thinking I would get "the latest 2.1 stable release",
> 
> You know, that's how I would have done it too. There is a problem that
> few mozilla developers fully understand how the build is supposed to
> work, so how can we expect new contributors to?
> 
> We're working on related things, and would love to fix that.
> 

Might I suggest moving to GIT ? It is SO much nicer than hg ....

> 
> > You know, it would be really, really nice if on your "hg.mozilla.org"  homepage you had a link in a nice big font to "THE LATEST STABLE FIREFOX AND XULRUNNER  SOURCE WITH LATEST BUG FIXES" .
> 
> I just added https://bugzilla.mozilla.org/show_bug.cgi?id=650518,
> which is similar to, but not what you want. I'm trying to figure out
> what you're asking for. Do you mean the source from Firefox 4? What do
> you mean by "latest bug fixes"?
> 

THANKS ! I'll continue discussion on that issue there .

> 
> > but the firefox directory is full of links to /home/firefox/src , via an extra
> >  '../../../' - should be '../..'
> 
> I suspect you might have needed a `make package` step? I'm honestly not sure.
> 
> 
> >  I am a software engineer and would be happy to assist - I could send patches , but they would probably involve doing the whole installation in a radically different way (using only make(1), no tar(1) ).
> 
> Awesome. I'd personally love it if you fixed:
> 
> https://bugzilla.mozilla.org/show_bug.cgi?id=648701
> https://bugzilla.mozilla.org/show_bug.cgi?id=648681

OK,  you're in luck, I'm in between projects at the moment, and I do have a make script library that is well up to the task.
I'll fix these bugs over the next 48-hours and submit patches for them.

> 
> I've CCed Ted, our build system maintainer. While I suspect that a
> radical rewrite of the build system is infeasible, we'd certainly love
> help in bringing it up to date. You can find the build system bugs at
> https://bugzilla.mozilla.org/buglist.cgi?product=Core&amp;component=Build%20Config&amp;resolution=---.
> 

I'm not suggesting a "radical rewrite" ,  just "skip the installer run phase on Linux and generate an install make file instead" -
it is a radically different way of installing the build results, but that should be all . I think the Linux distro maintainers do something
similar in their package install scripts, but I don't accept any code on my system from third parties - I compile everything from the
upstream sources, and fix all the issues - of which there are suprisingly few at the moment.

> 
> Thanks,
> Paul
> 
I'll fill in the bugzillas - speak to you there.
Thanks & Regards,
Jason

Re: [Contribute] Inquiry about Mozilla Coding
From: 
Jason Vas Dias <jason.vas.dias@gmail.com>
  To: 
Josh Matthews <josh@joshmatthews.net>
  CC: 
contribute <contribute@mozilla.org>
  Date: 
2011-04-16 19:34
   
On Saturday 16 April 2011 18:26:28 Josh Matthews wrote:
> Hi Jason,
> I'm curious where you ended up looking to figure all these steps out.
> For one thing, you went through significantly more effort than is
> required (for example, you don't need to build xulrunner separately).

Hi Josh - 
I was building gcc-4.6.0 when it got up to the point where it tries to
generate the gcj java  AWT / Swing gtk-cairo implementation + gapplet browser,
and realized my xulrunner / /usr/lib64/nspr/lib* files had not been rebuilt in
over a year ( via basically the same process I've been following since @1998
with netscape navigator - I don't install any pre-compiled binaries on my system).
 
I just go to "developer.mozilla.org"   / MOZILLA / and find :
https://developer.mozilla.org/En/Developer_Guide/Build_Instructions


> For future reference, the most simple build instructions can be found
> at https://developer.mozilla.org/En/Simple_Firefox_build .  

But I really needed libnspr + libnss + xulrunner + firefox for the gcc build

> Beyond
> that, mozilla-2.1 is the branch from which Firefox Mobile 4 was
> released, IIRC.  In order to turn the final build into a byte-for-byte
> copy of official release, I expect you'd have to enable
> --official-branding,

Aha! I think that was the source of the problem - I gave no '--*-branding' configure argument ,
and the installed */branding/* ended up being links into the source tree .

Next time I'll try to give a --*-branding argument - but I don't have time or disk space to do this ATM -
the whole build process took @8 hours and consumed @4GB of disk space , largely because of 
the horrible redundant copies of libxul.so created in the $OBJDIR/dist, that it would be the first
priority of any build/install process changes I made to remove .

All the best,
Jason

> but I'm unclear on the details.  Give the wiki
> page a whirl, and see if that solves your problems, and write back or
> file a bug if you have any further questions.  There are also build
> system developers who hang out in #developers on irc.mozilla.org who
> would be able to help you.
> 
> Good luck!
> Josh
> 


Many thanks Paul & Josh & Ted for your responses ! 

So I think I've found ONE section of code that could be changed to resolve ALL
my issues with the mozilla build installation process :

src/toolkit/mozapps/installer/packager.mk @ line 606 :

#
# JVD - hmm, on systems with working /usr/bin/install, is this really the best  
# way to install software ( tar ) ? 
#

# The install target will install the application to prefix/lib/appname-version
# In addition if INSTALL_SDK is set, it will install the development headers,
# libraries, and IDL files as follows:
# dist/include -> prefix/include/appname-version
# dist/idl -> prefix/share/idl/appname-version
# dist/sdk/lib -> prefix/lib/appname-devel-version/lib
# prefix/lib/appname-devel-version/* symlinks to the above directories
install:: stage-package
ifneq (,$(filter WINNT WINCE,$(OS_ARCH)))
	$(error "make install" is not supported on this platform. Use "make package" instead.)
endif
ifeq (bundle,$(MOZ_FS_LAYOUT))
	$(error "make install" is not supported on this platform. Use "make package" instead.)
endif
ifdef MOZ_OMNIJAR
	cd $(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)$(_BINPATH) && $(PACK_OMNIJAR)
endif
	$(NSINSTALL) -D $(DESTDIR)$(installdir)
	(cd $(DIST)/$(MOZ_PKG_DIR) && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DESTDIR)$(installdir) && tar -xf -)
	$(NSINSTALL) -D $(DESTDIR)$(bindir)
	$(RM) -f $(DESTDIR)$(bindir)/$(MOZ_APP_NAME)
	ln -s $(installdir)/$(MOZ_APP_NAME) $(DESTDIR)$(bindir)
ifdef INSTALL_SDK # Here comes the hard part
	$(NSINSTALL) -D $(DESTDIR)$(includedir)
	(cd $(DIST)/include && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DESTDIR)$(includedir) && tar -xf -)
	$(NSINSTALL) -D $(DESTDIR)$(idldir)
	(cd $(DIST)/idl && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DESTDIR)$(idldir) && tar -xf -)
# SDK directory is the libs + a bunch of symlinks
	$(NSINSTALL) -D $(DESTDIR)$(sdkdir)/sdk/lib
	if test -f $(DIST)/include/xpcom-config.h; then \
	  $(SYSINSTALL) $(IFLAGS1) $(DIST)/include/xpcom-config.h $(DESTDIR)$(sdkdir); \
	fi
	(cd $(DIST)/sdk/lib && tar $(TAR_CREATE_FLAGS) - .) | (cd $(DESTDIR)$(sdkdir)/sdk/lib && tar -xf -)
	$(RM) -f $(DESTDIR)$(sdkdir)/lib $(DESTDIR)$(sdkdir)/bin $(DESTDIR)$(sdkdir)/include $(DESTDIR)$(sdkdir)/include $(DESTDIR)$(sdkdir)/sdk/idl $(DESTDIR)$(sdkdir)/idl
	ln -s $(sdkdir)/sdk/lib $(DESTDIR)$(sdkdir)/lib
	ln -s $(installdir) $(DESTDIR)$(sdkdir)/bin
	ln -s $(includedir) $(DESTDIR)$(sdkdir)/include
	ln -s $(idldir) $(DESTDIR)$(sdkdir)/idl
endif # INSTALL_SDK

make-sdk:
	$(MAKE) stage-package UNIVERSAL_BINARY= STAGE_SDK=1 MOZ_PKG_DIR=sdk-stage
	@echo "Packaging SDK..."
	$(RM) -rf $(DIST)/$(MOZ_APP_NAME)-sdk
	$(NSINSTALL) -D $(DIST)/$(MOZ_APP_NAME)-sdk/bin
	(cd $(DIST)/sdk-stage && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DIST)/$(MOZ_APP_NAME)-sdk/bin && tar -xf -)
	$(NSINSTALL) -D $(DIST)/$(MOZ_APP_NAME)-sdk/host/bin
	(cd $(DIST)/host/bin && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DIST)/$(MOZ_APP_NAME)-sdk/host/bin && tar -xf -)
	$(NSINSTALL) -D $(DIST)/$(MOZ_APP_NAME)-sdk/sdk
	(cd $(DIST)/sdk && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DIST)/$(MOZ_APP_NAME)-sdk/sdk && tar -xf -)
	$(NSINSTALL) -D $(DIST)/$(MOZ_APP_NAME)-sdk/include
	(cd $(DIST)/include && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DIST)/$(MOZ_APP_NAME)-sdk/include && tar -xf -)
	$(NSINSTALL) -D $(DIST)/$(MOZ_APP_NAME)-sdk/idl
	(cd $(DIST)/idl && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DIST)/$(MOZ_APP_NAME)-sdk/idl && tar -xf -)
	$(NSINSTALL) -D $(DIST)/$(MOZ_APP_NAME)-sdk/lib
# sdk/lib is the same as sdk/sdk/lib
	(cd $(DIST)/sdk/lib && tar $(TAR_CREATE_FLAGS) - .) | \
	  (cd $(DIST)/$(MOZ_APP_NAME)-sdk/lib && tar -xf -)
	$(NSINSTALL) -D $(DIST)/$(SDK_PATH)
	cd $(DIST) && $(MAKE_SDK)

ifeq ($(OS_TARGET), WINNT)
INSTALLER_PACKAGE = $(DIST)/$(PKG_INST_PATH)$(PKG_INST_BASENAME).exe
endif
ifeq ($(OS_TARGET), WINCE)
INSTALLER_PACKAGE = $(DIST)/$(PKG_PATH)$(PKG_BASENAME).cab
endif



Now, I think the above code should be modified to :

o  IF a valid GNU coreutils /usr/bin/install program was found, use that
   with ($INSTALL_OPTIONS) rather than tar(1) to install software .

o  Create dependencies for each $(DESTDIR)/$(PREFIX)/ installed item, 
   rather than using any find | (tar or cpio) pipeline

o  If the user asks to install system NSPR and NSS3 and/or XUL, install them
   under
    $(DESTDIR)/$(PREFIX)/$(LIBDIR)/{nspr,nss,xul}
   and
    $(DESTDIR)/$(PREFIX)/include/{nspr,nss,xul}

   This would require new configure + mozconfig options:

    --enable-system-nspr-install 
    --with-system-nspr-libdir=
    --enable-system-nss-install
    --with-system-nss-libdir=
    --enable-system-xul-install
    --with-system-xul-libdir=

  AND DO NOT INSTALL COPIES OF ALL LIBRARIES INTO 
   /usr/lib{,32,64}/{firefox-*,xulrunner-*/ !!!!!
  INSTALL ONLY LINKS TO SYSTEM nspr, nss3, libxul libs !!!!

This would also greatly streamline the build, which takes up to @ 8 hours
on my 2.2Ghz x86_64, and consumes @ 4GB of disk space, largely because of
multiple redundant copies of libxul.so - why not create some dist/lib
directory and put only ONE copy of each lib in there ?

I think something like the above would make every linux firefox + libnss + libnspr + libxul distro maintainer's dream come true - they really have
to jump through some hoops at the moment to achieve the same result.

Reproducible: Always

Steps to Reproduce:
1. Try to build and install latest xulrunner + firefox

Actual Results:  
multiple duplicate copies of the same libraries are created in
  /usr/lib64/{firefox-,xulrunner-}* 
these directories also ended up containing links into the source tree.

Expected Results:  
only ONE copy of each library installed, and NO links into source tree.

see above
(Reporter)

Updated

8 years ago
Blocks: 650518, 648701, 648681
(Reporter)

Updated

8 years ago
See Also: → bug 107654

Comment 1

8 years ago
I don't see how this blocks those bugs, so am going to unmark them.
No longer blocks: 648681, 648701, 650518
(Reporter)

Updated

8 years ago

Comment 2

8 years ago
There are a bunch of different suggestions all wrapped up here, we need to unpack them:

* using tar|untar: this is the portable way of installing directory structures across all our supported architectures: we don't have rsync on Windows, and install doesn't portably install directory structures correctly on various systems. We might do better using a python-based install script, but I don't want to try using system install and forking the scripts for the "working install" and "non-working install" cases.

* links into the source tree: as far as I know, our current "make install" does not have links into the source tree. If it does, it's a bug, but should probably be separate from this big catchall bug. tar|untar breaks links by default using the flags we pass it.

* Duplicate libraries installed: I can't tell from your report what libraries in particular you are talking about... do you mean that libxul.so is installed by the xulrunner `make install` to /usr/lib64/xulrunner-*/libxul.so, and that the firefox `make install` also copies it to /usr/lib64/firefox-*/libxul.so? This may be an artifact of how Fennec did its installation --with-libxul-sdk but wanted a monolithic install at the end. It's not clear to me which way we want this "by default", since using a shared xulrunner is not a recommended configuration now that we're using short release cycles.

But in any case, it doesn't ever make sense to install a link: either we install the library because we want a monolithic install directory, or we don't install it at all and load it from the well-known "system location".
From my point of view, there is only one problem in the current install process: when using one or more of nspr, nss and libxul sdk from an external directory, make install will install a copy of these. These should probably be symbolic links instead.
(Reporter)

Comment 4

8 years ago
Request for comment : OK, so here's an OUTLINE of how I'm fixing this -
PLEASE DON'T TRY THIS AT HOME YET FOLKS !!  this is to be considered
a "pseudo-code" outline of how I intend to fix this bug, NOT a solution (yet).

IF platform is Linux, and if a $(which install) exists and is GNU coreutils
install, 
I propose escaping the above 'install:' target and providing a new one, in
which each invocation of 'tar' is replaced by : invocation of this makefile
(say src/toolkit/mozapps/installer/install.mk):





#!/usr/bin/gmake
#
# install.mk:
#
# file that can be invoked with GNU make(1) as 'make -f install.mk' instead of 
#  'tar -cpf - . | (cd $DESTDIR; tar -xpf -)' in some Mozilla dist/$DIR 
# to install a mozilla "package" on Linux .
#
# IF $MOZ_USE_SYSTEM_LIBDIR is set, will install TOP LEVEL lib*.so LIBRARIES
into $LIBDIR ;
# IF $MOZ_USE_SYSTEM_NS_LIBDIRS is set, will install top level libs (in some
dist/dir/ )
#    into ${LIBDIR}/nspr or ${LIBDIR}/nss .
#
# Jason Vas Dias (JVD) <jason.vas.dias@gmail.com>
#
ifneq '$(filter Linux linux Linux% %Linux linux% %linux,$(OS_ARCH))' ''
INSTALL:=$(strip $(shell which install))
INSTALL_VERSION=$(strip $(shell $(INSTALL) --version))
ifeq '$(filter install\ \(GNU\ coreutils\),$(INSTALL_VERSION))' ''
$(error Oops, GNU coreutils is required to install mozilla on $(OS_ARCH))
endif
CWD:=$(strip $(shell pwd))
INSTALLER_SRC:=$(topsrcdir)/toolkit/mozapps/installer/
COMPONENT:=$(strip $(shell $(INSTALLER_SRC)/mozilla_component_from_pwd.sh))
DESTINATION:=$(strip $(shell
$(INSTALLER_SRC)/mozilla.destination_for_component.sh $(COMPONENT) TOP))
PREFIX:=$(strip $(shell $(INSTALLER_SRC)/mozilla.destination_for_component.sh  
   $(COMPONENT) PREFIX))
DESTDIR:=$(strip $(shell $(INSTALLER_SRC)/mozilla.destination_for_component.sh 
   $(COMPONENT) DESTDIR))
LIBDIR:=$(strip $(shell $(INSTALLER_SRC)/mozilla.destination_for_component.sh  
   $(COMPONENT) LIBDIR))
BINDIR:=$(strip $(shell $(INSTALLER_SRC)/mozilla.destination_for_component.sh  
   $(COMPONENT) BINDIR))
SBINDIR:=$(strip $(shell $(INSTALLER_SRC)/mozilla.destination_for_component.sh 
   $(COMPONENT) SBINDIR))
SYSCONFDIR:=$(strip $(shell
$(INSTALLER_SRC)/mozilla.destination_for_component.sh  $(COMPONENT)
SYSCONFDIR))
LIBEXECDIR:=$(strip $(shell
$(INSTALLER_SRC)/mozilla.destination_for_component.sh  $(COMPONENT)
LIBEXECDIR))
DATAROOTDIR:=$(strip $(shell
$(INSTALLER_SRC)/mozilla.destination_for_component.sh $(COMPONENT)
DATAROOTDIR))
NSPRLIB:=$(strip $(shell $(INSTALLER_SRC)/mozilla.destination_for_component.sh 
   $(COMPONENT) NSPRLIBDIR))
NSSLIBDIR:=$(strip $(shell
$(INSTALLER_SRC)/mozilla.destination_for_component.sh     $(COMPONENT)
NSSLIBDIR))
XULLIBDIR:=$(strip $(shell
$(INSTALLER_SRC)/mozilla.destination_for_component.sh     $(COMPONENT)
XULLIBDIR))
ifneq '$(and
$(COMPONENT),$(DESTINATION),$(DESTDIR)$(PREFIX),$(LIBDIR),$(BINDIR),$(SBINDIR),$(SYSCONFDIR),$(LIBEXECDIR),$(DATAROOTDIR),$(topsrcdir),$(OBJDIR),$(installdir),$(MOZ_APP_NAME),$(filter
/dist/%, $(CWD)))' '' 
# looks like we are in some dist/$DIR in a mozilla build environment
CWD:=$(strip $(shell pwd))
# handle Solaris / HP-UX echo quirks :
ECHO_N:=$(strip $(shell \
  if [ -x /usr/ucb/echo ] && [ -x /bin/echo ]; then \
       /usr/ucb/echo -n /usr/ucb/echo' -n' \
  elif [ -x /bin/echo ]; then \
       /bin/echo -n /bin/echo' -n'; \
  elif [ -x /usr/bin/echo ]; then \
       /usr/bin/echo -n /usr/bin/echo' -n'; \
  fi \
))
ifneq '$(filter $(OBJDIR)/dist/%,$(CWD))' ''
# this is basically the same as what tar would do:
files:=$(strip $(shell find -type f -o -type l)) # hmm ... maybe some filter
here ?
$(eval \
$(foreach file, $(files), \
$(DESTDIR)/$(PREFIX)/$(strip $(shell \
ORIG_F=$(file)
F=$(file)
destination:
case $$\( $(INSTALLER_SRC)/mozilla.classify $$F | tr -d '\n' \) in \
   LINK\) \
    F=$(readlink $$F); goto destination ;; \
   NSPRLIB\) \
    $(ECHO_N) $(NSPRLIBDIR) ;; \
   NSSLIB\) \
        $(ECHO_N) $(NSSLIBDIR) ;;  \
   *LIB\) \ 
        $(ECHO_N) $(LIBDIR) ;;     \
   PROGRAM\) \
    $(ECHO_N) $(BINDIR) ;;     \
   DATA\) \
        $(ECHO_N) $(DATAROOTDIR)/$(MOZ_APP_NAME) ;;\
   DOCS\) \
        $(ECHO_N) $(DATAROOTDIR)/doc/$(MOZ_APP_NAME) ;;\
   CONF\) \
    $(ECHO_N) $(SYSCONFDIR)/$(MOZ_APP_NAME) ;; \
   *\)\
    $(ECHO_N) $(MOZ_APP_NAME);; \
esac \
))/$(file):$(CWD)/$(filename $(file))
    $(INSTALL) -d $(dirname $@)
    $(INSTALL) -t $(dirname $@) $(file)
))
(Reporter)

Comment 5

8 years ago
RE: Comment #2 - Benjamin - thanks for responding - I submitted last before I saw your comment.

Sure, tar is fine if you want to create a binary distribution - I'm
proposing just providing some kind of '--enable-system-mozilla-libs-install'
option that would create only ONE copy of each top-level shared library
 ( /usr/lib64/{firefox-,xulrunner}-*/lib*.so on my system )
in some /usr/lib64/{nss,nspr,xul} directories, and perhaps install 
the monolithic /usr/lib64/{firefox-,xulrunner-}*/ directories as links
to /usr/share/{firefox-,xulrunner-}/* and to /usr/lib64/{nss,nspr,xul}/* and
to /usr/include/{nspr,nss,xul} and to /usr/share/doc/{firefox-,xulrunner-}/ .

The existing code would be left unchanged; just, if you are on a Linux / UNIX
platform, and HAVE enabled '--enable-mozilla-install-into-system-tree' or 
something, and are NOT building a binary package or a source tar, a different layout will be selected ( in a new 'ifdef XXX ... endif' clause in packager.mk ).
(Reporter)

Comment 6

8 years ago
RE: Comment #3 - Mike - I agree :
> From my point of view, there is only one problem in the current install
> process: when using one or more of nspr, nss and libxul sdk from an external
> directory, make install will install a copy of these. These should probably be
> symbolic links instead.

Stage1 of these improvements will be to the configure script; IF the output of
the following command returns non-null AND some configure option like 
  --enable-mozilla-use-system-ns-libs is set, then configure will test for
the system mozilla libraries:
  'pkg-config nss3 nspr4 libxul --modversion && ..'
IF these all have sufficiently recent versions, it would build against them,
NOT build or install new libnss3, libnspr4, libxul, etc.
(Reporter)

Comment 7

8 years ago
And DURING the build, I'd really, really like to use ONE copy of the 
nspr4, nss3, and libxul libraries in $OBJDIR/dist/lib , not one set
of copies for each app - PLEASE - I have to find 4GB every time I want
to build firefox, because libxul.so is 633MB on my system.
We already have --with-system-{nss,nspr,libxul} etc options, which cause us to not build the in-tree copies.
(Reporter)

Comment 9

8 years ago
RE: Comment #8 :

Yes, but the current installer doesn't know that -
while firefox is correctly built against the system XUL SDK ,
the left-over directories from the xulrunner build a used by
the installer, and it ships separate physical copies of libxul.so 
in /usr/lib64/{firefox-,xulrunner-}* , and once complete, 
   /usr/lib64/firefox-4.0b13pre/xulrunner was a link to ../xulrunner ,
NOT ../xulrunner-2.0b13pre/ !!
That means the current installer ENFORCES you to do:
 $ make clean
between building xulrunner and building firefox, which to me defeats
the whole purpose of using --with-system-{nss,nspr,libxul} !!
(Reporter)

Comment 10

8 years ago
So the goals of the fix to this bug are basically to resolve all the
issues I have found when attempting to build the system nspr, nss,
and then building xulrunner and then firefox to use them,
when NOT doing 'make clean' in between builds of each component - 
which SHOULD work, since firefox is configured to use installed system 
nss, nspr, and XUL SDK - but didn't :

1. First building and installing nsprpub actually produced 32-bit 
   libraries installed in /usr/lib64 !  (sigh ...) 
   $ file /usr/lib64/{libnspr4.so,libplc4.so,libplds4.so}
/usr/lib64/libnspr4.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped
/usr/lib64/libplc4.so:  ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped
/usr/lib64/libplds4.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped

   But these were ignored during the xulrunner and firefox builds, which
   use 64-bit versions, and are configured to link at load-time to their
   own versions of these libraries, even though I had this in .mozconfig:

ac_add_options --with-system-nspr
ac_add_options --with-nspr-prefix=/usr

   So I was completely unaware of this until I tried to build a 64-bit nspr
   using program just now.

   To fix these specific issues, I propose new configure options :

   --enable-system-multiarch : build both 64-bit and 32-bit of NSPR, NSS and XUL
   --enable-system-multilib  : use ${LIBDIR}32 and ${LIBDIR}64 as $LIBDIR 
                               depending on 
                               1. no -m32 gcc flag: 
                                  if natively 64-bit, then ${LIBDIR}64
                                  ( redhat use /lib64 for 64-bit libs on native 64 bit platforms; other distros do not !
                                    otherwise /lib for native 64-bit platforms 
                                  ) 
                               2. with -m32 gcc flag (in $CFLAGS) :
                                  if natively 64-bit, then 
                                     ${LIBDIR} on redhat linuxes, 
                                  else
                                     ${LIBDIR}32 on others (gentoo)
                               3. with -m64 gcc flag
                                  if natively 32-bit, then /lib64
                                  else
                                    if   redhat ${LIBDIR}64
                                    else $LIBDIR
                               4 without any -m32 or -m64 - as I had specified:
                                   JUST USE $LIBDIR AND ADD NO -m{32,64} FLAGS !

                                   
   --enable-system-nspr-build: build and install public nspr libs, and link
                               to them during build + testing in  
                               dist/lib{,32,64} 
 
   --with-system-nspr-libdir=${PFX}/lib/nspr4  # example:
                                               # NSPR libs in /usr/lib/nspr4 
                               set the system NSPR library location - this
                               would override any libdir setting derived from
                               --enable-multilib 

   --enable-system-nss-build : build and install public nss3 libs, and link
                               to them during build + testing in dist/lib{32,64}

   --with-system-nss-libdir=${PFX}/lib/nss3 # example: nss libs in /usr/lib/nss3
                               this would override any libdir setting derived 
                               from --enable-multilib 

So first I'm going to work on getting the above to work, WITH the improved
installer make code, and then testing building the whole shebang with the
new configure and installer code.
I really don't think we want that many new configure options, especially to fix such an edge case. We can take fixes to things that are broken, like "32-bit libraries installed in /usr/lib64", but I don't think we want things like 
--enable-system-nspr-build or --enable-system-multiarch, unless I hear from current distro maintainers that these are things they would actually want.
(Reporter)

Comment 12

8 years ago
To clarify - the only reason the /usr/lib64/ 32-bit NSPR libraries did not
break my build is because firefox and xulrunner executables INCORRECTLY (IMHO) 
do runtime-linking to their own copies of all the NSS, NSPR and LIBXUL libs
in their own monolithic installation directories 
(/usr/lib64/{xulrunner-,firefox-}*b13.pre/)
which I had to MANUALLY move into a single copy in
 /usr/lib64/nspr / /usr/lib64/nss 
because of insufficient disk space ; even though I HAD built and installed
nsprpub BEFORE commencing any xulrunner / firefox build (which actually
installed only 32-bit libraries in my 64-bit library directory).
So perhaps people can understand why I feel driven to correct the mozilla
build and installation system.
I think you need to take a breath, and not overthink it. Speaking with both my Mozilla and Debian hats on, here.

If you want to build a system nspr and nss, just take the nspr and nss tarballs and build them. Then you can build the firefox code with these system libraries. (note nspr's configure requires a --enable-64bit flag on 64 bits architectures, see bug 375281). Note that nss's build system is quite ancient and doesn't have an install rule, but that's not really much of a problem.

You can pretty easily build firefox and xulrunner in two passes, see http://glandium.org/blog/?p=1258. Add ac_add_options --with-system-nspr and ac_add_options --with-system-nss to these instructions to use the system nspr and nss libraries.

Comment 3 is really the only thing that may need a fix. Possibly nss's build system could grow an install rule, but that'd be a separate bug (and who knows, it probably already exists).

Oh, and add separate pkg-config files for nspr and nss, there are already one or two bugs for that, too.
(In reply to comment #12)
> To clarify - the only reason the /usr/lib64/ 32-bit NSPR libraries did not
> break my build is because firefox and xulrunner executables INCORRECTLY (IMHO) 
> do runtime-linking to their own copies of all the NSS, NSPR and LIBXUL libs
> in their own monolithic installation directories 
> (/usr/lib64/{xulrunner-,firefox-}*b13.pre/)
> which I had to MANUALLY move into a single copy in
>  /usr/lib64/nspr / /usr/lib64/nss 
> because of insufficient disk space ; even though I HAD built and installed
> nsprpub BEFORE commencing any xulrunner / firefox build (which actually
> installed only 32-bit libraries in my 64-bit library directory).
> So perhaps people can understand why I feel driven to correct the mozilla
> build and installation system.

You need to read comment 3 again, that's your only real problem.
(Reporter)

Comment 15

8 years ago
RE: Comment #11 :
OK, as long as there is some way to actually build and install 32-bit and 64-bit
NSPR and NSS system libraries successfully and to get xulrunner and firefox
to use them and NOT to build or install their own copies.

And I'm not sure this is such an 'edge case' - MANY distros ship /usr/lib/{nspr,nss}  libraries, but there appears to be no way of getting mozilla
to build and install them out of the box .

MANY distros end up having some /usr/lib*/{nss,nspr} in ld.so.conf - if people
happen to put these directories BEFORE /usr/lib in ld.so.conf, then 
any xulrunner / firefox executable will incorrectly link against them.
I think this is something firefox should at least mitigate against by
adding -Wl,-R,$(DESTDIR)/$(MOZ_APP_NAME) to its $(LDFLAGS) .

I'm going to add these new configure options to my own version of your configure
script; you can select which patches you want to accept into future mozilla source.
(Reporter)

Comment 16

8 years ago
RE Comment #14: 
> You need to read comment 3 again, that's your only real problem
RE: Comment #3:
> From my point of view, there is only one problem in the current install
> process: when using one or more of nspr, nss and libxul sdk from an external
> directory, make install will install a copy of these. These should probably be
> symbolic links instead.

Thanks for responding, Mike ! Sorry if I had not properly addressed your Comment #3 - here is a better attempt:

As I said in Comment #6 - I wholeheartedly agree with Comment #3, with the
correction : 
   " there is only one problem in the current install"  - I think should be:
   " there is at least one problem in the current install" -
yes, perhaps the foremost of which is that firefox and xulrunner get shipped 
with files that cause them to load AT RUNTIME, WITH dlopen(), their shared 
library dependencies from their monolithic package install directories:

$ cat firefox-4.0b13pre/dependentlibs.list
libnspr4.so
libplc4.so
libplds4.so
libmozalloc.so
libnssutil3.so
libsoftokn3.so
libnss3.so
libssl3.so
libsmime3.so
libmozsqlite3.so
libxul.so


The xulrunner-stub executable is not actually LINKED by -l ld(1) args with
these libraries; it evidently reads its dependantlibs.list and uses dlopen()
to load them from those locations, even though I had this .mozconfig :

ac_add_options --with-libxul-sdk=/usr/lib64/xulrunner-devel-2.0b13pre
ac_add_options --with-system-libxul
ac_add_options --enable-system-nspr
ac_add_options --enable-system-nss

And on a related issue - I'm trying to figure out how to build a standalone nss,
and I get some objects in $OBJDIR/security/nss if I copy the source directory
of the same name there (after configure) , cd there, and type 'make' - eg: 
$ find . -name 'lib*.so'
./lib/util/Linux2.6_x86_glibc_PTH_DBG.OBJ/libnssutil3.so
./lib/freebl/Linux2.6_x86_32/Linux_SINGLE_SHLIB/libfreebl3.so
(but I have to manually link ../../../coreconf to $SRC/security/coreconf :-( )

IMHO, there really needs to be some separate makefile and / or "configure" shipped to enable a simple standalone nss build, as well as a standalone
nspr build, catered for in $SRC/nsprpub .
I don't want to get into downloading different versions of libnspr an libnss
source code when the correct versions come with the mozilla source, so 
"installing another source tar" is not an option for me.

So building with CFLAGS='-DUSE
(In reply to comment #15)
> RE: Comment #11 :
> OK, as long as there is some way to actually build and install 32-bit and
> 64-bit
> NSPR and NSS system libraries successfully and to get xulrunner and firefox
> to use them and NOT to build or install their own copies.
> 
> And I'm not sure this is such an 'edge case' - MANY distros ship
> /usr/lib/{nspr,nss}  libraries, but there appears to be no way of getting
> mozilla
> to build and install them out of the box .

All the distros I know (and that matter) ship nspr and nss in /usr/lib (or even /lib), not /usr/lib/{nspr,nss} and do so from independent nspr and nss tarballs. Even without independent tarballs, you can still build and install nspr and nss from the mozilla tree by invoking the same build scripts you would with the nspr and nss tarballs under nsprpub and security/nss.
(In reply to comment #16)
> The xulrunner-stub executable is not actually LINKED by -l ld(1) args with
> these libraries; it evidently reads its dependantlibs.list and uses dlopen()
> to load them from those locations

IIRC it doesn't fail if the files are not there. (as long as libxul.so's dependencies are in the ld.so search path)

> And on a related issue - I'm trying to figure out how to build a standalone
> nss,
> and I get some objects in $OBJDIR/security/nss if I copy the source directory
> of the same name there (after configure) , cd there, and type 'make' - eg: 
> $ find . -name 'lib*.so'
> ./lib/util/Linux2.6_x86_glibc_PTH_DBG.OBJ/libnssutil3.so
> ./lib/freebl/Linux2.6_x86_32/Linux_SINGLE_SHLIB/libfreebl3.so
> (but I have to manually link ../../../coreconf to $SRC/security/coreconf :-( )

add DIST=/some/path to the options you give to make.
(Reporter)

Comment 19

8 years ago
oops, as I was saying - 
So building nss with CFLAGS=${CFLAGS}' -DUSE_64  -I. -Impi -Iecl -I/us/include/nspr -DMP_API_COMPATIBLE -DMP_IOFUNC'
appears to work, though all of which had to be manually added to
security/nss/lib/*/Makefile .

I guess I should give this up and just cobble a standalone NSS build
together from what xulrunner builds in $OBJDIR/nss ?
(In reply to comment #19)
> oops, as I was saying - 
> So building nss with CFLAGS=${CFLAGS}' -DUSE_64  -I. -Impi -Iecl
> -I/us/include/nspr -DMP_API_COMPATIBLE -DMP_IOFUNC'
> appears to work, though all of which had to be manually added to
> security/nss/lib/*/Makefile .

You can pass options to make, e.g.
make NSPR_INCLUDE_DIR=/usr/include/nspr NSPR_LIB_DIR=/usr/lib USE_64=1
(Reporter)

Comment 21

8 years ago
RECOMMENDATION: at least provide a $OBJDIR/nss/Makefile (not currently generated)
to install a standalone nss from the NSS just built by (xulrunner, firefox), 
so after building either, users can type :
   $ make -f $OBJDIR/nss/Makefile install
to install nss in ${DESTDIR}/${PREFIX}/${LIBDIR}

Comment 22

8 years ago
I believe that if you're building/using system NSS, you should do it from the NSS source packages, not from Mozilla's build system. Same with NSPR. Trying to extract a system NSPR and NSS package from Mozilla's build system is not going to be a useful task or one which we take patches for. They are maintained by different groups and have different build systems and goals than the main libxul/firefox build.
(Reporter)

Comment 23

8 years ago
RECOMMENDATION:
 configure.in should make use of gcc's 'print-multi-os-directory' option:

$ gcc -print-multi-os-directory
../lib64

$ gcc -m32 -print-multi-os-directory
../lib32

to determine default $LIBDIR setting if none specified.
(Reporter)

Comment 24

8 years ago
RE: Comment #22:

Thanks Benjamin -

But all I mean by "nss" and "nspr" are the libraries and headers that my
installed libxul and / or firefox and xulrunner are built against; I have
no interest in maintaining any other copies of these headers or libraries.

The idea of building firefox or xulrunner against nspr+nss sources A and
then installing some other nspr+nss sources B seems just crazy to me.
(Reporter)

Comment 25

8 years ago
RECOMMENDATION: configure.in should NOT second-guess user's choice of $CFLAGS !!

In no circumstances EXCEPT when building a "sub-architecture" should ANY
configure.in or Makefile* or *.mk file code add any compiler -m{32,64} 
flag to CFLAGS .

By "sub-architecture" I mean building a   -m32 version of LIBRARIES ONLY
on a native 64-bit machine or building a  -m64 version of LIBRARIES ONLY
on a native 32-bit machine.

The make syntax test for this is simple:

is_native_64_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m64,$(CFLAGS)),$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return (sizeof(void*)==8);} int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f $${f}.c && f)))
is_native_32_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m32,$(CFLAGS)), $(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return (sizeof(void*)==8);} int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f $${f}.c && f))

(careful with bugzilla line-wrap - should only be TWO lines above! )
(Reporter)

Comment 26

8 years ago
Oops:

is_native_32_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m32,$(CFLAGS)),
$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return (sizeof(void*)==4);}
int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f $${f}.c && f))
(Reporter)

Comment 27

8 years ago
So, if those tests were in autoconf.mk or some "include-ed" config/*.mk,
 makefiles like nsprpub/Makefile can contain statements like:

CFLAGS := $(CFLAGS)$(if $(is_native_64_bit), -DUSE_64)
(Reporter)

Comment 28

8 years ago
is_native_64_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter
-m64,$(CFLAGS)),$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return
(sizeof(void*)==8);} int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f
$${f}.c && $$f)))
is_native_32_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m32,$(CFLAGS)),
$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return (sizeof(void*)==4);}
int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f $${f}.c && $$f))
(Reporter)

Comment 29

8 years ago
And somewhere in client.mk or a configure.in equivalent shell script
one could say:

 LIBDIR:=$(strip \
          $(if $(LIBDIR),$(LIBDIR),\
           $(if $(is_native_64_bit), \
            $(shell gcc -m64 -print-multi-os-directory | sed 's,^.*/,,g'),\
            $(shell gcc -m32 -print-multi-os-directory | sed 's,^.*/,,g') \
            )\
           ))
(Reporter)

Comment 30

8 years ago
is_native_64_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter
-m64,$(CFLAGS)),$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return
(sizeof(void*)==8);} int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f
$${f}.c && $$f && echo 1)))
is_native_32_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m32,$(CFLAGS)),
$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return (sizeof(void*)==4);}
int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f $${f}.c && $$f && echo 1))
(Reporter)

Comment 31

8 years ago
is_native_64_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter
-m64,$(CFLAGS)),$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return
(sizeof(void*)==8)?0:1;} int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f
$${f}.c && $$f)))
is_native_32_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m32,$(CFLAGS)),
$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return (sizeof(void*)==4)?0:1;}
int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f $${f}.c && $$f))
(Reporter)

Comment 32

8 years ago
is_native_64_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter
-m64,$(CFLAGS)),$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return
(sizeof(void*)==8)?0:1;} int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o
$$f
$${f}.c && $$f && echo 1)))
is_native_32_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m32,$(CFLAGS)),
$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return
(sizeof(void*)==4)?0:1;}
int main(){ return f();}' > $${f}.c; gcc $CFLAGS -o $$f $${f}.c && $$f && echo  1))
(Reporter)

Comment 33

8 years ago
Sorry for the re-posts, but important to get those tests right !

My new client.mk / configure.in + packager.mk will make use of them - 
and I suggest the mozilla build should do something similar - especially 
in nsprpub - so that users on native 64-bit machines do not have to
supply '-DUSE_64' in $CFLAGS - why should they have to ? 
On a native 64-bit machine they may be unable to 'USE' anything else !
(Reporter)

Comment 34

8 years ago
OK, I'll download the latest nspr and nss standalone source tars and adapt
their configure + install scripts to install from the mozilla build - but
I still think one should be able to install the nspr + nss libraries and
headers as used and built during a xulrunner or firefox build as 
"the system nspr + the system nss" - perhaps:

1. If client.mk finds these files:
   security/nss/configure.in
   security/nss/Makefile.in
   ( nsprpub/ versions already shipped )
AND there is some "MOZ_INSTALL_SYSTEM_NSPR" and/or "MOZ_INSTALL_SYSTEM_NSS"
variables set during the "client.mk" run , then any build would first
build and install nspr and nss as the system nss and nspr libraries ?

I guess that would have to be a patch to the new client.mk that you could either
accept or reject.
(Reporter)

Comment 35

8 years ago
RE: Comment #22:
 "They are maintained by different groups"

Couldn't "the mozilla group" ask "the nss / nspr group" to borrow their
./configure.in and ./Makefile.in files for use to ship a system NSS / NSPR 
after a xulrunner / firefox build ?

Comment 36

8 years ago
The point is that we don't use the in-tree code if you build --with-system-nss --with-system-nspr. So there just won't be those libraries in the mozilla tree.
(Reporter)

Comment 37

8 years ago
See, why should I even have to start thinking about the implications of these
differences between the mozilla nsprpub directory I have and the latest
available nspr+nss ? ( nss-3.12.9-with-nspr-4.8.7.tar.gz ) :

[ firefox@jvdspc:~/nss-3.12.9/mozilla 19:00:35 1240:738 ]
$ diff  -rq nsprpub ~/src/nsprpub
Only in /home/firefox/src/nsprpub: TAG-INFO
Files nsprpub/pr/include/md/_linux.h and /home/firefox/src/nsprpub/pr/include/md/_linux.h differ
Files nsprpub/pr/src/linking/prlink.c and /home/firefox/src/nsprpub/pr/src/linking/prlink.c differ
Files nsprpub/pr/src/misc/prnetdb.c and /home/firefox/src/nsprpub/pr/src/misc/prnetdb.c differ
[ firefox@jvdspc:~/nss-3.12.9/mozilla 19:00:54 1241:739 ]

Comment 38

8 years ago
Because you are trying to build using system-NSPR and system-NSS, which is strongly discouraged unless you are a Linux distributor: in that case you need to know the differences anyway.
(Reporter)

Comment 39

8 years ago
Re: Comment #36 -

Yes, Benjamin - sorry if I didn't make myself clear, but what I've been
trying to describe throughout this bug report is trying to do:

 1 . build and install nsprpub - no easily buildable nss in mozilla source,
     so have to skip that .

 2 . build and install xulrunner with '--with-system-nspr' - but it just
     seemed to ignore that and re-used the nsprlibs in the $OBJDIR .

 3 . build and install firefox with a .mozconfig :


mk_add_options AUTOCONF=autoconf-2.13
mk_add_options MOZ_OBJDIR=/home/firefox/x86_64
ac_add_options --prefix=/usr
ac_add_options --libdir=/usr/lib64
ac_add_options --disable-static
ac_add_options --enable-shared
ac_add_options --with-pic
ac_add_options --enable-application=browser
ac_add_options --enable-debug
ac_add_options --enable-default-toolkit=cairo-gtk2
ac_add_options --enable-plugins
ac_add_options --with-pthreads
ac_add_options --with-libxul-sdk=/usr/lib64/xulrunner-devel-2.0b13pre
ac_add_options --with-system-libxul
ac_add_options --with-system-nspr
ac_add_options --with-nspr-prefix=/usr
ac_add_options --with-system-libevent=/usr
ac_add_options --with-system-jpeg=/usr
ac_add_options --with-system-zlib=/usr
ac_add_options --with-system-bz2=/usr
ac_add_options --with-java-bin-path=/usr/java/bin
ac_add_options --with-java-include-path=/usr/java/include
ac_add_options --disable-strip
ac_add_options --disable-install-strip


with CFLAGS containing '-O2 -g -fPIC -DPIC -Wa,--compress-debug-sections' .

So many things about this process went wrong for me, so I thought - what's
the best way to improve it ?

And I came to the primary general conclusions :

o  there must be someway of installing the nspr and nss just built before
   the xulrunner build, so that ALL further code will use it, and so that
   ALL mozilla binaries AND plugins AND NSS / NSPR using code end up using
   ONE SINGLE COPY of the NSPR and NSS libs.

o  the way files get installed and the layout on Linux is in drastic need of
   improvement - primarily due to Mike's point about duplicate libraries, but
   also because autoconf users expect different default behaviour in honoring
   autoconf variables such as $LIBDIR, $BINDIR, $DATAROOTDIR, $DOCDIR, etc.

Comment 40

8 years ago
> Yes, Benjamin - sorry if I didn't make myself clear, but what I've been
> trying to describe throughout this bug report is trying to do:
> 
>  1 . build and install nsprpub - no easily buildable nss in mozilla source,
>      so have to skip that .
> 
>  2 . build and install xulrunner with '--with-system-nspr' - but it just
>      seemed to ignore that and re-used the nsprlibs in the $OBJDIR .

You are using the same objdir? You shouldn't: you should be using a different objdir for NSPR, XULRunner, and Firefox. That way the libraries are all in separate dist directories and get installed to the correct places independently.

> o  the way files get installed and the layout on Linux is in drastic need of
>    improvement - primarily due to Mike's point about duplicate libraries, but
>    also because autoconf users expect different default behaviour in honoring
>    autoconf variables such as $LIBDIR, $BINDIR, $DATAROOTDIR, $DOCDIR, etc.

In general we cannot provide what autoconf users expect: we don't provide a stable ABI, and don't intend to for the xulrunner/firefox bits (NSPR/NSS do, obviously). So it makes no sense to install our files to /usr/lib and /usr/share, etc. Instead we intend to keep the existing system of putting each xulrunner install in a separate directory (/usr/lib/xulrunner-N)... which does honor libdir, IIRC.
(Reporter)

Comment 41

8 years ago
RE: Comment #38: but you confuse "what" I'm trying to do with "why" .

Yes, personally, I'm very interested in why mozilla would use version X 
of nss + nspr for itself, and supply version Y as "the system NSS and NSPR",
particularly when most linux distributions make them all link to the
same nss and nspr library files, which is kind of why I'm persuing this
bug report.

But why should users, who expect to download the "latest and greatest mozilla source" and build xulrunner + firefox, then find they've either :
 a) built firefox and xulrunner against different versions of nspr and nss
    to other installed NSS and NSPR software, so that POTENTIALLY there could
    be  incompatibilities and weird binary linkage issues ;
 b) built all their other NSPR and NSS using software so that it uses
    libraries not used by xulrunner and firefox and they have to maintain
    two versions of it
I just don't get it.

All I want to do is install ONE copy of nspr and nss libraries and headers , which MUST be the same ones that firefox and xulrunner were built against.

I'm modifying a version of client.mk and packager.mk now to do just that,
and will post back here when done.
(Reporter)

Comment 42

8 years ago
RE: Comment #37 :
> Instead we intend to keep the existing system of putting each
> xulrunner install in a separate directory (/usr/lib/xulrunner-N ...

Yes, fine, but why not enable users to use different layouts if they enter
a different non-default configuration argument ?

And how are they expected to get firefox, xulrunner, and all other  NSS + NSPR
using code to the same libraries if they want to ? 

The ONLY way that can happen at the moment is through post-installation hacks
and scripts that alter how xulrunner and firefox are installed - do you want
to encourage that rather than provide some way of supporting use of a single
copy of NSS, NSPR + XUL libs by applications ?
(Reporter)

Comment 43

8 years ago
The only thing I'm suggesting, at a minimum, is to provide some configuration
option that will at least clean up the duplicate libraries installed and make
everything link to one copy . 

To me, the cleanest way of doing that is by escaping their creation in the first
place , and to just create one copy of system shared libraries .

If you're doing that, you might as well provide some "system NSPR + NSS" install
option, because effectively that's what you've done, just not in expected system
locations.

Maybe I should just keep these client.mk and packager.mk scripts to myself and
not bother posting here or discussing on this bug report if there's really no
interest of the community in any of this, but I find that hard to believe.

Comment 44

8 years ago
*If* you do this:

* build NSPR from its sources and install
* build NSS from its sources and install
* build xulrunner in obj-xulrunner --with-system-nspr --with-system-nss
* build firefox in obj-firefox --with-libxul-dir --with-system-nspr --with-system-nss

You will get exactly what you want, I think: XR/FF will build against the system headers and libraries, and not against the in-tree versions, and you shouldn't have duplicate libraries by default.

AFAICT this bug is either misguided or invalid.
(Reporter)

Comment 45

8 years ago
RE: Comment #44 :
>  * build NSS from its sources and install
The mozilla source give no means of doing so easily - but I've adapted
the nss configure.in & Makefile.in to install from a mozilla build -
I'm NOT going down the route of supporting two NSS + NSPR distros - 
that should be MY decision, not mozilla's .

> you shouldn't have duplicate libraries by default.

But I don't see how - if the make file always creates links to them in 
dist/{firefox,xulrunner}/lib*.so , then tar always copies them ? 

I mean, I see how LIBXUL_DIST is being set to the  --with-libxul-sdk value
in configure.in, and then it is used in xpcom/stub/Makefile.in to prefix
all the libraries written to dist/firefox/dependantlibs.list ; but after
the xulrunner build and install completed, I put these options in 
.mozconfig :
ac_add_options --with-libxul-sdk=/usr/lib64/xulrunner-devel-2.0b13pre
ac_add_options --with-system-libxul
ac_add_options --with-system-nspr
I do 'make -f client.mk' to reconfigure, and then note:
  $OBJDIR/dist/firefox DOES NOT EXIST AT THIS POINT 
Then I build firefox, and get a dist/firefox/dependentlibs.list :

$ cat ~/x86_64/dist/firefox/dependentlibs.list 

libnspr4.so
libplc4.so
libplds4.so
libmozalloc.so
libnssutil3.so
libsoftokn3.so
libnss3.so
libssl3.so
libsmime3.so
libmozsqlite3.so
libxul.so

And /usr/lib64/firefox-4.0b13pre gets its own copies of the libraries. 

Why, when everything has been reconfigured with --with-libxul-sdk-dir etc. ,
and NO firefox dir exists, does it create firefox/dependantlibs.list entries 
NOT prefixed by $(LIBXUL_DIST) ?

This is the problem I'm hoping to fix with the new configure.in and client.mk .

I'm actually glad that the build is capable of re-using the NSPR and NSS 
and libxul libraries just built , because it makes it very simple to add an "install system NSPR and NSS libs" option either as a configure option or as
separate script that that be run after a successful xulrunner and/or firefox build - I just wish it would use the new LIBXUL_DIST I reconfigured to prefix 
the libraries , and create the firefox/xulrunner link not to ../xulrunner, but
to ../xulrunner-$XULRUNNER_VERSION , when --with-libxul-sdk and --enable-system-libxul are given.

So then, what mozilla is doing is saying "the location of the system LIBXUL, NSPR and NSS libs is /usr/lib64/xulrunner-$XULRUNNER_VERSION" - that's OK,
but I'd still like the option to be able to use /usr/lib64/lib{nspr4,nss3}* or even  /lib64/lib{nspr4,nss3}* as Fedora is doing it nowadays.

Comment 46

8 years ago
NSS and NSPR sources are included in the Mozilla sources *only* for the in-tree case. You *should not* try to build system-NSPR and system-NSS using these sources. It's not the canonical location.

Please ignore depedentlibs.list, it is correct but irrelevant.

The *only* bug here, if there is a bug, is "And /usr/lib64/firefox-4.0b13pre gets its own copies of the libraries."

Are you using *separate objdirs" for the XR build and the FF build? You keep talking about dist/xulrunner and dist/firefox as if they were in the same tree, which is incorrect.
(Reporter)

Comment 47

8 years ago
Like, maybe this whole bug could be fixed if I could find the places in 
the make code it where it looks for an existing dist/xulrunner and creates 
that 
  dist/firefox/xulrunner -> ../xulrunner link
and
  firefox/dependantlibs.list , and, if $LIBXUL_SDK_DIR is set, prepends that
to LIBXUL_DIST so it prefixes each lib written to dependentlibs.list , then 
those firefox/lib*.so would NOT be created and tar would NOT create those
copies ?

That has to be a solution worth investigating ...

But that still leaves the issue of --enable-system-nspr-install and --enable-system-nss-install, which I'll have to implement with a post-build hack I guess
if no-one wants it in the mozilla build.
(Reporter)

Comment 48

8 years ago
RE: Comment #46 : 
 You keep talking about dist/xulrunner and dist/firefox as if they were in the 
 same tree, which is incorrect.

Yes, I'm trying to build from the same tree and fix the issues that arise ; 
I don't see how it can be "incorrect" to want to re-use the objects you've
just spent @ 4 hours building by re-configuring an OBJDIR - that is what
configure is so great at doing, ensuring dependencies get rebuilt - and they
are, all correctly, and firefox is built OK, the ONLY problems that arise
are the ones described above, and I don't quite understand why no-one thinks
they are worthwhile fixing . Why shouldn't you be able to reconfigure
to build firefox in an OBJDIR after you've built xulrunner ? There are
only a few minor tweaks required, it now seems, to get this to work .
(Reporter)

Comment 49

8 years ago
Forget replacing tar with per-item makefile rules - that would be, IMHO, the
right way forward to make mozilla installs fully configurably relocatable
to honor all autoconf variables, but if you don't ever want to do that,
fair enough. But if only these simple problems need to be fixed to enable
rebuilding firefox after xulrunner in the same $OBJDIR, why not enable it ?
(Reporter)

Comment 50

8 years ago
--with-libxul-sdk=/usr/lib64/xulrunner-devel-2.0b13pre

Aha ! It was because :

  $ ls /usr/lib64/xulrunner-devel-2.0b13pre/lib*so*
  $

The libraries for xulrunner are shipped to 

    /usr/lib64/xulrunner-2.0b13pre/lib*.so

but I HAVE to set --with-libxul-sdk=/usr/lib64/xulrunner-devel=2.0b13pre
in order for configure.in to allow the argument - otherwise 
it does not accept '--with-system-libxul' because 
$LIBXUL_SDK/include/xpcom-config.h is not found !!

So it is just a configure.in problem -

change configure.in to :
  o strip out any 'devel' from $LIBXUL_DIST set from $LIBXUL_SDK_DIR !!
(Reporter)

Comment 51

8 years ago
But actually, will that be enough ? Because xulrunner-2.0b13pre/{bin,include} etc
don't exist ? 
:
$ cd /usr/lib64
$ ls -ld xulrunner-2.0b13pre/*/. xulrunner-devel-2.0b13pre/*/.
drwxr-xr-x  3 firefox firefox  4096 Apr 15 17:16 xulrunner-2.0b13pre/chrome/.
drwxr-xr-x  2 firefox firefox  4096 Apr 15 17:15 xulrunner-2.0b13pre/components/.
drwxr-xr-x  5 firefox firefox  4096 Apr 15 13:58 xulrunner-2.0b13pre/defaults/.
drwxr-xr-x  2 firefox firefox  4096 Apr 15 17:16 xulrunner-2.0b13pre/dictionaries/.
drwxr-xr-x  2 firefox firefox  4096 Apr 15 13:58 xulrunner-2.0b13pre/icons/.
drwxr-xr-x  2 firefox firefox  4096 Apr 15 17:16 xulrunner-2.0b13pre/modules/.
drwxr-xr-x  2 firefox firefox  4096 Apr 15 17:15 xulrunner-2.0b13pre/plugins/.
drwxr-xr-x  6 firefox firefox  4096 Apr 15 17:16 xulrunner-2.0b13pre/res/.
drwxr-xr-x 10 root    root     4096 Apr 16 14:43 xulrunner-devel-2.0b13pre/bin/.
drwxr-xr-x  2 firefox firefox 61440 Apr 15 13:58 xulrunner-devel-2.0b13pre/idl/.
drwxr-xr-x 19 firefox firefox 61440 Apr 15 13:58 xulrunner-devel-2.0b13pre/include/.
drwxr-xr-x  2 firefox firefox  4096 Apr 15 13:56 xulrunner-devel-2.0b13pre/lib/.
drwxr-xr-x  3 root    root     4096 Apr 15 17:17 xulrunner-devel-2.0b13pre/sdk/.


I guess I should make 

  xulrunner-2.0b13-pre/{bin,idl,lib,sdk,include} links to 
  xulrunner-devel-2.0b13-pre/{bin,idl,lib,sdk,include} ? 

Where in the installer should that go if we're using tar , though ? hmm...
(Reporter)

Comment 52

8 years ago
Wow, some more surprises :

[ firefox@jvdspc:/usr/lib64/xulrunner-devel-2.0b13pre 00:17:47 1275:773 ]
$ ls -l /usr/lib64/xulrunner-devel-2.0b13pre/lib
lrwxrwxrwx 1 root root 44 Apr 15 17:18 /usr/lib64/xulrunner-devel-2.0b13pre/lib -> /usr/lib64/xulrunner-devel-2.0b13pre/sdk/lib
[ firefox@jvdspc:/usr/lib64/xulrunner-devel-2.0b13pre 00:19:05 1276:774 ]
$ ls -l /usr/lib64/xulrunner-devel-2.0b13pre/sdk/lib
total 424280
-rw-r--r-- 1 firefox firefox    822986 Apr 15 04:29 libcrmf.a
-rwxr-xr-x 1 firefox firefox     19217 Apr 15 00:16 libmozalloc.so
-rw-r--r-- 1 firefox firefox    292436 Apr 15 00:21 libmozreg_s.a
-rwxr-xr-x 1 firefox firefox   1019310 Apr 14 23:41 libnspr4.so
-rw-r--r-- 1 firefox firefox    248054 Apr 15 04:29 libnss.a
-rw-r--r-- 1 firefox firefox    730188 Apr 15 04:27 libnssutil.a
-rwxr-xr-x 1 firefox firefox     64657 Apr 14 23:41 libplc4.so
-rwxr-xr-x 1 firefox firefox     41478 Apr 14 23:41 libplds4.so
-rw-r--r-- 1 firefox firefox   1189336 Apr 15 04:29 libsmime.a
-rw-r--r-- 1 firefox firefox   2037838 Apr 15 04:29 libssl.a
-rw-r--r-- 1 firefox firefox    120440 Apr 15 00:22 libunicharutil_external_s.a
-rwxr-xr-x 1 firefox firefox     44132 Apr 15 13:56 libxpcom.so
-rw-r--r-- 1 firefox firefox   1626964 Apr 15 00:18 libxpcomglue.a
-rw-r--r-- 1 firefox firefox   2186856 Apr 15 00:18 libxpcomglue_s.a
-rw-r--r-- 1 firefox firefox   2166024 Apr 15 00:18 libxpcomglue_s_nomozalloc.a
-rwxr-xr-x 1 firefox firefox 421358434 Apr 15 13:50 libxul.so


So not only did the installer install /usr/lib64/firefox-4.0b13pre/lib*.so, but
also  /usr/lib64/xulrunner-2.0b13pre/lib*.so AND /usr/lib64/xulrunner-devel-2.0b13pre/sdk/lib/lib*.so !! 
No wonder it ran out of disk space !!
(Reporter)

Comment 53

8 years ago
And I THINK this is all because xulrunner-devel-2.0b13 did not have any
lib*.so created in it ( it did, but under it's sdk/lib/ ), while
xulrunner-2.0b13 DOES get lib*.so, but no include or sdk or bin,
so cannot be used as a --with-libxul-sdk=  value. NICE !

What is meant to be the idea of xulrunner-devel-2.0b13 ? Isn't xulrunner
a development tool , so why a separate -devel directory ? Can't I just
remove all '-devel-' references, which I think are only :

[ firefox@jvdspc:~/src 00:53:34 1280:778 ]
$ grep -Hn '\-devel-' config/autoconf.mk.in js/src/config/autoconf.mk.in
config/autoconf.mk.in:70:sdkdir         = $(libdir)/$(MOZ_APP_NAME)-devel-$(MOZ_APP_VERSION)
js/src/config/autoconf.mk.in:65:sdkdir          = $(libdir)/$(MOZ_APP_NAME)-devel-$(MOZ_APP_VERSION)

Why the -devel- ? 

I think this is messing up creating the dependentlibs.list because
LIBXUL_DIST does NOT get set to a directory containing lib*.so !!
Please can anyone clarify ? thanks...
(Reporter)

Comment 54

8 years ago
Created attachment 527301 [details] [diff] [review]
patch to src/tookit/mozapps/installer/packager.mk

OK, this patch to packager.mk seems to fix all my problems with the installation 
, so it is the simplest way out for me.

packager will now honor these environment variable settings:
 o MOZ_TAR_BYPASS=1 
   - enables new "make rule install" rather than old "tar install"

 o MOZ_INSTALL_UNIQUE_SHLIBS=1 ; MOZ_UNIQUE_SHLIBDIR=/usr/lib/mozilla_libs; 
   - enables the installer to ensure that it only ever installs one
     copy of a shared library, by installing them to $MOZ_UNIQUE_SHLIBDIR
     if they do not exist there.

As this does seem to install everythink OK on my system, I'm OK with this 
version - but it probably needs more testing. I'd be amazed if mozilla 
wanted to incorporate this patch, but at least I tried.
Attachment #527301 - Flags: review?
(Reporter)

Comment 55

8 years ago
Created attachment 527324 [details] [diff] [review]
patch to src/tookit/mozapps/installer/packager.mk

oops, this version DOES create links to $LIBDEST in original locations for shlibs
Attachment #527301 - Attachment is obsolete: true
Attachment #527301 - Flags: review?
(Reporter)

Comment 56

8 years ago
Created attachment 527352 [details] [diff] [review]
patch to src/tookit/mozapps/installer/packager.mk

better, safer version - NOT setting $MOZ_INSTALL_UNIQUE_SHLIBS DISABLES special shlib handling .

BTW, it is quite frightening seeing the output of make in syntax check mode
for this file with no variables set:

$ make -kn -f packager.mk 
packager.mk:41: /toolkit/mozapps/installer/package-name.mk: No such file or directory
make: *** No rule to make target `/toolkit/mozapps/installer/package-name.mk'.
make: Failed to remake makefile `/toolkit/mozapps/installer/package-name.mk'.
rm -rf / /.tar /.dmg stage-package
echo "Creating package directory..."


If I hadn't appended '-n' to the make arguments this could have removed 
my entire filesystem !!
Attachment #527324 - Attachment is obsolete: true
(Reporter)

Comment 57

8 years ago
Created attachment 527361 [details] [diff] [review]
patch to src/tookit/mozapps/installer/packager.mk

better version; make sdk_links depend on $(DESTDIR)/$(sdkdir)/?/.
Attachment #527352 - Attachment is obsolete: true
(Reporter)

Comment 58

8 years ago
Created attachment 527364 [details] [diff] [review]
patch to src/tookit/mozapps/installer/packager.mk

sorry for repost, this should be near final version now - this handles case
where "$@" = "$(DESTDIR)/.../$(shlib)" better.
Attachment #527361 - Attachment is obsolete: true

Comment 59

8 years ago
I believe that our understanding and goals are different enough that leaving this bug open will help nobody. Building multiple apps (XR and FF) in the same objdir is a complete non-goal, to the point where I would take a patch to throw an error if you tried it.
Status: UNCONFIRMED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → WONTFIX
(Reporter)

Comment 60

8 years ago
Created attachment 527367 [details] [diff] [review]
patch to src/tookit/mozapps/installer/packager.mk

final beta version 0.0.1
Attachment #527364 - Attachment is obsolete: true
(Reporter)

Comment 61

8 years ago
RE: Comment #59 - but I'm not the only user complaining about this :


See Comment #3 :    Mike Hommey [:glandium] 2011-04-18 08:27:35 PDT

From my point of view, there is only one problem in the current install
process: when using one or more of nspr, nss and libxul sdk from an external
directory, make install will install a copy of these. These should probably be
symbolic links instead.



It seems to that with the current tar-based installer, there is NO way to
change the target of a shared library in dist/X/X to a resulting link 
in the final $DESTDIR filesystem .


The patch is a first attempt at allowing users to specify that they
want some shared libraries to be installed as symlinks - that's all.

If you don't agree, fair enough, but please debate why here - don't 
just close the discussion.

This doesn't have to do with re-using the OBJDIR - though with the
patch, if you want to do so, specifying "MOZ_TAR_BYPASS=1" and
"MOZ_INSTALL_UNIQUE_SHLIBS=1" will safely allow you to do so.
Status: RESOLVED → UNCONFIRMED
Resolution: WONTFIX → ---

Updated

8 years ago
Status: UNCONFIRMED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → WONTFIX
(Reporter)

Comment 62

8 years ago
And also, if you really intend to prohibit building totally building
more than one app in the same objdir , please say so loudly in your 
build instructions :
try searching for 'OBJDIR' on :
https://developer.mozilla.org/en/Build_and_Install
https://developer.mozilla.org/En/Developer_Guide/Build_Instructions
and you find no mention of this issue.

I had to google 'site:developer.mozilla.org OBJDIR' to find, in a tiny
font at the bottom of the page, on:
https://developer.mozilla.org/en/Mozilla_Build_FAQ

"Can I build multiple Mozilla-based projects from a single source tree?
    Yes! Each project must be built in its own objdir.
"

But it does ! : $OBJDIR/browser !!

And I'm speaking to you via the excellent browser that resulted from such
a build - just please, provide some way of cleaning up the duplicate
libraries installed!
(Reporter)

Updated

8 years ago
Status: RESOLVED → UNCONFIRMED
Resolution: WONTFIX → ---

Comment 63

8 years ago
no. do not reopen this bug
Status: UNCONFIRMED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → WONTFIX
(Reporter)

Comment 64

8 years ago
Created attachment 527417 [details] [diff] [review]
patch to src/tookit/mozapps/installer/packager.mk

create $(DESTDIR)/bin properly
Attachment #527367 - Attachment is obsolete: true
(Reporter)

Comment 65

8 years ago
RE: Comment #63 : > no. do not reopen this bug

OK, but I now have to rebuild everything from scratch for the i686 libxul +
NSS + NSPR - I will now do so, and append a log of the shared library files
and links created by the patched packager.mk  .

Of course, now I know , I won't be using a shared $OBJDIR . That was never
the point of this bug report .

Wow , now I really understand why your build system hasn't changed in 12 years !
(Reporter)

Comment 66

8 years ago
Had to build alot of i686 dependencies - in fact, still doing so, so this may
take some time.

But to sum up, and to underscore what I consider to be the problem here :

In order to get firefox working, I had to do :

   $ cd /usr/lib64
   $ mkdir nspr
   $ find firefox-4.0b13pre xulrunner-2.0b13pre xulrunner-devel-2.0b13pre -type l |  while read l; do f=$l; while [ -L $f ]; do f=$(readlink $f); done; echo $f; rm -f $l; done | cpio -o -H ustar | tar -xpf - ;
   # ^- this copied some needed stuff from the source tree 
   $ find firefox-4.0b13pre xulrunner-2.0b13pre xulrunner-devel-2.0b13pre -name 'lib*.so' -a type f | while read f; do 
   if  ! file $f | egrep 'shared object' ; then echo 'BAD LIB: ' $; else
    if [ ! -e nspr/$f ]; then mv $f nspr; else rm -f $f; fi
   fi ; done
   $ cp -fp firefox-4.0b13pre/dependent.libs nspr

After several iterations of that, cleaning up 'BAD LIB:' duplicate
shared libraries that tar crashed trying to install because it ran 
out of disk space,  firefox works fine.  Thanks !

And I understand now that no-one at mozilla thinks any of the above 
was a problem. OK - so mozilla is the only package that requires patches
to its installation system on my host.
(Reporter)

Comment 67

8 years ago
Oops, that should have been :

$ find firefox-4.0b13pre xulrunner-2.0b13pre xulrunner-devel-2.0b13pre -type
l |  while read l; do f=$l; while [ -L $f ]; do f=$(readlink $f); done; echo
$f; rm -f $l; cd ${l#*/}; if [ -d $f ]; then tar -cpf - $f | tar -xpf -; else cp -fp $f ${l#/}; fi ; done
(Reporter)

Comment 68

8 years ago
$ find firefox-4.0b13pre xulrunner-2.0b13pre xulrunner-devel-2.0b13pre -type
l |  while read l; do f=$l; while [ -L $f ]; do f=$(readlink $f); done; echo
$f; rm -f $l; cd ${l#*/}; if [ -d $f ]; then tar -cpf - $f | tar -xpf -; else
cp -fp $f ${l#*/}; fi ; done
(Reporter)

Comment 69

8 years ago
Created attachment 529349 [details] [diff] [review]
patch to src/tookit/mozapps/installer/packager.mk to provide "MOZ_TAR_BYPASS" support

OK, as I said I would, I'm appending the final, tested version of the installer,
which uses a separate make rule and GNU coreutils install to install each
install target on the host .
Next I'll attach the make log from a successful xulrunner install :

$ make install DESTDIR=/tmp/test_install LIBDIR=/usr/lib32 MOZ_TAR_BYPASS=1 \
  MOZ_INSTALL_UNIQUE_SHLIBS=1 
Mozilla UNIX install will not use tar(1) - bypass enabled
Mozilla UNIX install: Library files will be installed in /tmp/test_install/xulrunner-2.0b13pre//usr/lib32
make[1]: Entering directory `/home/firefox/x86/xulrunner/installer'
Makefile:112: FULL_NSPR_CFLAGS=-I\${includedir}
Creating package directory...
Attachment #527417 - Attachment is obsolete: true
(Reporter)

Comment 70

8 years ago
Created attachment 529350 [details]
log file from ' make install MOZ_TAR_BYPASS=1 MOZ_INSTALL_UNIQUE_SHLIBS=1 '

Updated

7 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.