config/autoconf.mk.in does not honor $CFLAGS, $CXXFLAGS, or $LDFLAGS correctly

UNCONFIRMED
Unassigned

Status

UNCONFIRMED
8 years ago
7 months ago

People

(Reporter: jason.vas.dias, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

8 years ago
User-Agent:       Mozilla/5.0 (X11; Linux x86_64; rv:2.0b13pre) Gecko/20110415 Firefox/4.0b13pre
Build Identifier: 

In order to build the i686 version of the xulrunner application on an x86_64, one has to edit src/config/autoconf.mk.in 
:
$  $ diff -U0 config/autoconf.mk.in.2.0b13pre config/autoconf.mk.in
--- config/autoconf.mk.in.2.0b13pre     2011-04-26 16:01:25.000000000 +0100
+++ config/autoconf.mk.in       2011-04-24 22:48:01.000000000 +0100
@@ -307,3 +307,3 @@
-OS_CFLAGS      = $(OS_CPPFLAGS) @CFLAGS@
-OS_CXXFLAGS    = $(OS_CPPFLAGS) @CXXFLAGS@
-OS_LDFLAGS     = @LDFLAGS@
+OS_CFLAGS      := $(OS_CPPFLAGS) $(CFLAGS)   @CFLAGS@
+OS_CXXFLAGS    := $(OS_CPPFLAGS) $(CXXFLAGS) @CXXFLAGS@
+OS_LDFLAGS     := @LDFLAGS@ $(LDFLAGS)
@@ -382,2 +382,2 @@
-CC                 = @CC@
-CXX                = @CXX@
+CC                 := $(if $(CC),$(CC),gcc)
+CXX                := $(if $(CXX),$(CXX),c++)


Reproducible: Always

Steps to Reproduce:
1. Try to build xulrunner with environment:

declare -x AS="/usr/bin/as -32"
declare -x ASFLAGS="-Wa,-32"
declare -x CFLAGS="-march=i686 -mtune=generic -g -O2 -fPIC -DPIC -Wa,--compress-debug-sections"
declare -x LD="/usr/bin/ld -melf_i386"
declare -x LDFLAGS="-Wl,-L/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.0/32,-L/usr/lib64/gcc/x86_64-pc-linux-gnu/lib32,-L/usr/lib32,-L/lib32,-R/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.0/32:/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.0/lib32:/usr/lib32:/lib32,--dynamic-linker,/lib32/ld-linux.so.2"
declare -x PATH="/bin:/usr/bin:."
declare -x PKG_CONFIG_PATH="/usr/lib32/pkgconfig/"

Actual Results:  
the build tries to use 'c++' as $CXX , selecting native 64-bit operation when
I'm trying to build for non-native 32-bit sub-architecture ; the build fails
at the first link attempt.


Expected Results:  
My $CXX, $CC, $CFLAGS, $CXXFLAGS, $LD, and $LDFLAG settings should have been used and written to $OBJDIR/config/autoconf.mk .

Comment 1

8 years ago
configure should be reading the environment, not make: we don't want to pick up stuff from the environment after configure has been run.

I don't think this bug is valid: could you please check whether this happens if you set up the environment properly before running configure?
(Reporter)

Comment 2

8 years ago
Hi Ben - thanks for responding so quickly !

RE: > configure should be reading the environment, not make: 
    > we don't want to pick up stuff from the environment
    > after configure has been run.

I do :

1. Set up 32-bit build environment:

 $ . ~root/32.bit.env

 which is the 'declare -x' statements above -

$ export -p | egrep 'CC|CXX|LD|AS|FLAG'
declare -x AS="/usr/bin/as -32"
declare -x ASFLAGS="-Wa,-32"
declare -x CC="/usr/bin/gcc -m32"
declare -x CFLAGS="-m32 -march=i686 -mtune=generic -g -O2 -fPIC -DPIC -Wa,--compress-debug-sections"
declare -x CXX="/usr/bin/g++ -m32"
declare -x CXXFLAGS="-m32 -march=i686 -mtune=generic -g -O2 -fPIC -DPIC -Wa,--compress-debug-sections"
declare -x LD="/usr/bin/ld -melf_i386"
declare -x LDFLAGS="-m32 -Wl,-L/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.0/32,-L/usr/lib64/gcc/x86_64-pc-linux-gnu/lib32,-L/usr/lib32,-L/lib32,-R/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.0/32:/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.0/lib32:/usr/lib32:/lib32,--dynamic-linker,/lib32/ld-linux.so.2"
declare -x LD_RUN_PATH="/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.0/32:/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.0/lib32:/usr/lib32:/lib32:."


2. I edit my mozconfig :

$ cat ~/.mozconfig
mk_add_options AUTOCONF=autoconf-2.13
mk_add_options MOZ_OBJDIR=/home/firefox/x86
ac_add_options --enable-application=xulrunner
ac_add_options --prefix=/usr
ac_add_options --libdir=/usr/lib32
ac_add_options --disable-static
ac_add_options --enable-shared
ac_add_options --with-pic
ac_add_options --enable-debug
ac_add_options --enable-default-toolkit=cairo-gtk2
ac_add_options --enable-plugins
ac_add_options --with-pthreads
ac_add_options --with-system-libevent=/usr
ac_add_options --with-system-jpeg=/usr
ac_add_options --with-system-zlib=/usr
ac_add_options --with-system-bz2=/usr
ac_add_options --enable-system-cairo
ac_add_options --enable-system-lcms
ac_add_options --with-ft-prefix=/usr
ac_add_options --with-glib-prefix=/usr
ac_add_options --with-libIDL-prefix=/usr
ac_add_options --with-java-bin-path=/usr/java/bin
ac_add_options --with-java-include-path=/usr/java/include
ac_add_options --disable-strip
ac_add_options --disable-install-strip
ac_add_options --with-branding=unofficial
ac_add_options --disable-official-branding

3. I chdir to the mozilla source directory and run 'make -f client.mk' :

   $ cd ~firefox/src
   $ make -f client.mk

NOTE : $MOZ_OBJDIR == ~/x86 is a newly created, EMPTY directory !

That is all I do - no separate 'configure' step - and then my $CXX and $CXXFLAGS
settings are not used, unless I edit $topsrcdir/config/autoconf.mk.in to include
my $CC, $CXX, $CFLAGS, $CXXFLAGS, $LDFLAGS settings .

RE: > could you please check whether this happens if you set up the 
    > environment properly before running configure?

Please define "properly" here. And it is your client.mk script that is
responsible for setting up the environment in which configure is run, not me.
(Reporter)

Comment 3

8 years ago
Sorry, missed out mentioning 'setarch i686':

   

3. I chdir to the mozilla source directory and run 'make -f client.mk' :
   
$ uname -a
Linux jvdspc 2.6.38.2-jvd #1 SMP Sun Apr 3 00:55:29 BST 2011 x86_64 GNU/Linux

   $ setarch i686

$ uname -a
Linux jvdspc 2.6.38.2-jvd #1 SMP Sun Apr 3 00:55:29 BST 2011 i686 GNU/Linux

   $ . ~root/32.bit.env
   $ cd ~firefox/src
   $ make -f client.mk
(Reporter)

Comment 4

8 years ago
Also, your C++ code requires the '-fpermissive' option to compile under gcc-4.6.0 - I can raise a new bug on this if you like.

Comment 5

8 years ago
The environment is read here:
http://mxr.mozilla.org/mozilla-central/source/configure.in#83

and then substituted here:
http://mxr.mozilla.org/mozilla-central/source/configure.in#8920

Since this works in general, you probably need to add some echoes to the configure script to figure out where your values are being lost.

I typically configure like this for a linux cross-compile:

export PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig:/usr/share/pkgconfig
export CC="${CC:-gcc} -m32"
export CXX="${CXX:-g++} -m32"
export AR="ar"

configure_flags="$configure_flags --x-libraries=/usr/lib --target=i686-pc-linux"

export CROSS_COMPILE=1

$srcdir/configure $configure_flags "$@"
(Reporter)

Comment 6

8 years ago
one way to work-around this bug might be to do :

  $ make -f client.mk CC=/usr/bin/gcc' -m32' CXX=/usr/bin/g++' -m32' ...

etc. but I don't see why I should have to when these variables are 
already exported in my environment ;  to me, this would be part of 
fixing bug #648701 : 'Make the build steps familiar to new contributors' .
(Reporter)

Comment 7

8 years ago
Aha ! I think I found the fix : 
in client.mk, you need to :

   'export CC CXX CFLAGS CXXFLAGS LDFLAGS LD AS AR
   '

otherwise, these variables are NOT exported to commands run in rules , 
UNLESS you specify them on the make command line.

Comment 8

8 years ago
make -f client.mk CC=/usr/bin/gcc' -m32' CXX=/usr/bin/g++' -m32' sets Makefile variables, but not environment variables, so those won't end up affecting configure at all.

You could do CC=/usr/bin/gcc' -m32' CXX=/usr/bin/g++' -m32' make -f client.mk and that should work, I think. Like I said in the other bug, I haven't used client.mk in a long time, and it doesn't make much sense to use a mozconfig *and* environment variables separately. People normally use one or the other (if you're using mozconfig, just put the settings directly in your mozconfig file).
(Reporter)

Comment 9

8 years ago
RE: Comment #5:
 > I typically configure like this for a linux cross-compile

this is NOT a "cross-compile" - this is for a bi-arch build -
x86_64 is natively both 64-bit and 32-bit ; or I look at it
like 'the i686 architecture is a sub-architecture of x86_64' .
(Reporter)

Comment 10

8 years ago
re: Comment #8: see  $ info make 


5.7.2 Communicating Variables to a Sub-`make'
---------------------------------------------

Variable values of the top-level `make' can be passed to the sub-`make'
through the environment by explicit request.  These variables are
defined in the sub-`make' as defaults, but do not override what is
specified in the makefile used by the sub-`make' makefile unless you
use the `-e' switch (*note Summary of Options: Options Summary.).

   To pass down, or "export", a variable, `make' adds the variable and
its value to the environment for running each line of the recipe.  The
sub-`make', in turn, uses the environment to initialize its table of
variable values.  *Note Variables from the Environment: Environment.

   Except by explicit request, `make' exports a variable only if it is
either defined in the environment initially or set on the command line,
and if its name consists only of letters, numbers, and underscores.
Some shells cannot cope with environment variable names consisting of
characters other than letters, numbers, and underscores.


So you need to either explicitly export CC, CXX, CXXFLAGS , LD, LDFLAGS etc.
by 'export ...' statement(s) in client.mk or set them on the make command
line for them to be used in either a sub-make OR exported to the environment
of make rule commands.
(Reporter)

Comment 11

8 years ago
RE: 
  > if you're using mozconfig, just put the settings directly in 
  > your mozconfig file

This doesn't work:

  $ more ~/.mozconfig
  CXX=${CXX:=/usr/bin/g++ -m32}
  ...

Then client.mk barfs at the '{' character.
(Reporter)

Comment 12

8 years ago
typo in last comment:
  $ more ~/.mozconfig
  CXX=${CXX:-/usr/bin/g++ -m32}
  ...
( "=" -> '-' !)
(Reporter)

Comment 13

8 years ago
actually, I did try this :

 $ more ~/.mozconfig
mk_add_options CC:=$(if $(CC),$(CC),/usr/bin/gcc -m32)
mk_add_options CXX:=$(if $(CXX),$(CXX),/usr/bin/g++ -m32)

But then generated autoconf.mk gets "missing separator" errors .

And if these variables are not exported in client.mk it is in any case broken,
and how can you expect to fix bug #648701 if client.mk does not export them ?
(Reporter)

Comment 14

8 years ago
Suggested patch:
$ diff -U0  src/client.mk /tmp/client.mk
--- src/client.mk       2011-04-14 21:27:19.000000000 +0100
+++ /tmp/client.mk      2011-04-26 19:05:25.580642770 +0100
@@ -98 +98 @@
-AUTOCONF=$(error Couldn't find autoconf 2.13)
+AUTOCONF=$(error Could not find autoconf 2.13)
@@ -168,0 +169,34 @@
+# export every variable you want configure or sub-make scripts to pick up :
+export CC:=$(if $(CC),$(CC),cc)
+export CXX:=$(if $(CXX),$(CXX),c++)
+export CFLAGS:=$(if $(CFLAGS),$(CFLAGS))
+export CXXFLAGS:=$(if $(CXXFLAGS),$(CXXFLAGS))
+export LD:=$(if $(LD),$(LD))
+export LDFLAGS:=$(if $(LDFLAGS),$(LDFLAGS))
+export AS:=$(if $(AS),$(AS))
+export ASFLAGS:=$(if $(ASFLAGS),$(ASFLAGS))
+export PATH:=$(if $(PATH),$(PATH))
+export LD_LIBRARY_PATH:=$(if $(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH))
+export LD_RUN_PATH:=$(if $(LD_RUN_PATH),$(LD_RUN_PATH))
+export LD_PRELOAD:=$(if $(LD_PRELOAD),$(LD_PRELOAD))
+export PREFIX:=$(if $(PREFIX),$(PREFIX),/usr)
+export DESTDIR:=$(if $(DESTDIR),$(DESTDIR))
+export BINDIR:=$(if $(BINDIR),$(BINDIR),$(DESTDIR)/$(PREFIX)/bin)
+export SBINDIR:=$(if $(SBINDIR),$(SBINDIR),$(DESTDIR)/$(PREFIX)/sbin)
+is_native_64_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m64,$(CFLAGS)),$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return(sizeof(void*)==8)?0:1;} int main(){ return f();}' > $${f}.c; gcc $(CFLAGS) -o $$f $${f}.c && $$f && echo 1)))
+is_native_32_bit:=$(if $(filter -m32 -m64,$(CFLAGS)),$(filter -m32,$(CFLAGS)),$(strip $(shell f=$$(mktemp XXXXX); echo 'int f(){ return(sizeof(void*)==4)?0:1;} int main(){ return f();}' > $${f}.c; gcc $(CFLAGS) -o $$f $${f}.c && $$f && echo 1)))
+cc_v:=$(strip $(shell $(CC) --version))
+gcc_version:=$(strip $(shell echo '$(cc_v)' | sed -n '/[\(]GCC[\)]/{s/^.*[\ ]\([0-9]\.[0-9]\.[0-9]\)[\ ].*$$/\1/g;p}'))
+export LIBDIR:=$(strip \
+               $(if $(gcc_version),\
+                 $(if $(is_native_64_bit),$(shell cd $(DESTDIR)/$(PREFIX)/$(if $(LIBDIR),$(LIBDIR),/lib)/$$($(CC) -m64 -print-multi-os-directory);pwd),\
+                 $(if $(is_native_32_bit),$(shell cd $(DESTDIR)/$(PREFIX)/$(if $(LIBDIR),$(LIBDIR),/lib)/$$($(CC) -m32 -print-multi-os-directory);pwd),$(LIBDIR))\
+                  )\
+                )\
+               )
+export SYSCONFDIR:=$(if $(SYSCONFDIR),$(SYSCONFDIR),$(DESTDIR)/$(PREFIX)/../etc)
+export DATAROOTDIR:=$(if $(DATAROOTDIR),$(DATAROOTDIR),$(DESTDIR)/$(PREFIX)/share)
+export LIBEXECDIR:=$(if $(LIBEXECDIR),$(LIBEXECDIR),$(DESTDIR)/$(PREFIX)/libexec)
+export MANDIR:=$(if $(MANDIR),$(MANDIR),$(DATAROOTDIR)/man)
+export INFODIR:=$(if $(INFODIR),$(INFODIR),$(DATAROOTDIR)/info)
+
(Reporter)

Comment 15

8 years ago
Created attachment 528357 [details] [diff] [review]
patch against src/client.mk to export configure environment

Comment 16

8 years ago
If you exported the variables into the make environment, they would still be exported. This patch doesn't solve your problem, from what I can tell.
(Reporter)

Comment 17

8 years ago
Created attachment 528361 [details] [diff] [review]
patch against src/client.mk to export configure environment

better version: only set libdir to multi-os-directory if actually using gcc
Attachment #528357 - Attachment is obsolete: true
(Reporter)

Comment 18

8 years ago
RE: comment #16 : 
but one cannot have 
  'exported the variables into the make environment'
without some kind of patch unless you use 'make -e' .
I cannot fully test this yet as the firefox build
(with my modified autoconf.mk.in and packager.mk)
is still in progress, but I believe the latest version
of the patch would solve this problem.

Comment 19

8 years ago
Your shell exports them (that's what you said in comment 0). They should still be present in configure without make being involved.
(Reporter)

Comment 20

8 years ago
$ echo -e 'CXX=c++\nall:\n\tCXX is: $$CXX\n' > t652807.mk
$ CXX=g++ make -f t652807.mk
echo CXX is: $CXX
CXX is: c++

$ echo -e 'export CXX:=$(if $(CXX),$(CXX),c++)\nall:\n\techo CXX is: $$CXX\n' > t652807-2.mk
$ CXX=g++ make -f t652807-2.mk
echo CXX is: $CXX
CXX is: g++

QED
(Reporter)

Comment 21

8 years ago
oops:
$ echo -e 'CXX=c++\nall:\n\techo CXX is: $$CXX\n' > t652807.mk
$ CXX=g++ make -f t652807.mk
echo CXX is: $CXX
CXX is: c++

$ echo -e 'export CXX:=$(if $(CXX),$(CXX),c++)\nall:\n\techo CXX is: $$CXX\n' >
t652807-2.mk
$ CXX=g++ make -f t652807-2.mk
echo CXX is: $CXX
CXX is: g++

QED
(Reporter)

Comment 22

8 years ago
see, $CXX is defined as a 'deferred evaluation' variable by the
default autoconf.mk.in to be 'c++' ; either autoconf.mk.in must
be changed to 'export CXX:=$(if $(CXX),$(CXX),c++)' OR
client.mk must be changed to do the same for $CXX to
be inherited from the external shell environment of
"make -f client.mk", which runs "configure".
(Reporter)

Comment 23

8 years ago
incidentally, I think this is also why $OBJDIR is not being inherited from
the environment either, as for bug #648701 :
[reply] [-] Comment 56 Paul Biggar 2011-04-18 11:05:25 PDT

(In reply to comment #54)
> RE:  Comment #50:  OK, so autogen.sh should be just:
> 
> #/bin/bash
> export SRCDIR=${0%/*}
> export OBJDIR=`pwd`
> pushd $SRCDIR && autoconf-2.13 && ${SRCDIR}/configure $* && popd
> 
> ??

I would use `make -f client.mk configure-files` instead of calling autoconf.

And now I see why you wanted to create a .mozconfig file: so that `make -f
client.mk configure` would get it's values from somewhere. Hmmm. Is it
absolutely idiomatic that autogen.sh does the configure? It would be a lot
simpler if not.



See, the "simplest form of autogen.sh", ie. just :

   #  autoconf-2.13; ./configure

does NOT inherit $OBJDIR into client.mk and does NOT 'mkdir $OBJDIR; cd $OBJDIR'.

I think if $OBJDIR was correctly inherited from the external environment
into client.mk then configure would work as expected here.

Comment 24

8 years ago
OBJDIR is not a configure variable (at least currently), so that is entirely unrelated.

This discussion has totally run off the rails, because you're mixing up makefile variables in client.mk, environment variables in configure, and makefile variables in the build (autoconf.mk/rules.mk).

configure should (and I believe it already does) honor environment variables for CC/CXX/CFLAGS/etc. If it doesn't, that's a bug we should fix. If you think there's a bug in this regard, please provide very specific STR.

client.mk also happens to honor environment variables because it doesn't mess with the environment, so they should just end up in configure by default.

The main build (make, after configure has run) should *not* honor environment variables except those stored in autoconf.mk by configure, which is why the patch in comment 0 is incorrect.
(Reporter)

Comment 25

8 years ago

RE: > configure should (and I believe it already does) honor environment 
    > variables for CC/CXX/CFLAGS/etc. If it doesn't, that's a bug we should 
    > fix. If you think there's a bug in this regard, please provide very
    > specific STR.

Yes, neither configure nor client.mk honor $CC, $CXX or $CXXFLAGS, or $LDFLAGS -
 -  that is all this bug is about and has been the subject of every comment -
don't see what is "off the rails" about anything here - bug #648701 is about
user confusion caused by the configure and client.mk scripts to honor
autoconf variables such as $OBJDIR in the expected way, so I believe that
is relevant to this bug too.

What is "STR" -  the strings in the source files that need changing ? 
those are shown in the patches above, but I'll explain why again,
with, for instance "CXX" .

Neither configure nor client.mk can pass the value of the environment variable
"CXX" from the external invoking shell to a sub-make or rule command, because
"CXX" is defined to be a deferred evaluation variable at autoconf.mk.in 
@ line 377 :
"
# Temp hack.  It is not my intention to leave this **** in here for ever.
# Im talking to fur right now to solve the problem without introducing 
# NS_USE_NATIVE to the build system -ramiro.
NS_USE_NATIVE = @NS_USE_NATIVE@

CC		    = @CC@
CXX		    = @CXX@

CC_VERSION	= @CC_VERSION@
CXX_VERSION	= @CXX_VERSION@

GNU_AS		= @GNU_AS@
GNU_LD		= @GNU_LD@
GNU_CC		= @GNU_CC@
GNU_CXX		= @GNU_CXX@
"

what gets substituted for "@CXX@" here is "c++" , NOT the "/usr/bin/g++ -m32"
I had defined in the environment of the shell from which I ran 
"make -f client.mk" .

So currently EVERY setting for $VAR that you want to differ from the default
@VAR@ value written to autoconf.mk must be a make override on the command line.

All I'm suggesting is allowing users to specify values for variables that
get written to autoconf.mk in the environment of the invoking shell, 
as configure does when run outside the client.mk environment, writing
values of variables such as $CC, $CXX, $CFLAGS, $CXXFLAGS, and $LDFLAGS
to config.sh .

So either the first patch must be applied :
o autoconf.mk.in must read values to be written to autoconf.mk from the 
  environment
or something like the second patch must be applied:
o if the user has specified a value for one of the autoconf inherited
  variables, export that value to sub-makes and rule commands so
  that , in this case, when "configure" is run those values are 
  in the environment and override default settings for "@VAR@" variables
  so would also get written to autoconf.mk

Nowhere above did I suggest that after configure, the external shell environment should have any effect on anything. There just needs to
be a way of getting values such as "$CXX" and "$CC" from the
shell running "make -f client.mk" into $OBJDIR/config/autoconf.mk - there
isn't currently.

Comment 26

8 years ago
Jason, configure is what generates autoconf.mk. It has rules which set CXX based on the environment. I need you to add debugging printfs to configure to figure out why CXX isn't getting the value you expect. It works for most people.

(STR means "steps to reproduce").
(Reporter)

Comment 27

8 years ago
Sorry I can't fully test or investigate this right now -
after resolving various difficulties with the 32-bit
installations of openssl, openssh, curl, libssh2 and dbus that
caused my 32-bit xulrunner build to bomb out, it is now 
proceeding again . Once complete, I'll run 'bash -xf' / strace
to pin down exactly why variables such as $CXX are not being 
inherited by the client.mk environment, but I'm pretty certain
it is because they are not declared as 'export ' variables in client.mk.

Comment 28

4 years ago
I can confirm that a lot of targets do not honor LDFLAGS.

Updated

7 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.