1231577 - setting a nondefault MOZ_OBJDIR doesn't build (file not found under ./mozilla)

Status: RESOLVED INVALID

(SeaMonkey :: Build Config, defect)

Description

[linuxcbon]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39
Build ID: 20151210023514

Steps to reproduce:

- create .mozconfig containing :
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/my-object-dir
- run make -f client.mk build


Actual results:

/seamonkey-2.39/mozilla/configure: line 88: Error: command not found
/seamonkey-2.39/mozilla/configure: line 90: [environment,: command not found
Failed eval'ing the following:
Error running mach:
    ['environment', '--format=configure']


Expected results:

build ok

Comment 1

[Ian Neal]

Looking at https://developer.mozilla.org/en-US/docs/Simple_SeaMonkey_build have you tried setting an object directory path that does not contain environment variables?

Comment 2

[Tony Mechelynck [:tonymec]]

(In reply to Ian Neal from comment #1)
> Looking at https://developer.mozilla.org/en-US/docs/Simple_SeaMonkey_build
> have you tried setting an object directory path that does not contain
> environment variables?

According to https://developer.mozilla.org/en-US/docs/Configuring_Build_Options#Building_with_an_Objdir , the default (and recommended) value is
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@

The downside of this is that, in rare cases, the @CONFIG_GUESS@ may change. IIRC, when I still at times compiled SeaMonkey (it took all night and needed an additional swapfile set in advance, or the libxul link phase would fail) the value was IIRC .../obj-x86-unknown-linux-gnu or something not much different. But earlier I'd had another value without the "unknown" but I don't remember what I got instead.

@linuxcbon: what is the exact text of the configure lines which give an error, and of a few lines around them? (In the axample in comment #0, let's say lines 85 to 93 of the Firefox configure at .../seamonkey-2.39/mozilla/configure

BTW, oughtn't that seamonkey-2.39 directory, which IIUC is a comm-release clone, be named comm-release instead? Though I don't think the name change makes a difference.

One possible thing to try is to force a full build by removing the objdir and all its contents. (The next run of make ought to rebuild it.) Or if even that doesn't work, start afresh with a comm-central or comm-aurora build instead of comm-release: at least the present state of these two repositories is known to be healthy enough to produce nightly builds for Linux on the automated build machines, so using them will avoid trying to build from a source which doesn't build.

Comment 3

[linuxcbon]

It seems my build environment had an unknown problem, can't reproduce the error...I close the bug.

Comment 4

[linuxcbon]

I finally found how to reproduce it.

I only use 2 options in .mozconfig :
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/1234
ac_add_options --disable-gstreamer

# make -f client.mk build
I get 
OSError: [Errno 2] No such file or directory: '/home/tototo/seamonkey-2.39/mozilla/1234'

# mkdir mozilla/1234
# make -f client.mk build
I get
/home/tototo/seamonkey-2.39/mozilla/configure: line 88: Ambiguous: command not found
Failed eval'ing the following:
Ambiguous object directory detected. We detected that both /home/tototo/seamonkey-2.39/1234 and /home/tototo/seamonkey-2.39/mozilla/1234 could be object directories. This is typically caused by having a mozconfig pointing to a different object directory from the current working directory. To solve this problem, ensure you do not have a default mozconfig in searched paths.

I am lost here.

Comment 5

[linuxcbon]

I can confirm there is a bug.
Without the MOZ_OBJDIR line, the compilation runs.

Comment 6

[Tony Mechelynck [:tonymec]]

(In reply to linuxcbon from comment #4)
> I finally found how to reproduce it.
> 
> I only use 2 options in .mozconfig :
> mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/1234
> ac_add_options --disable-gstreamer
> 
> # make -f client.mk build
> I get 
> OSError: [Errno 2] No such file or directory:
> '/home/tototo/seamonkey-2.39/mozilla/1234'
> 
> # mkdir mozilla/1234
> # make -f client.mk build
> I get
> /home/tototo/seamonkey-2.39/mozilla/configure: line 88: Ambiguous: command
> not found
> Failed eval'ing the following:
> Ambiguous object directory detected. We detected that both
> /home/tototo/seamonkey-2.39/1234 and
> /home/tototo/seamonkey-2.39/mozilla/1234 could be object directories. This
> is typically caused by having a mozconfig pointing to a different object
> directory from the current working directory. To solve this problem, ensure
> you do not have a default mozconfig in searched paths.
> 
> I am lost here.

Hmmm.. AFAICT, when generating and running configure for the mozilla-release clone grafted at ./mozilla/ on your comm-release clone, the build phase somehow forgets to set TOPSRCDIR to your top comm-release directory, i.e. to the _parent_ of the top mozilla-release directory.

To me this smells of using a set of repository changesets where SeaMonkey (or at least the code it borrows from Firefox) won't build at all because the build structure is wrong.

Ohhhh... STOP PRESS: Tou did not request --enable-application=suite ! This tries to build Firefox from the comm-release directory, which is of course totally wrong. I'm setting INVALID for this reason, but nevertheless I'm explaining in more detail what to do if adding that line is not enough to get a working build.

Are you sure you were using autoconf 2.13 and not the most recent version? This is important because more recent autoconf versions are incompatible with the Mozilla build system: so make sure that the autoconf213 package is installed and that "autoconf-2.13 --version" answers "Autoconf version 2.13". Trying to symlink the autoconf-2.13 name to some more recent version WILL NOT WORK.

If you do indeed have that version, then I suggest that you first try building some SeaMonkey version which is known to be buildable, so I suggest the following (logging via e.g. "2>&1 |tee -a build.log" is optional but recommended; I haven't mentioned it below). Also, this assumes a bash shell or compatible; step 1 and possibly some other steps will have to be written differently for the csh/tcsh shells.
1. export PYTHONUNBUFFERED=unbuffered
-- this makes sure that the output from Python commands later on (including Mercurial and client.py) will be displayed in the exact order they were emitted.
2. cd ~
3. hg clone -U hg.mozilla.org/comm-central/
-- This sets up a "comm-central" directory below your home directory, and copies the comm-central metadata into it but doesn't yet set up the current directory. It may avoid timeouts during the clone process.
4. cd comm-central
5. (Optional) Edit the .hg/hgrc as you would like it for Mozilla; in particular the following settings are recommended if you don't yet have them at some higher level, such as in your ~/.hgrc (indeed some but not all of them would sit better in your ~/.hgrc):
  [diff]
  git          = true
  showfunc     = true
  unified      = 8
  
  [extensions]
  mq           = true
  pager        = true

  [pager]
  pager        = less
       # or   view -     or ...
  attend       = annotate, cat, diff, qdiff, export, help, log, glog, config

  [ui]
  editor       =
       # add your favourite editor; mine is vim
  merge        =
       # add your favourite diff editor; mine is vimdiff
  username     = linuxcbon <[e-mail]>
  verbose      = true
6. hg update -r tip
-- this requires no remote access; it just builds your local directory tree from the metadata downloaded at step 3.
7. python client.py checkout
-- this updates your comm-central tree (which at this point should be trivial) but it also set up, at the correct point in the tree, clones of everything else that will be needed for the build.
8. add the following mozconfig; it intentionally differs slightly from the one you gave at comment #4:
  echo "# SeaMonkey config file" > mozconfig
  echo "ac_add_options --enable-application=suite" >> mozconfig
  echo "mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@
  echo "ac_add_options --enable-optimize --disable-debug" >> mozconfig
  # or the opposite (--disable-optimize --enable-debug) if you want a debug build
9. The big one! Try building:
  date; make -f client.mk || echo 'exit status' $? ; date
-- The above displays a date-time stamp, runs the build, displays the exit status if nonzero, and finally another timestamp. IT SHOULD BE LOGGED even if nothing else is, because in my experience it may take a long time and the ultimate reason of failure is not always on the last page.
-- Of course, the following steps MUST NOT be attempted if building was not successful.
10. Build an installable distribution
  date; make -f client.mk package || echo 'exit status' $? ; date
11. If step 10 is successful, you can install the resulting .tar.bz2 just like you would one freshly downloaded from ftp.mozilla.org. If you already have a working SeaMonkey, don't install this one over it unless you indeed want to replace it, so here are two examples, both of which assume that you already have SeaMonkey in /usr/local/seamonkey/ :

1. cd to where the package lives (the .tar.bz2, .txt, .json and a few others). I think it is somewhere inside your object directory (whose name start with obj- and is at top level) but it might be lower down or even higher up.

(a) Install separately, under your home directory:
2. tar -jxvf seamonkey-2.43a1.*.tar.bz2 -C ~
3. pushd ~/bin
4. ln -svf ../seamonkey/seamonkey
5. popd

(b) Replace the existing /usr/local/seamonkey/ (after su --login root)
2. rm -Rvf /usr/local/seamonkey
3. tar -jxvf seamonkey-2.43a1.*.tar.bz2 -C /usr/local
4. ls -l /usr/local/bin/seamonkey

Notes:
- Step b.2 is important because obsolete files leftover from a previous install could otherwise make your newly installed build fail.
- If at step b.4 You don't see a soft link pointing to your newly installed build (typically to ../seamonkey/seamonkey) then a 5th step is necessary (only the 1st time):
b.5. pushd /usr/local/bin && ln -svf ../seamonkey/seamonkey && popd || echo 'exit status' $?

Comment 7

[Tony Mechelynck [:tonymec]]

In comment #6 at step 8 I forgot >> mozconfig on the OBJDIR-setting line. It should of course be added.

Comment 8

[linuxcbon]

# autoconf --version
autoconf (GNU Autoconf) 2.69

Here is another test case for seamonkey :

With these settings :
ac_add_options --disable-gstreamer
ac_add_options --prefix=/usr
ac_add_options --enable-application=suite
I could compile OK a 100% working seamonkey, which I am using right now for 
updating the bug report.

With these settings :
ac_add_options --disable-gstreamer
ac_add_options --prefix=/usr
ac_add_options --enable-application=suite
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/H4CK3R
I get error message :
OSError: [Errno 2] No such file or directory: '/home/tototo/seamonkey-2.39/mozilla/H4CK3R'

So the problem really comes from MOZ_OBJDIR.

I dont have hg or use it.
I tried this :
# export PYTHONUNBUFFERED=unbuffered
# date; make -f client.mk || echo 'exit status' $? ; date

I got as usual :
/home/tototo/seamonkey-2.39/configure
loading cache ./config.cache
/home/tototo/seamonkey-2.39/mozilla/configure: line 88: Error: command not found
...
OSError: [Errno 2] No such file or directory: '/home/tototo/seamonkey-2.39/mozilla/H4CK3R'
...
client.mk:375: recipe for target '/home/tototo/seamonkey-2.39/H4CK3R/Makefile' failed
make: *** [/home/tototo/seamonkey-2.39/H4CK3R/Makefile] Error 2
exit status 2

Comment 9

[Tony Mechelynck [:tonymec]]

(In reply to linuxcbon from comment #8)
> # autoconf --version
> autoconf (GNU Autoconf) 2.69

Have you got autoconf 2.13 also? Autoconf 2.69 CANNOT be used to build SeaMonkey (or Firefox)

At the shell prompt, try to autocomplete
# autocon<Tab><Tab><Tab>

You should see the above autoconf (2.69) but also another version, usually named autoconf213 or autoconf-213 or autoconf-2.13 or something like that. The exact program name might depend on your Linux distro, on mine (openSUSE) it's autoconf-2.13 and when I check which version it thinks it is:

# autoconf-2.13 --version
Autoconf version 2.13

ONLY THIS AUTOCONF VERSION can be used to generate the configure scripts needed by Mozilla programs. Later versions DON'T work.

If you don't see autoconf 2.13, then find out if the corresponding package is available on your distro, or else check the Mozilla documentation:
https://developer.mozilla.org/en-US/docs/Simple_SeaMonkey_build#Installing_build_tools
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Linux_Prerequisites#Build_Tools

(The second link above explains how to install autoconf 2.13 from source if you cannot find it precompiled; it also explains that to buils Mozilla you MUST use Python 2, not Python 3, and what to do if your default "python" interpreter is Python 3.)

Comment 10

[Tony Mechelynck [:tonymec]]

Oh, and next time configure says "command not found", PLEASE copy the corresponding line of configure (which is a script generated by autoconf) and paste it into this bug.

Comment 11

[linuxcbon]

I don't have autoconf-2.13, I have autoconf 2.69. I don't think that's not the problem, since I can build seamonkey with any other options.
My python is 2.7.10

loading cache ./config.cache
/home/tototo/seamonkey-2.39/mozilla/configure: line 88: Error: command not found
/home/tototo/seamonkey-2.39/mozilla/configure: line 90: [environment,: command not found

configure lines show :
83: case "$ac_option" in
84:
85:  -bindir | --bindir | --bindi | --bind | --bin | --bi)
86:    ac_prev=bindir ;;
87:  -bindir=* | --bindir=* | --bindi=* | --bind=* | --bin=* | --bi=*)
88:    bindir="$ac_optarg" ;;
89:
90:  -build | --build | --buil | --bui | --bu)
91:    ac_prev=build ;;

whole file can be found at http://pastebin.com/cq5W0c5s

Comment 12

[Justin Wood (:Callek)]

(In reply to linuxcbon from comment #11)
> I don't have autoconf-2.13, I have autoconf 2.69. I don't think that's not
> the problem, since I can build seamonkey with any other options.
> My python is 2.7.10

autoconf 2.13 is indeed required, 2.69 has breaking/different changes that makes this all not work.

Comment 13

[Tony Mechelynck [:tonymec]]

So this bug is (at least for the time being) INVALID for having tried to build using unsupported tools (in this case: autoconf other than 2.13)

@linuxcbon: This is not finished.
1. Find and install autoconf 2.13. Maybe your linux distro has it as a separate package, in which case the easiest is to get it from there. If it doesn't, try finding it elsewhere, and as a last resort you can even compile it from source; see also both MDN links in comment #9.
2. In your mozconfig, change the objdir so that is starts @TOPSRCDIR@/obj- like the default name does. This makes it a directory which Mercurial will disregard on update.
3. Start from scratch:
  3a. Make a backup of your mozconfig outside the comm-* clone, just in case.
  3b. Remove everything that was put there by brevious build attempts: either
    make distclean (if the Makefile defines it), or
    make clean (otherwise), or
    you can also "clean house the big way", by starting a new clone, see comment 6. In that case you should make sure to clone something that does indeed build, and alas, there have been no Linux nightlies these latest days, so I recommend that you wait until there are, otherwise you might run into a build error which is, so to say, "not your fault".
  3c. If your mozconfig is not in the new (or cleaned-up) clone, put it back from the backup.
4. Try building, see comment #6.

If you run into the same problems after all that, come back here and give a full account of them. Maybe they'll warrant reopening the bug, or if they aren't really the same problems, then maybe opening a new bug would be less confusing for the people who have to fix it.

Comment 14

[Tony Mechelynck [:tonymec]]

P.S. Autoconf-2.13 can (and should) be installed in a way that does not conflict with the latest autoconf, and in particular with a different executable name and without uninstalling autoconf 2.69 (or later). This in order not to break anything other than Mozilla which would depend on using a "modern" autoconf.