js/src/aclocal.m4 includes files outside of topsrcdir, `./mach build` fails at configure
Categories
(Firefox Build System :: General, defect, P5)
Tracking
(firefox86 fixed)
Tracking | Status | |
---|---|---|
firefox86 | --- | fixed |
People
(Reporter: robwu, Assigned: robwu)
References
(Blocks 1 open bug)
Details
Attachments
(1 file)
I tried to run ./mach build
, but it consistently failed, even after running ./mach bootstrap
and clobber. It appears to be caused by the fact that the relative path in js/src/aclocal.m4
is resolved relative to the object directory, instead of js/src/
. Because I have nested topsrcdir in another (older/unrelated) topsrcdir, an incompatible file is included from the unrelated topsrcdir, which results in an error in the configure
step of the build.
I came to this conclusion as follows:
- The following directories are relevant:
- /path/to/gecko/ <- some old checkout, from before bug 1641786
- this is significant because of the old
build/autoconf/compiler-opts.m4
file containingdebug-label
.
- this is significant because of the old
- /path/to/gecko/worktree2/ <-- latest checkout from yesterday.
- /path/to/gecko/worktree2/objdir-debug/ <-- from mozconfig: MOZ_OBJDIR=./objdir-debug
- /path/to/gecko/ <- some old checkout, from before bug 1641786
- The output of
./mach build
(relevant parts below) refers towith-debug-label
. - Even after removing all traces of
with-debug-label
from repository, the same error returned, andwith-debug-label
magically appeared injs/src/old-configure
. - With
strace -o/tmp/trace -fs 10000 ./mach build
, I see that../../build/autoconf/compiler-opts.m4
is read, but that it contained an old version of the file, not from the topsrcdir that I am using.- The command that was being run is:
"/usr/bin/m4" "-I/path/to/gecko/worktree2/build/autoconf" "-I/path/to/gecko/worktree2/js/src" "-DAC_LOCALDIR=/path/to/gecko/worktree2/js/src" "autoconf.m4" "/path/to/gecko/worktree2/js/src/old-configure.in"
- Called from
/path/to/gecko/worktree2/build/autoconf/autoconf.sh --localdir="/path/to/gecko/worktree2/js/src" "/path/to/gecko/worktree2/js/src/old-configure.in"
- The command that was being run is:
- When I changed
MOZ_OBJDIR=./objdir-debug
toMOZ_OBJDIR=./objdir-prefix/objdir-debug
, the bug went away.
To reproduce, recreate the directory structure above (the compiler-opts.m4
file + a checkout in worktree2
) and run ./mach build
with the following mozconfig file:
ac_add_options --enable-debug-symbols
ac_add_options --enable-debug
mk_add_options MOZ_OBJDIR=./objdir-debug
OS: ArchLinux,
m4 (GNU M4) 1.4.18
Relevant parts of ./mach build
output:
0:03.24 Adding configure options from /path/to/gecko/worktree2/.mozconfig
0:03.24 --enable-debug-symbols
0:03.24 --enable-debug
...
0:21.76 creating ./config.data
0:21.79 js/src> Refreshing /path/to/gecko/worktree2/js/src/old-configure with /path/to/gecko/worktree2/build/autoconf/autoconf.sh
...
0:25.56 js/src> updating cache ./config.cache
0:25.57 js/src> creating ./config.data
0:25.60 js/src> ERROR: Missing option in `@old_configure_options` in /path/to/gecko/worktree2/js/sub.configure: --with-debug-label
...
Exception: Process executed with non-0 exit code 1: ['/path/to/.mozbuild/_virtualenvs/mach/bin/python', '/path/to/gecko/worktree2/configure.py']
Comment 1•4 years ago
|
||
I'm able to reproduce this as described when doing a ./mach build
from the nested source directory.
Assignee | ||
Comment 2•4 years ago
|
||
Linking this to mach-busted.
As reported, the work-around for developers is to change MOZ_OBJDIR
to a deeper directory.
It's not pretty, but it works until this is fixed.
Comment 3•4 years ago
|
||
Hey, this is a bit of an weird workflow. Why do you have two Firefox checkouts, one of them nested within the other?
Due to the resource constraints of the build team, I'm not sure that we'll be able to address this.
Assignee | ||
Comment 4•4 years ago
|
||
I have many of them to be able to work on multiple features in parallel.
I still have one copy of the .git
database due to the use of git worktree
.
Do you have an guesses why this bug is happening?
Comment 5•4 years ago
|
||
It's not clear to me why this is happening just yet.
A more common workflow is to have your projects adjacent to each other, like so:
/path/to/dev/
- gecko/
- [all of the files of a checkout]
- worktree2/
- [all of the files of another checkout]
If you move worktree2
to the parent directory, does the issue go away?
Assignee | ||
Comment 6•4 years ago
|
||
It's not clear to me why this is happening just yet.
That's unfortunate. My workflow had already been unblocked by the MOZ_OBJDIR=
workaround. Directory traversel outside of topsrcdir looks like dangerous, and I was hoping to learn whether this bug could potentially result in bad consequences. I acknowledge that you may not have time to look into this.
If you move
worktree2
to the parent directory, does the issue go away?
Of course, because the file lookup would fail. But it's just a work-around, like setting MOZ_OBJDIR
.
As for why the directories are nested: By having a common parent working dir, I can simply add a new worktree with:
$ cd /path/to/gecko/
$ git worktree add worktree2 HEAD
$ cd worktree2
To create a worktree in an adjacent directories, the command would just need to change to:
$ cd /path/to/gecko/
$ git worktree add ../worktree2 HEAD
$ cd ../worktree2
or
$ cd /path/to/
$ git -C gecko worktree add ../worktree2 HEAD
$ cd worktree2
Comment 7•4 years ago
|
||
it's just a work-around, like setting MOZ_OBJDIR.
Eh, I'd argue that the more common workflow with work trees is to have them adjacent to each other, not nested.
Directory traversel outside of topsrcdir looks like dangerous, and I was hoping to learn whether this bug could potentially result in bad consequences.
You're right that something is wrong here, and that is worth investigating. I just can't quite justify working on this right now (but, of course, I'd be happy to take a look at a patch!)
Updated•4 years ago
|
Assignee | ||
Comment 8•4 years ago
|
||
I know what's happening.
autoconf.sh
is called (via https://searchfox.org/mozilla-central/rev/014fe72eaba26dcf6082fb9bbaf208f97a38594e/build/moz.configure/old.configure#201-219)
- The working dir is the object dir (from
MOZ_OBJDIR
).
m4
program is called at https://searchfox.org/mozilla-central/rev/014fe72eaba26dcf6082fb9bbaf208f97a38594e/build/autoconf/autoconf.sh#124-125
- The working dir is still
MOZ_OBJDIR
m4
is passed-I/path/to/gecko/worktree2/js/src
as one of its parameters, which is supposedly used to look up inclusions fromaclocal.m4
. As a reminder,aclocal.m4
looks like this:.. builtin(include, ../../build/autoconf/compiler-opts.m4)dnl ..
- According to the documentation of
m4
, the working directory is read first - https://www.gnu.org/software/m4/manual/m4-1.4.15/html_node/Search-Path.htmlm4
allows included files to be found in other directories than the current working directory.If the
--prepend-include
or-B
command-line option was provided (see Invoking m4), those directories are searched first, in reverse order that those options were listed on the command line. Then m4 looks in the current working directory. Next comes the directories specified with the--include
or-I
option, in the order found on the command line. Finally, if theM4PATH
environment variable is set, it is expected to contain a colon-separated list of directories, which will be searched in order. - In short, the order of lookup is
--prepend-include
/-B
(not set), working directory (set toMOZ_OBJDIR
), and only then--include
/-I
(set)
To fix this, one of the following options look:
- Somehow replace
../../
with the absolute path to topsrcdir of (or prepend the directory ofjs/src/aclocal.m4
) in js/src/aclocal.m4 - Edit
autoconf.sh
to add--prepend-include
or-B
in the options. - Change
old.configure
to change the working directory to the location of the directory containingold-configure
I don't know how to do the first one (heck before today I didn't even know how m4 worked...),
the second option doesn't sound great because autoconf.sh
is a (vendored) copy from autoconf.
The third option doesn't look too bad, and seems to fix the issue. I'm going to attach a patch.
Assignee | ||
Comment 9•4 years ago
|
||
js/src/aclocal.m4 contains includes starting with ../../
.
As explained in https://bugzilla.mozilla.org/show_bug.cgi?id=1680862#c8,
m4 will first try to resolve this path relative to the working directory
and only if that path doesn't exists, fall back to the location set by
localdir
(from -I
).
The working directory is usually MOZ_OBJDIR, an immediate subdirectory
of topsrcdir, so ../../
resolves to a location outside of topsrcdir.
Usually, that path does not exist, and m4 falls back to localdir
that
was passed via -I
.
But if that path existed and is incompatible with the current Gecko
checkout, then the build will fail (see bug report). To prevent this
from happening, this patch fixes the working directory to localdir
,
so that m4 will immediately find the expected file.
Updated•4 years ago
|
Updated•4 years ago
|
Comment 10•4 years ago
|
||
Comment 11•4 years ago
|
||
bugherder |
Description
•