872077 - make mach work with objdirs placed anywhere

Status: NEW

(Firefox Build System :: General, defect)

Description

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Attachment: allow arbitrary objdir

One way that the current build system can work is, ignoring client.mk, you can do

  mkdir some-objdir
  cd some-objdir
  ../gecko/configure
  make

to set up a relative srcdir and put the objdir anywhere with whatever name.

I'd like to replicate that with mach, and on windows., e.g.:

  mkdir some-objdir && cd some-objdir
  ../gecko/mach build

There's a few gross bits that need to be munged, largely for the initial step of getting configure to run to set up the objecdir.  The attached patch seems to work, but has grossness.

The MOZ_OBJDIR ?= is weird, I think it can be there without the ifndefs, and maybe at the start of the file.  We just need to pull in $MOZ_OBJDIR from the environment if it's set.

The path munging is weird, and was basically happening because I had $PYTHON set in my environment (for a system install of python in c:\Python27).  It really needs to have backslashes, because some tools use it that only know about windows-style paths.

Comment 1

[Gregory Szorc [:gps]]

Comment on attachment 749341 [details] [diff] [review]
allow arbitrary objdir

Review of attachment 749341 [details] [diff] [review]:
-----------------------------------------------------------------

I like the idea, but I have concerns. Specifically, I worry that the supported scenarios for objdir selection are getting too complex. You have CWD, environment variables, mach in PATH, mozinfo files, ... . Where does it end?

I almost wonder if we are trending in a direction where scenarios like this require a |mach init| or something. I dunno.

As long as this doesn't regress existing workflows in obvious ways, I'll probably r+ it.

Comment 2

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Ah right, I already filed this bug. :/  This was fixed for "mach test" in bug 920679, but it's still an issue for "mach build", since that calls out to client.mk (is there a bug for killing client.mk?).

The patch here tries to support "../gecko/mach build" from an empty objdir, but it equally doesn't work if configure is run manually in the objdir, and mach build is used to build.  In that instance, mach finds the objdir, but client.mk doesn't, so it starts reconfiguring in srcdir/obj-foo.

Comment 3

[Gregory Szorc [:gps]]

At this point, I don't think anybody in build land cares for client.mk. Having |mach build| call client.mk is just a giant hack until we can afford the time to kill client.mk. We've been holding off because the payout doesn't seem too high and thus it's hard to justify against other work.

I will gladly r+ patches that start moving bits of client.mk to Python. Keep in mind release automation still uses client.mk, so all their interfaces need preserved. FWIW, making the interface between release automation and the build system more defined and possibly switching everything to use mach commands was a possible goal for Q4. Again, it has to take the back seat to e.g. build perf. Give us more people to work on the build and warts like these can get fixed sooner.

Comment 4

[Gregory Szorc [:gps]]

Attachment: Overhaul mach's directory detection

Talked to a few people at the DOM work week last week about why they
don't use mach. Most everyone said that if they had a way to work from
the source directory while referencing any arbitrary objdir that they'd
be happy. vlad's original patch started down that road. This patch
builds on top of it.

There's still a lot left before this patch is ready, including tests. I
need to get some stuff in 29. Then, I hope to finish this up.

Comment 5

[[e-mail]]

(In reply to Vladimir Vukicevic from comment #0)
> One way that the current build system can work is, ignoring client.mk, you
> can do
> 
>   mkdir some-objdir
>   cd some-objdir
>   ../gecko/configure
>   make
> 
> to set up a relative srcdir and put the objdir anywhere with whatever name.

Note that as of bug 572529 a basic build works with client.mk too:

  mkdir some-objdir
  cd some-objdir
  make -f ../gecko/client.mk

You'd need to specify configure options via a custom MOZCONFIG though. (It used to be possible to use conditional programming in your ../gecko/.mozconfig but as of bug 762358 .mozconfig no longer knows where your objdir is.)