1122851 - Don't attempt to run configure when not running ./mach build

Status: REOPENED

(Firefox Build System :: Mach Core, enhancement)

Description

[(no longer active)]

This breaks my workflow these days all the time.  As I switch between topic branches, I sometimes just want to go back to a branch and run some test there with the binary I have in my objdir.  As soon as I run ./mach run, mach updates the build backend, which leads to:

$ ./mach mochitest browser/components/sessionstore/test/browser_522545.js
Build configuration changed. Regenerating backend.
Reticulating splines...
Finished reading 2751 moz.build files in 2.39s
Processed into 7651 build config descriptors in 1.88s
Backend executed in 3.29s
2297 total backend files; 0 created; 0 updated; 2297 unchanged; 0 deleted; 141 -> 854 Makefile
Total wall time: 7.86s; CPU time: 6.41s; Efficiency: 82%; Untracked: 0.29s
STOP!  /Users/ehsan/moz/src/configure.in has changed, and your configure is out of date.
STOP!  /Users/ehsan/moz/src/js/src/configure.in has changed, and your configure is out of date.
Please rerun autoconf and re-configure your build directory.
Please rerun autoconf and re-configure your build directory.
To ignore this message, touch "/Users/ehsan/moz/src/configure",
To ignore this message, touch "/Users/ehsan/moz/src/js/src/configure",
but your build might not succeed.
but your build might not succeed.
make: *** [/Users/ehsan/moz/src/configure] Error 1
make: *** Waiting for unfinished jobs....
make: *** [/Users/ehsan/moz/src/js/src/configure] Error 1

At which point I do as told:

$ touch configure js/src/configure
$ ./mach mochitest browser/components/sessionstore/test/browser_522545.js
STOP!  /Users/ehsan/moz/src/configure has changed and needs to be run again.
STOP!  /Users/ehsan/moz/src/js/src/configure has changed and needs to be run again.
Please rerun it.
Please rerun it.
To ignore this message, touch "/Users/ehsan/moz/src/obj-ff-clang-plugin.noindex/config.status",
To ignore this message, touch "/Users/ehsan/moz/src/obj-ff-clang-plugin.noindex/js/src/config.status",
but your build might not succeed.
but your build might not succeed.
make: *** [config.status] Error 1
make: *** Waiting for unfinished jobs....
make: *** [js/src/config.status] Error 1

Then touching the config.status files allows me to run the test.

Can we not attempt to do anything with configure as part of mach mochitest (and really anything other than mach build)?

Comment 1

[(no longer active)]

(Oh and I forgot to say that the same thing happens when CLOBBER is touched, adding another round to the above two steps.)

Comment 2

[Gregory Szorc [:gps]]

It's not the configure check that's triggering: it's the moz.build check. We detect that a moz.build file (or a file it includes) has changed and we go scan the world. This in turn checks of configure is up to date.

Unfortunately, we can't make "don't scan" the default because if it were, you could update a test manifest (e.g. xpcshell.ini) and those changes wouldn't get picked up when you run a testing command. That would result in tons of confusion, especially to newcomers.

Fortunately, there is a gross workaround. Define MOZBUILD_BACKEND_CHECKED=1 in your environment and these checks should be disabled. Use at your own risk. Its use isn't supported. But it does what you want.

Comment 3

[(no longer active)]

Suppressing these checks was not what I was asking for.  I was asking for only processing the moz.build changes (that are required to for example pick up new tests as you noted) and not attempting to see if the rest of the build config (things such as the configure and config.status scripts) have changed.

Also, I'm not sure if the MOZBUILD_BACKEND_CHECKED=1 workaround works due to <https://dxr.mozilla.org/mozilla-central/source/config/rules.mk#522>.  (And even if I was not running mach, that would cause the changes to test manifests and whatnot to not be picked up, right?)

Comment 4

[Gregory Szorc [:gps]]

moz.build processing depends on the state of configure (actually it's output - config.status).

You are asking us to undermine real build dependencies.

Switching between topic branches currently and will forever cause a lot of build system invalidation due to mtimes and content changing. I feel your pain. I really do. But adding another footgun scares me.

Can you not maintain separate working copies as a workaround?

Comment 5

[(no longer active)]

(In reply to Gregory Szorc [:gps] from comment #4)
> moz.build processing depends on the state of configure (actually it's output
> - config.status).
> 
> You are asking us to undermine real build dependencies.

No.  Again, here is what I'm asking:

When running tests through mach (and *only* when running tests, do not do this for any other command), assume that if config.status (and configure) is out of date, or in CLOBBER is touched, we can ignore it, since to the best of my knowledge the only thing that we actually need to do before running tests is process the test manifests, and that can be done even with a stale config.status (I'm assuming that is the case... is there a practical case where that would break down?)

> Switching between topic branches currently and will forever cause a lot of
> build system invalidation due to mtimes and content changing. I feel your
> pain. I really do. But adding another footgun scares me.
> 
> Can you not maintain separate working copies as a workaround?

I have tried to switch to that model several times.  I find it too hard to track the working directories and over time I tend to checking out topic branches in the same tree again...  :/  Perhaps I'll switch again if this can't be fixed.