1386497 - Add a moz.configure post-configure, per-build primitive

Status: NEW

(Firefox Build System :: General, enhancement)

Description

[Gregory Szorc [:gps]]

I've been hacking a bit on moz.configure checks a bit lately. I like the ability to define dependencies and execution tends to sort itself out.

One thing led to another and I had the (possibly crazy) idea that we should leverage moz.configure to perform various per-build tasks that quack like configure tasks. These include:

* Determining the version control changeset
* Setting the build id

We could even shoehorn some other tasks in there, such as:

* Resetting ccache/sccache statistics
* Starting the sccache daemon
* Querying watchman for files changed since the last build and doing something with that data
* Looking for local VCS commits and automatically opting in to artifact builds if there are no binary-impacting changes since the last available binary artifacts from CI

In my mind, I was thinking that this "build configure" would resolve build-time state (as opposed to one-time "configure" state). Its output would be a set of supplemental config and defines that would be overlayed on top of config.status.

Two quick observations:

1) One could leverage moz.configure's DAG as a build system of sorts and we could use moz.configure to invoke a build backend. This would allow us to perform sequential build steps from configure. It would also allow us to perform post-build actions. I don't think I'm keen on doing this. But it's interesting to consider for limited use cases, such as replacing client.mk.

2) If the execution time of configure is ~0.0s when nothing has changed, then there doesn't need to be a build-time configure: we can just invoke configure again every build and it will assemble build-time state as needed.

Anyway, I thought I'd file a bug and throw the idea out there. I would like to move stuff in variables.py into something that looks a lot more like configure. I just don't know what that is and this idea sprung to mind.

Comment 1

[Mike Hommey [:glandium]]

FWIW, 2) was my idea long term

Comment 2

[Gregory Szorc [:gps]]

What's preventing us from invoking moz.configure on every build invocation? (Read: how much work is it to just do the right thing?)

Comment 3

[Mike Hommey [:glandium]]

We need to figure out the right level of caching. Plus, as long as old-configure exists, that still needs to run.

Comment 4

[Gregory Szorc [:gps]]

For a per-build configure, I was thinking it would be sufficient to exclude old-configure from that invocation. So that leaves caching. But I have no clue how to even approach that problem.