Closed Bug 1207893 Opened 4 years ago Closed 4 years ago

Allow to create multiple build backends at once

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(firefox44 fixed)

RESOLVED FIXED
mozilla44
Tracking Status
firefox44 --- fixed

People

(Reporter: glandium, Assigned: glandium)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files, 1 obsolete file)

The longest part of running `mach build-backend -b alternative_backend` is reading moz.build and emitting the corresponding data. Which is suboptimal, since it's already been done when running configure or `mach build-backend`.

This bug is about allowing mach build-backend to build multiple backends in one pass.
Blocks: 1207897
Depends on: 1207882
- Make all backends report the time spent in their own execution
- Change how the data is collected for the reader and emitter such that
  each of them is aware of its own data, instead of everything being
  tracked by the backend.

This is meant to open the door to multiple backends running from the
same execution of config.status.
Attachment #8665651 - Flags: review?(gps)
Currently, we set a flag on each object to know whether it has been consumed
by the backend. This doesn't work nicely when multiple backends try to consume
the same objects.
Attachment #8665652 - Flags: review?(gps)
When running mach `build-backend` or `config.status`, it is now possible to
pass multiple backends to the --backend/-b option, so that they can share
moz.build reading and object emitting.

The command line syntax is however maybe a little awkward:
  mach build-backend -b Backend1 Backend2

but supporting with `-b Backend1 -b Backend2` requires more argument parser
twiddling (action='append' doesn't work out of the box with choices, we'd
need a custom action class)

Sample output:
$ ./mach build-backend -b RecursiveMake FasterMake CppEclipse
Reticulating splines...
Finished reading 2540 moz.build files in 2.05s
Processed into 9347 build config descriptors in 2.13s
RecursiveMake backend executed in 2.41s
  2526 total backend files; 0 created; 1 updated; 2525 unchanged; 0 deleted; 125 -> 949 Makefile
FasterMake backend executed in 0.17s
  4 total backend files; 0 created; 0 updated; 4 unchanged; 0 deleted
CppEclipse backend executed in 0.05s
Generated Cpp Eclipse workspace in "/home/glandium/eclipse_obj-x86_64-unknown-linux-gnu".
If missing, import the project using File > Import > General > Existing Project into workspace

Run with: eclipse -data /home/glandium/eclipse_obj-x86_64-unknown-linux-gnu

Total wall time: 7.04s; CPU time: 6.63s; Efficiency: 94%; Untracked: 0.23s
Attachment #8665654 - Flags: review?(gps)
With a small change in config_status.py to still display VS and Android backend advertisement.
Attachment #8665654 - Attachment is obsolete: true
Attachment #8665654 - Flags: review?(gps)
Attachment #8665699 - Flags: review?(gps)
Blocks: 1208380
Comment on attachment 8665651 [details] [diff] [review]
Refactor how build backend execution is summarized

Review of attachment 8665651 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/backend/base.py
@@ -28,4 @@
>  from .configenvironment import ConfigEnvironment
> -
> -
> -class BackendConsumeSummary(object):

IIRC one of the reasons I originally made this as a separate class was to prevent variable proliferation on a shared base class. But I suppose the new names are specific enough that it won't be a problem.
Attachment #8665651 - Flags: review?(gps) → review+
Attachment #8665652 - Flags: review?(gps) → review+
Comment on attachment 8665699 [details] [diff] [review]
Allow to create multiple build backends at once

Review of attachment 8665699 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/config_status.py
@@ +12,5 @@
>  import os
>  import sys
>  import time
>  
> +from argparse import ArgumentParser

Thank you for killing optparse.
Attachment #8665699 - Flags: review?(gps) → review+
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.