905879 - Make build status/tier tracking more robust

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Gregory Szorc [:gps]]

The code for tracking build status (aka tier traversal) was pieced together quickly with the goal of landing something that worked. In bug 883209, I'm putting together patches for incorporating resource monitoring into the build system. A goal of that patch is to correlate resource usage with events during the build, including tier traversal.

Unfortunately, the current logic for tracking tiers is not up to snuff. It only tracks the currently running tiers. It doesn't hold historical information. It doesn't support concurrently executing directories. There's lots of assertions for state tracking.

In this bug, we'll clean the code up a bit and make the resource monitoring patch easier to grok.

Comment 1

[Gregory Szorc [:gps]]

Attachment: More robust tier tracking

Per request in bug 883209, I've split the tier monitor refactoring into
a separate patch.

Among the refactoring of the BUILDSTATUS emitting and tracking code, I
also snuck a change into the precompile tier to rejiggle dependencies a
little. It saves a few extra $(MAKE) calls and makes it so that tier is
tracked properly (since it doesn't have the usual subtiers).

Comment 2

[Mike Hommey [:glandium]]

Comment on attachment 791051 [details] [diff] [review]
More robust tier tracking

Review of attachment 791051 [details] [diff] [review]:
-----------------------------------------------------------------

::: config/makefiles/precompile/Makefile.in
@@ +25,5 @@
>  endef
>  
> +default::
> +	$(call BUILDSTATUS,TIER_START  precompile IPDL WebIDL XPIDL)
> +	+$(MAKE) export

might as well do $(MAKE) ipdl webidl xpidl, and get rid of the export rule. (and if you're worried about -jN, yes, that works properly with it)

::: config/rules.mk
@@ +731,5 @@
>  tier_$(1)::
> +ifdef TIER_$(1)_CUSTOM
> +	$$(foreach dir,$$($$@_dirs),$$(call SUBMAKE,,$$(dir)))
> +else
> +	$(call BUILDSTATUS_P,BUILDSTATUS TIER_START $(1))

Relying on printf is actually fragile when doing parallel builds. Just replace SUBTIERS with TIER_START $(1) in the one-liner you're replacing.

Comment 3

[Gregory Szorc [:gps]]

(In reply to Mike Hommey [:glandium] from comment #2)
> ::: config/makefiles/precompile/Makefile.in
> @@ +25,5 @@
> >  endef
> >  
> > +default::
> > +	$(call BUILDSTATUS,TIER_START  precompile IPDL WebIDL XPIDL)
> > +	+$(MAKE) export
> 
> might as well do $(MAKE) ipdl webidl xpidl, and get rid of the export rule.
> (and if you're worried about -jN, yes, that works properly with it)

Until we refactor how tiers work, the export target is here to preserve |make export| behavior.

Comment 4

[Gregory Szorc [:gps]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/0e046ece4b23

Comment 5

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/0e046ece4b23