687515 - decursify build - make-makefile calls

Status: RESOLVED INVALID

(Firefox Build System :: General, defect)

Description

[Joey Armstrong [:joey]]

Setup a container makefile template for encapsulating make-makefile calls to process Makefile.in sources and generate Makefile files.

This chunk of build logic is pretty much self contained and can be used as a scaled down guineapig for deploying the decursify build logic bundled in ticket 623617.

Comment 1

[Joey Armstrong [:joey]]

Attachment: container makefile configured to process sandbox make-makefile commands

Some notes about container makefiles/makefilegen can be found here:
http://wiki.mozilla.org/User:JoeyArmstrong/makefiles/makefilegen

This patch contains logic from bug 623617 that has been configured to process a subset of container makefile tasks.  Files will be gathered and container makefiles generated as needed beneath the generated directory.


The makefile config/makefiles/container.mk will limit processing to the make-makefile command which reads Makefile.in templates to generate 'Makefile' files.

The patch in bug # 687511 (make-makefile) will be needed for an actual build.

Unit tests can be launched by either:
  gmake -C obj*/config/makefiles/test check
  -or-
  cd config/makefiles/test and running the *.t* scripts.

Comment 2

[[e-mail] (:bhearsum)]

Comment on attachment 565574 [details] [diff] [review]
container makefile configured to process sandbox make-makefile commands

Review of attachment 565574 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry Joey, I don't think I'm capable of reviewing this after all

Comment 3

[Chris Cooper [:coop] (he/him)]

Comment on attachment 565574 [details] [diff] [review]
container makefile configured to process sandbox make-makefile commands

Review of attachment 565574 [details] [diff] [review]:
-----------------------------------------------------------------

As we discussed in our 1x1, there may be issues WRT FindBin and pymake as we saw in bug 687511.

Some nits, but patch looks good otherwise.

::: client.mk
@@ +312,3 @@
>  configure-files: $(CONFIGURES)
>  
> +configure-doit:

Ah, the -doit suffix. :)

::: config/makefiles/container.mk
@@ +57,5 @@
> +
> +#CONTAINER_ARGS = -k
> +#CONTAINER_ARGS = -j1 --debug=v
> +#CONTAINER_ARGS = -j3
> +#CONTAINER_ARGS += -l8

Commented out for a reason?

@@ +69,5 @@
> +bootstrap:
> +	umask 022 && mkdir -p $(MOZ_OBJDIR)/dist/include
> +
> +TIME=time
> +#TIME += --portability

Coming later?

::: config/makefiles/makefilegen.excl
@@ +9,5 @@
> +config/makefiles/
> +
> +##########################################
> +## $(PWD)/$(dist_libdir) - pwd breaks path
> +## gcc  arena.o -L/mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/dist/lib -Xlinker -rpath /mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/nsprpub/lib/tests//mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/dist/lib -lplc4 -lplds4 -lnspr4 -lpthread -o arena

Are you meaning to exclude something here that you're not?

::: config/makefiles/makefilegen.pl
@@ +32,5 @@
> +##------------------##
> +##---] INCLUDES [---##
> +##------------------##
> +use FindBin;
> +use lib $FindBin::Bin;

Check for issues with pymake (see bug 687511).

::: config/makefiles/makefilegen.pm
@@ +37,5 @@
> +
> +##------------------##
> +##---] INCLUDES [---##
> +##------------------##
> +use FindBin;

Check for issues with pymake (see bug 687511).

::: config/makefiles/templates/makefile.makemake.tmpl
@@ +80,5 @@
> +	    --top $(TOPSRCDIR) \
> +	    --obj $(MOZ_OBJDIR) \
> +	    $<
> +
> +## HACK: DEPTH != $top while building js/src

Ugh.

::: config/makefiles/test/makefilegen.tpl
@@ +24,5 @@
> +
> +##------------------##
> +##---] INCLUDES [---##
> +##------------------##
> +use FindBin;

Check for issues with pymake (see bug 687511).

::: js/src/config/makefiles/container.mk
@@ +57,5 @@
> +
> +#CONTAINER_ARGS = -k
> +#CONTAINER_ARGS = -j1 --debug=v
> +#CONTAINER_ARGS = -j3
> +#CONTAINER_ARGS += -l8

Same as for config/makefiles/container.mk: choosing one here?

@@ +69,5 @@
> +bootstrap:
> +	umask 022 && mkdir -p $(MOZ_OBJDIR)/dist/include
> +
> +TIME=time
> +#TIME += --portability

Coming later?

::: js/src/config/makefiles/makefilegen.excl
@@ +9,5 @@
> +config/makefiles/
> +
> +##########################################
> +## $(PWD)/$(dist_libdir) - pwd breaks path
> +## gcc  arena.o -L/mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/dist/lib -Xlinker -rpath /mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/nsprpub/lib/tests//mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/dist/lib -lplc4 -lplds4 -lnspr4 -lpthread -o arena

Are you meaning to exclude something here that you're not?

::: js/src/config/makefiles/makefilegen.pl
@@ +32,5 @@
> +##------------------##
> +##---] INCLUDES [---##
> +##------------------##
> +use FindBin;
> +use lib $FindBin::Bin;

Check for issues with pymake (see bug 687511).

::: js/src/config/makefiles/makefilegen.pm
@@ +37,5 @@
> +
> +##------------------##
> +##---] INCLUDES [---##
> +##------------------##
> +use FindBin;

Check for issues with pymake (see bug 687511).

Comment 4

[Joey Armstrong [:joey]]

(In reply to Chris Cooper [:coop] from comment #3)
> Comment on attachment 565574 [details] [diff] [review] [diff] [details] [review]
> container makefile configured to process sandbox make-makefile commands
> 
> Review of attachment 565574 [details] [diff] [review] [diff] [details] [review]:
> -----------------------------------------------------------------
> 
> ::: config/makefiles/container.mk
> @@ +57,5 @@
> > +
> > +#CONTAINER_ARGS = -k
> > +#CONTAINER_ARGS = -j1 --debug=v
> > +#CONTAINER_ARGS = -j3
> > +#CONTAINER_ARGS += -l8
> 
> Commented out for a reason?

Just for the convenience of typing (~deleting) one character to enable.
Flags are present for build profiling.


> @@ +69,5 @@
> > +bootstrap:
> > +	umask 022 && mkdir -p $(MOZ_OBJDIR)/dist/include
> > +
> > +TIME=time
> > +#TIME += --portability
> 
> Coming later?

Oddly enough --portability is not portable to all platforms.
Yes a platform conditional will be added to include the flag.


> ::: config/makefiles/makefilegen.excl
> @@ +9,5 @@
> > +config/makefiles/
> > +
> > +##########################################
> > +## $(PWD)/$(dist_libdir) - pwd breaks path
> > +## gcc  arena.o -L/mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/dist/lib -Xlinker -rpath /mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/nsprpub/lib/tests//mozilla/sandbox/finalfrontier/obj-x86_64-unknown-linux-gnu/dist/lib -lplc4 -lplds4 -lnspr4 -lpthread -o arena
> 
> Are you meaning to exclude something here that you're not?

This was only a placeholder for a failure condition to check on.  Will have to check and see if this is still an issue.  Comment can be removed in the interim.


> ::: config/makefiles/makefilegen.pl
> @@ +32,5 @@
> > +##------------------##
> > +##---] INCLUDES [---##
> > +##------------------##
> > +use FindBin;
> > +use lib $FindBin::Bin;
> 
> Check for issues with pymake (see bug 687511).

pending

 
> ::: config/makefiles/makefilegen.pm
> @@ +37,5 @@
> > +
> > +##------------------##
> > +##---] INCLUDES [---##
> > +##------------------##
> > +use FindBin;
> 
> Check for issues with pymake (see bug 687511).
> 
> ::: config/makefiles/templates/makefile.makemake.tmpl
> @@ +80,5 @@
> > +	    --top $(TOPSRCDIR) \
> > +	    --obj $(MOZ_OBJDIR) \
> > +	    $<
> > +
> > +## HACK: DEPTH != $top while building js/src
> 
> Ugh.

Yep special case land, there are at least two that require a workaround.

Comment 5

[Gregory Szorc [:gps]]

make-makefile was removed in bug 774032.

Some of the code in the patch may still be relevant, I dunno. But, the premise of the bug is now definitely invalid.