872062 - moz.build: a few remaining SUBMAKEFILE assignments to convert

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Joey Armstrong [:joey]]

./mobile/locales/Makefile.in:SUBMAKEFILES += \
./mobile/android/locales/Makefile.in:SUBMAKEFILES += \
./media/webrtc/Makefile.in:NO_SUBMAKEFILES_RULE = 1
./media/mtransport/third_party/Makefile.in:NO_SUBMAKEFILES_RULE = 1
./b2g/locales/Makefile.in:SUBMAKEFILES += \
./b2g/locales/Makefile.in:repackage-win32-installer: $(call ESCAPE_SPACE,$(WIN32_INSTALLER_IN)) $(SUBMAKEFILES) libs-$(AB_CD)
./browser/locales/Makefile.in:SUBMAKEFILES += \
./browser/locales/Makefile.in:repackage-win32-installer: $(call ESCAPE_WILDCARD,$(WIN32_INSTALLER_IN)) $(SUBMAKEFILES) libs-$(AB_CD)

Comment 1

[Joey Armstrong [:joey]]

Attachment: move SUBMAKEFILES to mozbuild (logic).

Comment 2

[Joey Armstrong [:joey]]

Comment on attachment 753901 [details] [diff] [review]
move SUBMAKEFILES to mozbuild (logic).

Add SUBMAKEFILES as a passthrough variable.

Comment 3

[Joey Armstrong [:joey]]

Comment on attachment 753901 [details] [diff] [review]
move SUBMAKEFILES to mozbuild (logic).

Redirect to b::c while greg is away

Comment 4

[Joey Armstrong [:joey]]

Comment on attachment 753901 [details] [diff] [review]
move SUBMAKEFILES to mozbuild (logic).

Mike may be on his way to Japan so redirect the review again.

Comment 5

[Joey Armstrong [:joey]]

Comment on attachment 753901 [details] [diff] [review]
move SUBMAKEFILES to mozbuild (logic).

ping on the code review

Comment 6

[(not currently active) Ted Mielczarek]

Comment on attachment 753901 [details] [diff] [review]
move SUBMAKEFILES to mozbuild (logic).

Review of attachment 753901 [details] [diff] [review]:
-----------------------------------------------------------------

This feels weird to me, since we're explicitly tying ourselves to the Makefile backend here. I notice that this construct only occurs in l10n-related Makefiles (looks like only four Makefiles total), and those Makefiles have a mess of custom rules, so maybe we need to think about a bigger picture here instead of a passthrough conversion?

Comment 7

[Gregory Szorc [:gps]]

I agree with Ted that we should look at the bigger picture here. Let's revisit this down the road.

Comment 8

[Joey Armstrong [:joey]]

(In reply to Gregory Szorc [:gps] from comment #7)
> I agree with Ted that we should look at the bigger picture here. Let's
> revisit this down the road.

If we revisit this "down the road" the conversions and Makfile.in deprecation will continue to drag on and on and on and on... Adding passthrough variables in no way alters existing logic and will be one step closer to Makefile.in removal.  This only migrates value storage from Makefile.in to moz.build.

When the time comes a python routine can be written to handle variable logic, just snap that in.  Values are already in moz.build, if 'SUBMAKEFILES' does not make sense in a mozbuild context { once backend.mk is removed } just deprecate the "passthrough variable" or rename/repurpose it for alternate subroutine use.

Comment 9

[(not currently active) Ted Mielczarek]

I think we agree in the general sense--for most variables we should just do the passthrough conversion, get them into moz.build and worry about making the data model nicer at a later date. In the specific sense though there are a few variables like this that just don't make sense to pull into the moz.build world. Either they're specific to just a few Makefiles or they're very tied to custom rules in Makefile.in files, making a moz.build passthrough translation nonsensical unless we deal with the larger issues.

Instead of just porting SUBMAKEFILES to cover these few Makefiles, we should be asking "how can we fix these Makefiles to work in the moz.build world without all their custom rules that require setting SUBMAKEFILES to work properly?"

Comment 10

[Joey Armstrong [:joey]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #9)
> I think we agree in the general sense--for most variables we should just do
> the passthrough conversion, get them into moz.build and worry about making
> the data model nicer at a later date. In the specific sense though there are
> a few variables like this that just don't make sense to pull into the
> moz.build world. Either they're specific to just a few Makefiles or they're
> very tied to custom rules in Makefile.in files, making a moz.build
> passthrough translation nonsensical unless we deal with the larger issues.

The task at hand is was to finish porting Makefile.in content to mozbuild so we can deprecate usage.  While Makefile.in files persist this project will not have a sunset date.

If variable names are nonsensical within the context of mozbuild fine, we could just rename the variable for temporary usage until the variable can be deprecated.

> Instead of just porting SUBMAKEFILES to cover these few Makefiles, we should
> be asking "how can we fix these Makefiles to work in the moz.build world
> without all their custom rules that require setting SUBMAKEFILES to work
> properly?"

Yes do I agree the underlying logic needs work and will need to be implemented at some point.  But I do not think converting/deprecating Makefile.in needs to block on providing python subroutines for all the logic.

Pass all variables through so Makefile.in can be removed.  Then at some point in the future anyone can go in, rip out the passthrough logic and replace that and rules.mk with python logic.

I see all of this as step #2 for the conversion.
Step #1 is migrating all Makefile.in content into moz.build so files can be removed.

Comment 11

[(not currently active) Ted Mielczarek]

I don't think this buys you anything. You can't get rid of these Makefile.in files without converting the rules and logic they contain. In the process of doing that you'll make the SUBMAKEFILES assignments unnecessary. Why do the extra work of porting these over only to obsolete them before the project is complete?

I agree we should be striving to get things converted as soon as possible. I don't agree that we should blindly convert things just because they make for a net move of lines from Makefile.in to moz.build.

Comment 12

[Joey Armstrong [:joey]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #11)
> I don't think this buys you anything. You can't get rid of these Makefile.in
> files without converting the rules and logic they contain. In the process of
> doing that you'll make the SUBMAKEFILES assignments unnecessary. Why do the
> extra work of porting these over only to obsolete them before the project is
> complete?

Because I see this as another step closer to deprecating Makefile.in.  We are already a few quarters into the transition and only a handful of variables having been converted so far.  At the current rate this effort will be going on for a very looooong time.

If instead we start bulk porting everything possible as verbatim/passthrough variables.  Even if that activity is incomplete, temporary or would be a NOP given no change in existing functionality.  There are gains to be expected from the activity.

First it allows most people to get out of the business of maintaing makefile content.  The are able to exclusively maintain content in python, pickup sorted file lists, syntax checking and any other protections provided.

Next even if some rules linger chances are very good that the logic has been duplicated across several makefiles.  We could implement mozbuild/python routines + testing to mimic or extend the behavior { yes, which will need to happen at some point }.  But a faster and safer op would be to first refactor/copy &paste the logic verbatim into an included makefile.  That allows purging more content from Makefile.in right now.

> I agree we should be striving to get things converted as soon as possible. I
> don't agree that we should blindly convert things just because they make for
> a net move of lines from Makefile.in to moz.build.

I think we should be.  As more content migrates to python that will open the effort and mozbuild up to more people.  As passthrough variables are added more people than just mshal and I can work on the data conversions.  With variables converted over to mozbuild people also have a cleaner entry point to begin working on adding variable specific python routines. This helps thread the entire process so the mozbuild conversion can finally be put to rest.

Comment 13

[Joey Armstrong [:joey]]

Closing per comment #6, only 6 references to SUBMAKEFILES remain in Makefile.in