752873 - mobile/android/base/Makefile.in: $(shell cat) deps

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Joey Armstrong [:joey]]

mobile/android/base/Makefile.in
===============================

Data loading 'shell cat' commands in this makefile can be optimized.  Another patch has wrapped these calls in a conditional so they will only impose overhead once when a lib* target is being processed.

Optimizations:

  o If the *.mn files are only used for makefile dependencies contents can be re-ordered to one file per line removing the need for calling the 'tr' command.

  o Replace shell overhead with a makefile include.  Add make targets to derive *.mk from *.mn data, possibly defining SYNC variables as needed.  Add deps to regenerate individual makefiles when *.mn changes.  A simple include of the data makefile should be enough to force regeneration.


SYNC_JAVA_FILES=$(shell cat $(topsrcdir)/mobile/android/sync/java-sources.mn | tr '\n' ' ';)
SYNC_PP_JAVA_FILES=$(shell cat $(topsrcdir)/mobile/android/sync/preprocess-sources.mn | tr '\n' ' ';)
SYNC_THIRDPARTY_JAVA_FILES=$(shell cat $(topsrcdir)/mobile/android/sync/java-third-party-sources.mn | tr '\n' ' ';)
SYNC_RES_DRAWABLE=$(shell cat $(topsrcdir)/mobile/android/sync/android-drawable-resources.mn | tr '\n' ' ';)
SYNC_RES_DRAWABLE_LDPI=$(shell cat $(topsrcdir)/mobile/android/sync/android-drawable-ldpi-resources.mn | tr '\n' ' ';)
SYNC_RES_DRAWABLE_MDPI=$(shell cat $(topsrcdir)/mobile/android/sync/android-drawable-mdpi-resources.mn | tr '\n' ' ';)
SYNC_RES_DRAWABLE_HDPI=$(shell cat $(topsrcdir)/mobile/android/sync/android-drawable-hdpi-resources.mn | tr '\n' ' ';)
SYNC_RES_LAYOUT=$(shell cat $(topsrcdir)/mobile/android/sync/android-layout-resources.mn | tr '\n' ' ';)
SYNC_RES_VALUES=$(shell cat $(topsrcdir)/mobile/android/sync/android-values-resources.mn | tr '\n' ' ';)

Comment 1

[(not currently active) Ted Mielczarek]

I'm not really sure if these files are used for anything else, CCing rnewman who is apparently the author of the script that creates them:
https://github.com/mozilla-services/android-sync/blob/develop/fennec-copy-code.sh#L45

If they only exist for the sake of the Fennec build, then we should be able to just generate Makefiles instead. Perhaps something like:
echo "SYNC_JAVA_SOURCES := $SOURCEFILES" > $SYNC/java-sources.mk

Then we could include that directly.

Comment 2

[Richard Newman [:rnewman]]

We actually want to switch to using the file input to javac, rather than filenames-as-arguments, so we will still want some kind of manifest file for some of these. (That's Bug 736463.)

At that point the source files will be file-per-line.

I'd be very happy to take patches for these post-release; Fennec's and Sync's build system are a little bit rushed, to say the least.

I'm not wedded to a particular approach; I just suck at Make, and needed a solution quickly that wouldn't involve editing Fennec's Makefile every time we add a Sync source file.

These manifests are only used for Fennec's build.

Comment 3

[John Ford [:jhford] CET/CEST Berlin Time]

I've created a pull requeset https://github.com/mozilla-services/android-sync/pull/185, which creates the manifests with one file per line

Comment 4

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: remove the tr call

This patch corresponds to my pull request in github.  This removes the call to tr from the mozilla build system.  I'll put together a patch that generates a makefile as well as the manifests.

Comment 5

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Comment on attachment 622419 [details] [diff] [review]
remove the tr call

Review of attachment 622419 [details] [diff] [review]:
-----------------------------------------------------------------

If everything still works, r=me.

Comment 6

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: alternate approach where the android-sync build system generates the makfile and we include it

This doesn't have the sample makefile generated, but I'll include that.

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: sample makefile generated by android-sync changes

This is a sample makefile generated by my android-sync changes that I'm about to add to my pull request on github

Comment 8

[John Ford [:jhford] CET/CEST Berlin Time]

Comment on attachment 622424 [details] [diff] [review]
alternate approach where the android-sync build system generates the makfile and we include it

this works, https://tbpl.mozilla.org/?tree=Try&rev=d70761ec02ed and http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-d70761ec02ed/try-android/try-android-bm14-try1-build1657.txt.gz

This does depend on https://github.com/mozilla-services/android-sync/pull/185 to work.  I did my own sync code drop for the try run.

Comment 9

[Richard Newman [:rnewman]]

Merged with fixes:

https://github.com/mozilla-services/android-sync/commit/35d76d9946a258ca40cd2b46aad64938f3dd2011

Landed on inbound:

https://hg.mozilla.org/integration/mozilla-inbound/rev/3a783d744843
https://hg.mozilla.org/integration/mozilla-inbound/rev/b13f64598c37

Comment 10

[John Ford [:jhford] CET/CEST Berlin Time]

And on mozilla-central:

http://hg.mozilla.org/mozilla-central/rev/c7bb680df93b
http://hg.mozilla.org/mozilla-central/rev/654ac86492e8

Comment 11

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/3a783d744843
https://hg.mozilla.org/mozilla-central/rev/b13f64598c37