Closed Bug 1019058 Opened 6 years ago Closed 6 years ago

Incremental build for single app is slow

Categories

(Firefox OS Graveyard :: Gaia::Build, defect)

x86_64
Windows 7
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: ochameau, Assigned: ochameau)

Details

(Whiteboard: [perf-reviewed])

Attachments

(1 file, 1 obsolete file)

46 bytes, text/x-github-pull-request
yurenju
: review+
Details | Review
We don't have datazilla for the build system, but we may easily have regressed build system performances over time. The time it takes to build a simple app feels too long.

For example it takes 9s to build clock app, most of the time is passed in app-makefiles rule of the Makefile (7 to 8s).
It appears that almost half of that time is passed in some Makefile evaluation over here:
https://github.com/mozilla-b2g/gaia/blob/master/Makefile#L522
  $(eval $(call app-makefile-template,$(notdir $(appdir)),$(appdir))) \
Even if we use APP=clock, we eval/call app-makefile-template for all apps and for some reasone, even if we don't execute anything for all of them but clock, it takes several seconds to execute...

By tweaking that particular code, I shrinked down from 9s to 5s clock build time, with only 3s (out of 7/8s) for app-makefiles rules.
I think it would be worth spending some time profiling the build system and coming up with small and surgical patches like that.
Attached file Pull request 19882
Attachment #8432619 - Flags: review?(yurenju.mozilla)
Attached file Pull request 19886 (obsolete) —
Here is another patch, more complex, but this one comes with more than just performances benefit. It would help revive about:gaia addon by simplifying the Makefile and easing building gaia from just JS script.

Before and ffter executing build script/Makefile per app, we execute a bunch of js script from build/. Before: svoperapps, webapp-manifest, contact-import-services, keyboard-layouts, settings, webapp-shared. After: post-manifest, copy-build-stage-data, webapp-optimize, multilocale, webapp-zip. The key thing here is that there is some small overhead in executing xpcshell many times.

With the previous patch applied and this new one, for APP=clock make, It lower down build time from 4.3s to 3.0s (-30%).

I just asked for feedback as I haven't had time to test/verify this patch.

A natural followup of this patch would be to implement app-makefiles in JS and merge pre-app app-makefiles and post-app into a single one, but that would require getting rid of all app-specific Makefiles. The gain should be even more significant for complete build as we are running one xpcshell instance per app.
Attachment #8432660 - Flags: feedback?(yurenju.mozilla)
Comment on attachment 8432660 [details] [review]
Pull request 19886

I really like this idea for handling dependencies from Makefile to js and that can help about:gaia working again :-)

and that's right we need to get rid of all apps Makefile which is I'm working on bug 1017490.
Attachment #8432660 - Flags: feedback?(yurenju.mozilla) → feedback+
Comment on attachment 8432619 [details] [review]
Pull request 19882

looks good and low risk for landing in 2.0, r=yurenju

and let's file another follow up for pull request 19886.
Attachment #8432619 - Flags: review?(yurenju.mozilla) → review+
Comment on attachment 8432660 [details] [review]
Pull request 19886

I took some time to test it more and everything looks fine so far.
I just rebased it and resolved a conflict with addition of optimize-clean.
Attachment #8432660 - Flags: review?(yurenju.mozilla)
Whiteboard: [perf-reviewed]
Assignee: nobody → poirot.alex
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Attachment #8432660 - Attachment is obsolete: true
Attachment #8432660 - Flags: review?(yurenju.mozilla)
You need to log in before you can comment on or make changes to this bug.