Last Comment Bug 707569 - Speed up |make package|
: Speed up |make package|
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: Build Config (show other bugs)
: Trunk
: All All
: -- normal (vote)
: mozilla15
Assigned To: John O'Duinn [:joduinn] (please use "needinfo?" flag)
:
Mentors:
Depends on: 654448 707577 707578 707580 718295 718391
Blocks: 748452
  Show dependency treegraph
 
Reported: 2011-12-04 14:17 PST by Mounir Lamouri (:mounir)
Modified: 2012-05-11 10:05 PDT (History)
14 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
have "build_and_deploy" target use new "fast-package" (1.48 KB, patch)
2012-05-07 17:30 PDT, John O'Duinn [:joduinn] (please use "needinfo?" flag)
khuey: review+
Details | Diff | Splinter Review

Description Mounir Lamouri (:mounir) 2011-12-04 14:17:46 PST
Originally |make package| was only used very rarely to create package for releases so it's speed wasn't a big deal but nowadays it is used to generate android package (.apk) which has to be called every time a developer wants to test a change on his device.
On my laptop doing so requires around 1 minute. This is ridiculously slow compared to the time it takes me to compile a change.

For what I've seen, the main reason of this slowness is that invoking |make package| deletes all outputs files and recreates them. I think we should keep them and only update them if needed.
If our current behavior has some good reasons to exists, we should at least create a fast-package rule that would basically set up a env variable and call |make package| which wouldn't delete everything.
Comment 1 Mounir Lamouri (:mounir) 2011-12-04 15:45:11 PST
With the patches in the bugs in the dependency list, |make fast-package|, the speed improvement is a bit higher than 50% (goes from around 1m10s to 25-30s).
Comment 2 Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please) 2011-12-04 22:53:17 PST
Can we use |make package| instead of |make fast-package|?
The only risk I see is to have some old files you have deleted in your package but it's not really important for release build I guess since they're probably made from scratch.
Comment 3 Mounir Lamouri (:mounir) 2011-12-04 23:05:56 PST
(In reply to Vivien Nicolas (:vingtetun) from comment #2)
> Can we use |make package| instead of |make fast-package|?
> The only risk I see is to have some old files you have deleted in your
> package but it's not really important for release build I guess since
> they're probably made from scratch.

I'm all for that but I guess some people might not really like the idea so using a different target to begin with looked like a good alternative to me. If khuey says making all changes directly to |make package| is fine, I will do it gladly :)
Comment 4 Axel Hecht [:Pike] 2011-12-05 00:32:05 PST
When dealing with packaging, be careful to not break being able to run multiple repacks in a row.

I did check the patches, and they seem to only affect the installer-stage target and before, which is fine. The l10n stuff only touches INNER_MAKE_PACKAGE and INNER_UNMAKE_PACKAGE, and after.
Comment 5 Mike Hommey [:glandium] 2012-01-15 23:11:32 PST
There are only really two things that make most of the time in make package: creating the js shell zip, which is pointless except on tinderbox builds, and xpt.py, which is very slow (bug 654448).
Comment 6 Justin Wood (:Callek) 2012-01-15 23:54:20 PST
(In reply to Mounir Lamouri (:volkmar) (:mounir) from comment #3)
> (In reply to Vivien Nicolas (:vingtetun) from comment #2)
> > Can we use |make package| instead of |make fast-package|?
> > The only risk I see is to have some old files you have deleted in your
> > package but it's not really important for release build I guess since
> > they're probably made from scratch.
> 
> I'm all for that but I guess some people might not really like the idea so
> using a different target to begin with looked like a good alternative to me.
> If khuey says making all changes directly to |make package| is fine, I will
> do it gladly :)

We'd have problems on dep builds if we did that, fwiw. Especially when a component is moved filenames, and thus the old component name and the new get loaded, name conflicts, test failures, and broken builds will arise. This is not a problem on any clobber build as you said, but MOST devs and almost-all non-nightly builds are non-clobber as well, which is why I feel this is a bad idea...

We could however try and detect a clobber and *ONLY*THAT*BUILD* do make fast-package by default when make package is called... (but future make package calls will always do the slower package) ;-)
Comment 7 Mike Hommey [:glandium] 2012-01-16 00:03:08 PST
Arguably, there is nothing to optimize for when doing a clobber build...
Comment 8 Benoit Girard (:BenWa) 2012-02-17 10:18:27 PST
I filed bug 727960 outlining how we can test changes by passing make package.
Comment 9 John O'Duinn [:joduinn] (please use "needinfo?" flag) 2012-04-29 19:27:07 PDT
per meeting w/ted, joey friday, I'll take this and see what I can do to help.
Comment 10 John O'Duinn [:joduinn] (please use "needinfo?" flag) 2012-05-07 17:30:06 PDT
Created attachment 621801 [details] [diff] [review]
have "build_and_deploy" target use new "fast-package"
Comment 11 John Ford [:jhford] 2012-05-07 17:51:05 PDT
Comment on attachment 621801 [details] [diff] [review]
have "build_and_deploy" target use new "fast-package"

http://hg.mozilla.org/integration/mozilla-inbound/rev/7d96ff6113c7
Comment 12 Ed Morley [:emorley] 2012-05-08 03:24:21 PDT
https://hg.mozilla.org/mozilla-central/rev/7d96ff6113c7

Note You need to log in before you can comment on or make changes to this bug.