Closed Bug 707569 Opened 8 years ago Closed 8 years ago

Speed up |make package|

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla15

People

(Reporter: mounir, Assigned: joduinn)

References

Details

Attachments

(1 file)

Originally |make package| was only used very rarely to create package for releases so it's speed wasn't a big deal but nowadays it is used to generate android package (.apk) which has to be called every time a developer wants to test a change on his device.
On my laptop doing so requires around 1 minute. This is ridiculously slow compared to the time it takes me to compile a change.

For what I've seen, the main reason of this slowness is that invoking |make package| deletes all outputs files and recreates them. I think we should keep them and only update them if needed.
If our current behavior has some good reasons to exists, we should at least create a fast-package rule that would basically set up a env variable and call |make package| which wouldn't delete everything.
Depends on: 707577
Depends on: 707578
Depends on: 707579
Depends on: 707580
With the patches in the bugs in the dependency list, |make fast-package|, the speed improvement is a bit higher than 50% (goes from around 1m10s to 25-30s).
Can we use |make package| instead of |make fast-package|?
The only risk I see is to have some old files you have deleted in your package but it's not really important for release build I guess since they're probably made from scratch.
(In reply to Vivien Nicolas (:vingtetun) from comment #2)
> Can we use |make package| instead of |make fast-package|?
> The only risk I see is to have some old files you have deleted in your
> package but it's not really important for release build I guess since
> they're probably made from scratch.

I'm all for that but I guess some people might not really like the idea so using a different target to begin with looked like a good alternative to me. If khuey says making all changes directly to |make package| is fine, I will do it gladly :)
When dealing with packaging, be careful to not break being able to run multiple repacks in a row.

I did check the patches, and they seem to only affect the installer-stage target and before, which is fine. The l10n stuff only touches INNER_MAKE_PACKAGE and INNER_UNMAKE_PACKAGE, and after.
Depends on: 718295
There are only really two things that make most of the time in make package: creating the js shell zip, which is pointless except on tinderbox builds, and xpt.py, which is very slow (bug 654448).
(In reply to Mounir Lamouri (:volkmar) (:mounir) from comment #3)
> (In reply to Vivien Nicolas (:vingtetun) from comment #2)
> > Can we use |make package| instead of |make fast-package|?
> > The only risk I see is to have some old files you have deleted in your
> > package but it's not really important for release build I guess since
> > they're probably made from scratch.
> 
> I'm all for that but I guess some people might not really like the idea so
> using a different target to begin with looked like a good alternative to me.
> If khuey says making all changes directly to |make package| is fine, I will
> do it gladly :)

We'd have problems on dep builds if we did that, fwiw. Especially when a component is moved filenames, and thus the old component name and the new get loaded, name conflicts, test failures, and broken builds will arise. This is not a problem on any clobber build as you said, but MOST devs and almost-all non-nightly builds are non-clobber as well, which is why I feel this is a bad idea...

We could however try and detect a clobber and *ONLY*THAT*BUILD* do make fast-package by default when make package is called... (but future make package calls will always do the slower package) ;-)
Arguably, there is nothing to optimize for when doing a clobber build...
Depends on: 718391
I filed bug 727960 outlining how we can test changes by passing make package.
per meeting w/ted, joey friday, I'll take this and see what I can do to help.
https://hg.mozilla.org/mozilla-central/rev/7d96ff6113c7
Assignee: nobody → joduinn
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla15
No longer depends on: 707579
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.