Speed up |make package|

RESOLVED FIXED in mozilla15

Status

()

Core
Build Config
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: mounir, Assigned: joduinn)

Tracking

Trunk
mozilla15
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

6 years ago
Originally |make package| was only used very rarely to create package for releases so it's speed wasn't a big deal but nowadays it is used to generate android package (.apk) which has to be called every time a developer wants to test a change on his device.
On my laptop doing so requires around 1 minute. This is ridiculously slow compared to the time it takes me to compile a change.

For what I've seen, the main reason of this slowness is that invoking |make package| deletes all outputs files and recreates them. I think we should keep them and only update them if needed.
If our current behavior has some good reasons to exists, we should at least create a fast-package rule that would basically set up a env variable and call |make package| which wouldn't delete everything.
(Reporter)

Updated

6 years ago
Depends on: 707577
(Reporter)

Updated

6 years ago
Depends on: 707578
(Reporter)

Updated

6 years ago
Depends on: 707579
(Reporter)

Updated

6 years ago
Depends on: 707580
(Reporter)

Comment 1

6 years ago
With the patches in the bugs in the dependency list, |make fast-package|, the speed improvement is a bit higher than 50% (goes from around 1m10s to 25-30s).
Can we use |make package| instead of |make fast-package|?
The only risk I see is to have some old files you have deleted in your package but it's not really important for release build I guess since they're probably made from scratch.
(Reporter)

Comment 3

6 years ago
(In reply to Vivien Nicolas (:vingtetun) from comment #2)
> Can we use |make package| instead of |make fast-package|?
> The only risk I see is to have some old files you have deleted in your
> package but it's not really important for release build I guess since
> they're probably made from scratch.

I'm all for that but I guess some people might not really like the idea so using a different target to begin with looked like a good alternative to me. If khuey says making all changes directly to |make package| is fine, I will do it gladly :)

Comment 4

6 years ago
When dealing with packaging, be careful to not break being able to run multiple repacks in a row.

I did check the patches, and they seem to only affect the installer-stage target and before, which is fine. The l10n stuff only touches INNER_MAKE_PACKAGE and INNER_UNMAKE_PACKAGE, and after.
(Reporter)

Updated

6 years ago
Depends on: 718295
There are only really two things that make most of the time in make package: creating the js shell zip, which is pointless except on tinderbox builds, and xpt.py, which is very slow (bug 654448).
(In reply to Mounir Lamouri (:volkmar) (:mounir) from comment #3)
> (In reply to Vivien Nicolas (:vingtetun) from comment #2)
> > Can we use |make package| instead of |make fast-package|?
> > The only risk I see is to have some old files you have deleted in your
> > package but it's not really important for release build I guess since
> > they're probably made from scratch.
> 
> I'm all for that but I guess some people might not really like the idea so
> using a different target to begin with looked like a good alternative to me.
> If khuey says making all changes directly to |make package| is fine, I will
> do it gladly :)

We'd have problems on dep builds if we did that, fwiw. Especially when a component is moved filenames, and thus the old component name and the new get loaded, name conflicts, test failures, and broken builds will arise. This is not a problem on any clobber build as you said, but MOST devs and almost-all non-nightly builds are non-clobber as well, which is why I feel this is a bad idea...

We could however try and detect a clobber and *ONLY*THAT*BUILD* do make fast-package by default when make package is called... (but future make package calls will always do the slower package) ;-)
Arguably, there is nothing to optimize for when doing a clobber build...
(Reporter)

Updated

6 years ago
Depends on: 718391
I filed bug 727960 outlining how we can test changes by passing make package.
Blocks: 748452
per meeting w/ted, joey friday, I'll take this and see what I can do to help.
Depends on: 654448
Created attachment 621801 [details] [diff] [review]
have "build_and_deploy" target use new "fast-package"
Attachment #621801 - Flags: review?(khuey)
Attachment #621801 - Attachment is patch: true
Attachment #621801 - Flags: review?(khuey) → review+
Comment on attachment 621801 [details] [diff] [review]
have "build_and_deploy" target use new "fast-package"

http://hg.mozilla.org/integration/mozilla-inbound/rev/7d96ff6113c7
https://hg.mozilla.org/mozilla-central/rev/7d96ff6113c7
Assignee: nobody → joduinn
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla15
No longer depends on: 707579
You need to log in before you can comment on or make changes to this bug.