Closed Bug 987267 Opened 7 years ago Closed 7 years ago
use gzip compression for hg bundles
hg operations are gonna take as long or longer than builds once we implement caching on windows. According to glandium's stats, hg unbundling only happens on ~3% of the builds, but it does lead to some surprisingly-slow build times. Simply zipping up the whole repo ends in a 3x speed up, switching to gzip bundles is a 1.4x speedup. Lets swap to gzip bundles asap(or just hg clone --uncompressed on windows which would be fastest) ++ mkdir mozilla-central ++ cd mozilla-central ++ hg init . ++ hg unbundle ../mozilla-central.hg adding changesets adding manifests adding file changes added 174831 changesets with 983111 changes to 153611 files (run 'hg update' to get a working copy) real 31m59.826s user 0m0.015s sys 0m0.015s ++ cd mozilla-central ++ hg init . ++ hg unbundle ../mozilla-central.gz.bundle adding changesets adding manifests adding file changes added 174831 changesets with 983111 changes to 153611 files (run 'hg update' to get a working copy) real 22m59.116s user 0m0.015s sys 0m0.030s ++ rm -fR mozilla-central ++ unzip -q mozilla-central.zip real 11m19.762s user 0m0.015s sys 0m0.000s 1.1G mozilla-central.zip 914M mozilla-central.7z 790M mozilla-central.gz.bundle 631M mozilla-central.hg
Switching to gzip is easy. We'll need to regenerate the bundles (see https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Create_new_hg_bundles) zipping .hg would require changes in hg-tool.py
err, wrong position
7 years ago
Attachment #8395828 - Flags: review?(catlee) → review+
Comment on attachment 8395828 [details] [diff] [review] gzip.bundle.diff https://hg.mozilla.org/build/tools/rev/44faaa1a3937
Attachment #8395828 - Flags: checked-in+
Regenerated the bundles.
(In reply to Rail Aliiev [:rail] from comment #4) > Regenerated the bundles. <3 Any reason to keep this open? We can pursue other options in another bug another time.
7 years ago
Assignee: nobody → rail
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Summary: hg bundles suck → use gzip compression for hg bundles
23 minutes to do an unbundle from a gzipped bundle and 11:20 to unzip?! My computer can do a gzip unbundle in 8 minutes. And 4 of those should go away with bundle2 (I'd like to add cache data into bundles so clients don't need to regenerate it).
You need to log in before you can comment on or make changes to this bug.