1225673 - Generate reordered bundles

Status: NEW

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Gregory Szorc [:gps]]

Mercurial has a bundle.reorder=true config option that will change the order of entries in bundles to minimize deltas between revisions. Bundles take longer to generate because Mercurial has to re-delta-fy the data. But, the bundles are smaller and thus take less resources to download and apply.

Let's produce reordered bundles as part of our bundle generation CRON.

I'm not yet sure if we should always produce reordered bundles or if we should produce reordered bundles in addition to default bundles. We may only have CPU to produce reordered bundles for certain repos.

Also, there is a minor chance that producing bundles where the order doesn't match up to exactly what would happen if you did `hg clone` could produce issues in automation. However, anything relying on the numeric revision number instead of the SHA-1 is busted anyway. Just something to watch out for.

Comment 1

[Gregory Szorc [:gps]]

It's taking an insane amount of time to generate a reordered bundle for mozilla-central. Like close to an hour at this point. This feels significantly slower than what my new Skylake computer at home was able to do :/

Anyway, I don't think we'll be able to enable reordering globally because of the overhead.

Comment 2

[Gregory Szorc [:gps]]

76 minutes CPU time and 82 minutes wall time to generate the reordered mozilla-central bundle on hgssh1. Boo. Definitely can't do this globally.