use stream clone bundles to seed repos during hgweb bootstrap
Categories
(Developer Services :: Mercurial: hg.mozilla.org, enhancement)
Tracking
(Not tracked)
People
(Reporter: sheehan, Assigned: sheehan)
References
Details
Attachments
(1 file)
The hgweb bootstrap procedure is pretty slow at the moment, since the entire process depends on performing an hg pull from hgssh to download each repository. Instead we should use stream clone bundles to seed most of the repo data (as we do in CI for instance) and then follow up with an hg pull to get the remaining repository data.
|
Assignee | |
Comment 1•6 years ago
|
||
At the moment the bootstrap procedure relies entirely on hg pull
to transfer repository data during the bootstrap procedure. This is
because bootstrap simply calls the same function from within it's
code that the replication system would call when running
hg replicatesync to force synchronize a repo on all hgweb mirrors.
This implementation detail causes bootstrapping to take much longer
than necessary.
This commit replaces the call to create an empty repository and then
hg pull the data into it, with a call to hg clone --stream that
falls back to an empty repo on failure. In production this should result
in the repo being seeded from a clonebundle using the Mozilla CDN,
which will dramatically speed up the bootstrap process.
We add a fallback to regular empty repository + hg pull due to the
test environment not having a clonebundle available for download. Since
there is no clonebundle to be used, the stream clone attempts to directly
access the underlying revlogs and other files in .hg directory of a
given repository, causing a permissions error. This should be avoided
in production, and in the worst case, we will fall back to the current
slow behaviour.
Pushed by cosheehan@mozilla.com:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/848fe5481f07
vcsreplicator: use stream clones in bootstrapping procedure r=mhentges
Description
•