Closed Bug 1670930 Opened 4 years ago Closed 4 years ago

make stream clones the default in robustcheckout

Categories

(Developer Services :: Mercurial: robustcheckout, enhancement)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: sheehan, Assigned: sheehan)

References

Details

Attachments

(2 files)

In bug 1665379, glandium identified a pool of macOS mini PCs that were not cloning via stream clonebundles from hgmo. Since these macs are outside of any cloud where we run Firefox CI, they aren't picked up by our server-side optimization to serve a stream clone bundle.

Since robustcheckout is used in most places in CI, we should make stream clones the default behaviour for robustcheckout. The server-side optimization can stand as-is, which will force bundles to be downloaded from the same AWS region, if it is available. But now our on-premises CI will get stream clones, as long as they use robustcheckout.

Assignee: nobody → sheehan

Stream clones are faster than compressed clones on very fast,
high bandwidth networks. In CI, where robustcheckout is used,
we almost certainly have a very fast high bandwidth network to
take advantage of. This commit changes clone operations within
robustcheckout to use stream clone bundles unconditionally. The
optimization around serving a bundle from the same AWS/GCP region
as the originating request IP will still take place server-side,
but now tasks in Mozilla DCs and any other niche environment will
be stream-clone by default.

The change itself simply involves adding stream=True to the
clone function, however many tests need to be updated to reflect
the stream clone output. We use a regex on several lines to paper
over timing output in tests, including one regex where the decimal
value is conditional (ie sometimes the output is 15 seconds and
sometimes the output is 15.4 seconds).

Pushed by cosheehan@mozilla.com:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/4bc2df61a2be
robustcheckout: make stream clones the default behaviour r=dhouse

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED

Vendors the latest robustcheckout from version-control-tools, revision
db4a0fb8baa9. Changes include adding a --noupdate flag, removing a
compatibility version check that caused Mercurial version upgrades to
unnecessarily fail, and making stream clones the default behaviour.

Pushed by cosheehan@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/234d3b9d13a0
update robustcheckout to state at db4a0fb8baa9 in version-control-tools r=dhouse
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: