Generate zstd bundles with high compression

RESOLVED FIXED

Status

Developer Services
Mercurial: hg.mozilla.org
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: gps, Assigned: gps)

Tracking

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(4 attachments)

(Assignee)

Description

a year ago
Mercurial 4.1 introduces support for the zstd compression format.

Compared to zlib, zstd bundles for mozilla-unified can be several hundred megabytes smaller. See https://www.mercurial-scm.org/repo/hg/rev/41a8106789ca for some numbers.

This means that people cloning Firefox with clone bundles could potentially save hundreds of megabytes from the transfer, allowing them to acquire the Firefox source code faster and with lower risk of e.g. a connection drop.

I'd like to roll out generation of zstd bundles for certain repositories (namely mozilla-central and mozilla-unified) *before* Mercurial 4.1 is running on the SSH and HTTP endpoints. We would do this by introducing a new virtualenv with Mercurial 4.1 and then changing the bundle generation timer to run from that virtualenv. There should be little risk involved with doing this: our big reason for not adopting new Mercurial releases as soon as they come out is stability on the SSH and HTTP endpoints and compatibility with extensions. The bundle generation mechanism shouldn't have these concerns.
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
(Assignee)

Comment 5

a year ago
I'm on PTO early next week and probably won't have time to deploy this until Wednesday at the earliest. At that point, we can just install Mercurial 4.1 final instead of RC. If you r+, I'll likely wait for 4.1 final before landing and deploying.
Assignee: nobody → gps
Status: NEW → ASSIGNED

Comment 6

a year ago
mozreview-review
Comment on attachment 8831375 [details]
ansible/hg-ssh: create a virtualenv with Mercurial 4.1 (bug 1331084);

https://reviewboard.mozilla.org/r/107938/#review109538
Attachment #8831375 - Flags: review?(glob) → review+

Comment 7

a year ago
mozreview-review
Comment on attachment 8831376 [details]
hgserver: use Mercurial 4.1 for generating clone bundles (bug 1331084);

https://reviewboard.mozilla.org/r/107940/#review109540
Attachment #8831376 - Flags: review?(glob) → review+

Comment 8

a year ago
mozreview-review
Comment on attachment 8831377 [details]
hgserver: support generating zstd bundles (bug 1331084);

https://reviewboard.mozilla.org/r/107942/#review109542
Attachment #8831377 - Flags: review?(glob) → review+

Comment 9

a year ago
mozreview-review
Comment on attachment 8831378 [details]
hgserver: generate zstd bundles for mozilla-central-gd and v-c-t (bug 1331084);

https://reviewboard.mozilla.org/r/107944/#review109544
Attachment #8831378 - Flags: review?(glob) → review+
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)

Comment 14

a year ago
Pushed by gszorc@mozilla.com:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/9da7ff5a4d0f
ansible/hg-ssh: create a virtualenv with Mercurial 4.1 ; r=glob
https://hg.mozilla.org/hgcustom/version-control-tools/rev/b7706f32bbe3
hgserver: use Mercurial 4.1 for generating clone bundles ; r=glob
https://hg.mozilla.org/hgcustom/version-control-tools/rev/2bd6a73c1f15
hgserver: support generating zstd bundles ; r=glob
https://hg.mozilla.org/hgcustom/version-control-tools/rev/6b36f450cadd
hgserver: generate zstd bundles for mozilla-central-gd and v-c-t ; r=glob
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
(Assignee)

Updated

a year ago
Blocks: 1336190
You need to log in before you can comment on or make changes to this bug.