1350437 - Upgrade to Mercurial 4.1

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Gregory Szorc [:gps]]

+++ This bug was initially created as a clone of Bug #1298944 +++

We should upgrade TaskCluster images to Mercurial 4.1 for performance reasons. This will get us zstd between automation and hg.mo once 4.1 is deployed on hg.mo.

Comment 1

[Gregory Szorc [:gps]]

Attachment: Bug 1350437 - Install Mercurial 4.1.1+;

We've been running Mercurial 3.9 in automation for a while. Mercurial
4.1 is out. It has the usual performance improvements and bug fixes,
making it an attractive upgrade. But what really makes it enticing
is support for zstandard compression over the wire protocol. This will
reduce server-side CPU load and make transfers between 4.1+ servers
faster.

So, let's upgrade to Mercurial 4.1.1.

The produced packages are built from the current tip of the
stable Mercurial branch, not exactly 4.1.1. Specifically, they
correspond to revision ed5b25874d99. I did this because there is
a patch in the stable branch that drastically improves performance
on repos with many heads. When 4.1.2 is released in a few days, we
can upgrade to it and do away with the one-off.

Review commit: https://reviewboard.mozilla.org/r/124760/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/124760/

Comment 2

[Gregory Szorc [:gps]]

Attachment: Bug 1350437 - Bump image_builder version and hash;

We recently changed how image compression works (bug 1350447) and
upgraded to Mercurial 4.1.1 (this bug). It's time to roll a new
image_builder Docker image.

This commit updates the version and hash of the image. I was
unable to push this image to the taskcluster account on Docker
Cloud. Not sure why - I can access the repository just fine via
the web UI. You can pull indygreg/image_builder:1.3.0 and re-push
that to taskcluster/image_builder:1.3.0.

Review commit: https://reviewboard.mozilla.org/r/124762/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/124762/

Comment 3

[Gregory Szorc [:gps]]

Try push for this is at https://treeherder.mozilla.org/#/jobs?repo=try&revision=889a412fdfcdccb9e7efb4979b4a4217bd6d05ab.

aki: you'll want to wait for dustin's review before whitelisting the new hash.
dustin: if you r+ this, could you please follow the instructions in the commit message to re-publish the image to the taskcluster account (since I apparently don't have permissions).

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8852565 [details]
Bug 1350437 - Install Mercurial 4.1.1+;

https://reviewboard.mozilla.org/r/124760/#review127288

I might prefer to have the packages named "4.1.1+" or something like that, but if you're going to upgrade to 4.1.2 soon anyway, it's not a big deal.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8852566 [details]
Bug 1350437 - Bump image_builder version and hash;

https://reviewboard.mozilla.org/r/124762/#review127290

::: commit-message-f99ad:11
(Diff revision 1)
> +
> +This commit updates the version and hash of the image. I was
> +unable to push this image to the taskcluster account on Docker
> +Cloud. Not sure why - I can access the repository just fine via
> +the web UI. You can pull indygreg/image_builder:1.3.0 and re-push
> +that to taskcluster/image_builder:1.3.0.

Hm, digest doesn't match:

Digest: sha256:ceaaf92511cfbff711598005585127953873332c62f245dcf1892510c4eb371f
Status: Downloaded newer image for indygreg/image_builder:1.3.0

Comment 6

[Gregory Szorc [:gps]]

Comment on attachment 8852566 [details]
Bug 1350437 - Bump image_builder version and hash;

https://reviewboard.mozilla.org/r/124762/#review127290

> Hm, digest doesn't match:
> 
> Digest: sha256:ceaaf92511cfbff711598005585127953873332c62f245dcf1892510c4eb371f
> Status: Downloaded newer image for indygreg/image_builder:1.3.0

I've pushed that tag multiple times. Try `docker pull indygreg/image_builder@sha256:ceaaf92511cfbff711598005585127953873332c62f245dcf1892510c4eb371f`.

Comment 7

[Gregory Szorc [:gps]]

Comment on attachment 8852565 [details]
Bug 1350437 - Install Mercurial 4.1.1+;

https://reviewboard.mozilla.org/r/124760/#review127288

That's a fair criticism. If I were installing these packages on a real server, I'd care about this. But since these are essentially local-only files and we don't have to worry about upgrading packages using the system package manager, I'm not worried.

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

ok, pushed :)

1.3.0: digest: sha256:ceaaf92511cfbff711598005585127953873332c62f245dcf1892510c4eb371f size: 3440

Comment 9

[Gregory Szorc [:gps]]

Comment on attachment 8852566 [details]
Bug 1350437 - Bump image_builder version and hash;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/124762/diff/1-2/

Comment 10

[Gregory Szorc [:gps]]

Attachment: Bug 1350437 - Add image_builder 1.3.0 hash to allow list;

We just cut a new image_builder version. We need to add its hash
to the allow list.

Review commit: https://reviewboard.mozilla.org/r/124800/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/124800/

Comment 11

[Gregory Szorc [:gps]]

Comment on attachment 8852612 [details]
Bug 1350437 - Add image_builder 1.3.0 hash to allow list;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/124800/diff/1-2/

Comment 12

[Gregory Szorc [:gps]]

https://github.com/mozilla-releng/scriptworker/pull/103 is also on file to update the allow list.

I guess we now wait for the allow list changes to land and get deployed.

Comment 13

[Aki Sasaki (not active)]

Comment on attachment 8852612 [details]
Bug 1350437 - Add image_builder 1.3.0 hash to allow list;

https://reviewboard.mozilla.org/r/124800/#review127324

Comment 14

[Aki Sasaki (not active)]

Comment on attachment 8852566 [details]
Bug 1350437 - Bump image_builder version and hash;

https://reviewboard.mozilla.org/r/124762/#review127326

Comment 15

[Ian Moody [:Kwan] (UTC+0)]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7175a2fbf504
Install Mercurial 4.1.1+; r=dustin
https://hg.mozilla.org/integration/autoland/rev/c5cecddbe344
Bump image_builder version and hash; r=aki

Comment 16

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/7175a2fbf504
https://hg.mozilla.org/mozilla-central/rev/c5cecddbe344