1399918 - mozharness vcstools should allow for default-push path when using hg

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect, P2)

Description

[Simon Fraser [:sfraser] ⌚️GMT]

At the moment during merge day tasks we're often cloning mercurial repositories using ssh:

https://dxr.mozilla.org/mozilla-central/rev/f9a5e9ed62103c84e4cde915f4d08f1ce71be83e/testing/mozharness/configs/merge_day/beta_to_release.py#45-46

Mercurial allows us to specify a default-push path, so we could clone over https have an hgrc file like this:

[paths]
default = https://hg.mozilla.org/releases/mozilla-beta
default-push = ssh://hg.mozilla.org/releases/mozilla-beta

Only pushes will use ssh, and pulls will use https, as recommended.

This would require an extra field to be supported, listing any extra paths to be added in https://dxr.mozilla.org/mozilla-central/rev/f9a5e9ed62103c84e4cde915f4d08f1ce71be83e/testing/mozharness/mozharness/base/vcs/mercurial.py#332 and some modification to the hgrc file (looks like it might be compatible with configparser/ConfigParser? Safer if so)

Comment 1

[Chris AtLee [:catlee]]

We should definitely fix this before the next merge day.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Might be too late for this cycle, but ...

Presumably the goal here is a fast https clone via a bundle, while maintaining ssh for the push. If the config was switched to https then the destination for the push can be passed on the command line without needing to set up an hgrc. And MercurialRepoManipulationMixin already supports a push_dest in the config, 
 https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/repo_manipulation.py#151
so something like this:

diff --git a/testing/mozharness/configs/merge_day/beta_to_release.py b/testing/mozharness/configs/merge_day/beta_to_release.py
--- a/testing/mozharness/configs/merge_day/beta_to_release.py
+++ b/testing/mozharness/configs/merge_day/beta_to_release.py
@@ -34,6 +34,7 @@ config = {
     "tools_repo_url": "https://hg.mozilla.org/build/tools",
     "tools_repo_branch": "default",
-    "from_repo_url": "ssh://hg.mozilla.org/releases/mozilla-beta",
-    "to_repo_url": "ssh://hg.mozilla.org/releases/mozilla-release",
+    "from_repo_url": "https://hg.mozilla.org/releases/mozilla-beta",
+    "to_repo_url": "https://hg.mozilla.org/releases/mozilla-release",
+    "push_dest": "ssh://hg.mozilla.org/releases/mozilla-release",

     "base_tag": "FIREFOX_RELEASE_%(major_version)s_BASE",

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Ah, but that won't work because we're pushing to more than one repo - a tag into 'from', all the history into 'to'.

Comment 4

[Johan Lorenzo [:jlorenzo]]

Yesterday was mergeday. For the record, I manually edited $repo/.hg/hgrc, just like sfraser pointed out in comment 0. I had introduced a few errors (bad copy and paste). I also needed to edit ~ubuntu/.ssh/config to get the correct username:

> Host hg.mozilla.org git.mozilla.org
>    User [e-mail]
>    Compression yes
>    ServerAliveInterval 300

Also, kmoir had defined .hgrc, which I completed:

> [ui]
> #ssh = ssh -C -i ~/.ssh/kmoir_rsa.pub
> ssh = ssh -C -i ~/.ssh/jlorenzo_rsa.pub
> 
> [extensions]
> strip = 

(The strip extension is handy in case one push goes wrong). Thanks to that config, I was able to pull from https and push via ssh.

[1] https://github.com/mozilla/releasewarrior/blob/108bd1ff0a5840a412500efa95ee1a9b8a996a6b/how-tos/merge-and-staging-instance.md

Comment 5

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1399918 - mozharness vcstools should allow for default-push path when using hg

Review commit: https://reviewboard.mozilla.org/r/198104/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/198104/

Comment 6

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8926856 [details]
Bug 1399918 - mozharness vcstools should allow for default-push path when using hg

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/198104/diff/1-2/

Comment 7

[Johan Lorenzo [:jlorenzo]]

For the record, this patch is not needed for Firefox 57. The review requests can happen after 57 is shipped.

Comment 8

[Simon Fraser [:sfraser] ⌚️GMT]

Comment on attachment 8926856 [details]
Bug 1399918 - mozharness vcstools should allow for default-push path when using hg

https://reviewboard.mozilla.org/r/198104/#review203350

Looks good to me - the use of ConfigParser is great. It should be easily caught when the python3 compatibility work catches up, too.

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8926856 [details]
Bug 1399918 - mozharness vcstools should allow for default-push path when using hg

https://reviewboard.mozilla.org/r/198104/#review203516

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/edef5e008fe5
mozharness vcstools should allow for default-push path when using hg r=nthomas,sfraser

Comment 11

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/edef5e008fe5