1784232 - Add `base_rev` parameter in taskgraph

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, enhancement)

Description

[Johan Lorenzo [:jlorenzo]]

In order to solve [1] in all cases, we need taskgraph to be able to compute the list of changed files in a range of revisions. This way, taskgraph doesn't need to rely on hg.m.o/json-automationrelevance or api.github.com/repos/compare. This means taskgraph will be able to compute the list only with the local clone.

taskgraph handles both hg and git so we have to keep this in mind. On top of my head, I'm thinking of these cases:

Hg

1. base_rev and head_ref on the same branch
2. base_rev and head_ref on different branches of the same repo

Git

1. base_rev and head_ref on the same branch
2. base_rev and head_ref on different branches of the same repo
3. base_rev and head_ref in different repos
4. head_ref is a PR
5. base_rev is null commit
6. base_rev doesn't exist

Number 5 and 6 may seem out of place at a first glance. Although, it's pretty standard in the git workflow. If you push a new branch, then the null commit is provided no matter if you're on Github or Gitlab. If you force-push a branch, then the base_rev doesn't exist anymore.

I added base_rev[2] in the fork of taskgraph I created last year. I also handled the 2 corner cases[3][4] I mentioned above. Let's see if I can port that upstream 🙂

[1] https://github.com/taskcluster/taskgraph/issues/81
[2] https://gitlab.com/jobgraph-dev/jobgraph/-/commit/285682359d1f3b70b2fab33b16e309573b66b72e
[3] https://gitlab.com/jobgraph-dev/jobgraph/-/commit/8644c2b9f7569da81e5f9d465438cf77a6b10236
[4] https://gitlab.com/jobgraph-dev/jobgraph/-/commit/f0f5106acfd8149f0aa1f614eef27fff73c3c897

Comment 1

[Johan Lorenzo [:jlorenzo]]

Attachment: [taskgraph] Add base_rev parameter in taskgraph

Comment 2

[Johan Lorenzo [:jlorenzo]]

Attachment: [scriptworker] Let CoT support BASE_REF and BASE_REV

Comment 3

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1784232 - part 1: Add `base_rev` parameter to hg-push context r=aki

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/9616a7e79438
part 1: Add `base_rev` parameter to hg-push context r=aki

Comment 5

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1784232 - part 2: Use new build-decision image r=jcristau

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/fae86ad70194
part 2: Use new build-decision image r=jcristau

Comment 7

[Johan Lorenzo [:jlorenzo]]

https://github.com/mozilla/build-relengdocs/commit/7b0dc69c721f93657e55dd28fcff03932809c6ea

Comment 8

[Johan Lorenzo [:jlorenzo]]

Attachment: [ronin_puppet] Bump scriptworker (and scriptworker_script) to 43.5.0

Comment 9

[Johan Lorenzo [:jlorenzo]]

Attachment: [scriptworker-scripts] Ran pin.sh to bump scriptworker to 43.5.0

Comment 10

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1784232 - part 4: Let hooks allow `base_revision` r=ahal

Comment 11

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1784232 - Bump taskgraph to v3.2.0 r=ahal

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/88b8c2378d27
part 3: Let hooks allow `base_revision` r=ahal

Comment 13

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/ci/ci-configuration/rev/03037c56a257
Backed out changeset 88b8c2378d27 on suspicion of breaking gecko decision tasks.

Comment 14

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1784232 - part 3: Fix missing revision in cron jobs r=jcristau

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/98192abe93da
part 3: Fix missing revision in cron jobs r=jcristau

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/43aefcbf24bf
part 4: Let hooks allow `base_revision` r=ahal

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f737234929d4
Bump taskgraph to v3.0.0 r=ahal

Comment 18

[Cristina Cozmuta (:CrissCozmuta)]

https://hg.mozilla.org/mozilla-central/rev/f737234929d4

Comment 19

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backed out for breaking mach commands with git-cinnbar clone:
https://hg.mozilla.org/mozilla-central/rev/65e0896af25d207be5573f3a9e205bb350967473

Comment 20

[Johan Lorenzo [:jlorenzo]]

Clearining NI. New revision is ready in Phab.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/13f5f52e65c5
Bump taskgraph to v3.2.0 r=ahal

Comment 22

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/13f5f52e65c5

Comment 23

[Mike Hommey [:glandium]]

Another regression: bootstrapping fails because: Could not parse URL: hg::https://hg.mozilla.org/mozilla-unified

Comment 24

[Eemeli Aro [:eemeli]]

After https://hg.mozilla.org/integration/autoland/rev/13f5f52e65c5, clobbered builds on my M1 Macbook first fail with an error

...
 0:08.18 checking for cbindgen... no
 0:08.18 ERROR: Cannot find cbindgen. Please run `mach bootstrap`,
 0:08.18 `cargo install cbindgen`, ensure that `cbindgen` is on your PATH,
 0:08.18 or point at an executable with `CBINDGEN`.

Running mach bootstrap doesn't help for that, but cargo install cbindgen does. However, after that I get a later failure that isn't so straightforward to figure out:

...
 0:09.96 ERROR: Cannot find a wasi sysroot. Please give its location with --with-wasi-sysroot. Or build with --without-wasm-sandboxed-libraries.

Comment 25

[Johan Lorenzo [:jlorenzo]]

(In reply to Mike Hommey [:glandium] from comment #23)

Another regression: bootstrapping fails because: Could not parse URL: hg::https://hg.mozilla.org/mozilla-unified

Discussed on #developers (Element). I managed to reproduce when origin is set as a remote repository. I'm on a fix. As a temporary workaround, you can git remote rename origin mozilla just for the time being

(In reply to Eemeli Aro [:eemeli] from comment #24)

After https://hg.mozilla.org/integration/autoland/rev/13f5f52e65c5, clobbered builds on my M1 Macbook first fail with an error

Discussed on #developers (Element) too. This time, I couldn't repro. As a matter of fact, I doubt this commit is the one that clobbered builds. In fact, Monday was mergeday and as of now, [1] is the last commit that introduced a clobber. Moreover, this commit didn't change anything related to rust nor cbindgen. Thus, I don't think this is a regression caused by the taskgraph bump. I might be missing something though, and I'm happy to reconsider my conclusions if new elements come into play.

[1] https://hg.mozilla.org/mozilla-central/rev/d42e0ca4bb3e3d7fa475687da045300b07a20db1

Comment 26

[Eemeli Aro [:eemeli]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #25)

(In reply to Eemeli Aro [:eemeli] from comment #24)

After https://hg.mozilla.org/integration/autoland/rev/13f5f52e65c5, clobbered builds on my M1 Macbook first fail with an error

Discussed on #developers (Element) too. This time, I couldn't repro. As a matter of fact, I doubt this commit is the one that clobbered builds. In fact, Monday was mergeday and as of now, [1] is the last commit that introduced a clobber. Moreover, this commit didn't change anything related to rust nor cbindgen. Thus, I don't think this is a regression caused by the taskgraph bump. I might be missing something though, and I'm happy to reconsider my conclusions if new elements come into play.

[1] https://hg.mozilla.org/mozilla-central/rev/d42e0ca4bb3e3d7fa475687da045300b07a20db1

Nevertheless, specifically reverting the referenced commit allows for a build to succeed on my machine, whereas with it the build fails as above. I have no idea why, and only identified this by bisecting.

I first hit this issue when rebasing my current stack from yesterday's central to today's, and encountered the first error without clobbering.

Comment 27

[Mike Hommey [:glandium]]

Thus, I don't think this is a regression caused by the taskgraph bump. I might be missing something though, and I'm happy to reconsider my conclusions if new elements come into play.

You're missing the part of the log that is not pasted in comment 24 and that says the same thing as comment 23, and prevents the auto-bootstrap stuff to work, which makes configure not find things it would normally download.

Comment 28

[Johan Lorenzo [:jlorenzo]]

Attachment: [mozilla-repo-urls] feat: Support git-cinnabar URLs

Comment 29

[Eemeli Aro [:eemeli]]

Ah, true, I'd missed out seeing this in the logs earlier, when building with the commit included:

 0:01.41 WARNING: Could not parse URL: hg::https://hg.mozilla.org/mozilla-unified

I see that (and the above failures) when switching from a branch that includes a revert to one that doesn't.

After applying the recommended workaround

git remote rename origin mozilla

the print in the build log is

 0:01.17 error: No such remote 'origin'

but the build does succeed.

Comment 30

[Johan Lorenzo [:jlorenzo]]

Attachment: [taskgraph] Bump dependencies to get mozilla-repo-urls v0.1.0

Comment 31

[Johan Lorenzo [:jlorenzo]]

Thank you both for the extra information. I agree comment 24 is a regression caused by this current bug. The URL parsing error is fixed in the library, it's now a matter of bumping it.

:ahochheiden, would you have the bandwidth to bump mozilla-repo-urls from 0.0.3[1] to 0.1.0? I'm still facing bug 1782785 and I cannot make this bump on my machine. I remember you have a patch that allows you to bump python deps on yours :)

[1] https://hg.mozilla.org/integration/autoland/rev/13f5f52e65c5#l38.576

Comment 33

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

I sent the patch with the vendor of mozilla-repo-urls at 0.1.0 Johan over Slack.

Comment 34

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1784232 - part 2: Vendor `mozilla-repo-urls` at 0.1.0 r=ahal

Comment 35

[Johan Lorenzo [:jlorenzo]]

Thank you very much :ahochheiden! I tested the bump on my local git-cinnabar clone, with origin set as remote and I confirm it fixes comment 24 and bug 1791797 comment 0.

Comment 36

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/de201abcf97d
part 2: Vendor `mozilla-repo-urls` at 0.1.0 r=ahal

Comment 37

[Nika Layzell [:nika] (ni? for response)]

This new patch doesn't appear to fix the issue for me, when I try to apply it. I still get errors complaining about paths like origin/HEAD being ambiguous, and eventually it crashes due to trying to run:

subprocess.CalledProcessError: Command '('/usr/bin/git', 'merge-base', 'origin/branches/default/tip', 'acd5a70dbae85b86d8d1090745f26ab0af0569eb')' returned non-zero exit status 128.

It appears that the code assumes that origin/branches/default/tip is the relevant upstream base commit to find the merge-base with, which I don't believe is correct (I have this vague memory of this not being even guaranteed to be central, so you might find the merge-base with e.g. beta? though I may be misremembering the specific issue). Due to my git configuration locally, the upstream which should be selected is origin/central (as I have configured git to re-name the long name of origin/bookmarks/central to origin/central).

This automated process perhaps could recover from errors better, to avoid blocking the ability to bootstrap and/or build firefox if the local checkout's configuration isn't quite what we're expecting.

Comment 38

[Nika Layzell [:nika] (ni? for response)]

To add to this, it seems like if I replace https://searchfox.org/mozilla-central/rev/52da19becaa3805e7f64088e91e9dade7dec43c8/third_party/python/taskcluster_taskgraph/taskgraph/util/vcs.py#364 with an implementation which just returns "origin/central" it works for me. I think the issue is that _get_default_branch_from_remote_query returns the name of a branch which isn't actually available locally (in my case origin/branches/default/tip) meaning that I end up with other code failing when it tries to merge-base with it.

Comment 39

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

I'm no longer able to use ./mach try chooser — it's trying to find the merge-base with origin/master, but the remote named origin is gecko-dev and I'm on a detached head based on a different remote, a cinnabar clone of hg::https://hg.mozilla.org/mozilla-central, so as far as Git can tell that's an unrelated history.

I've also noticed that during builds it's trying to resolve fork/HEAD which doesn't exist; the branch's upstream (in the Git sense) is fork/<branchname>, where fork is my GitHub fork of gecko-dev, so I guess that's where it's getting that from, but none of what's happening is helpful for finding the m-c base revision. Nothing is obviously broken yet, but maybe it's going to fail to download build tools in the future.

Comment 40

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/de201abcf97d

Comment 41

[Mathew Hodson]

Maybe it would be useful to have an option for remote in the .mozconfig file that could be set to the upstream base branch. MozPhab already has something similar in its .moz-phab-config file. I have it set like this:

[git]
remote = origin/master
command_path = git.exe

Comment 42

[Johan Lorenzo [:jlorenzo]]

Thank you all for taking the time to understand the error, find this bug, try a couple things and provide detailed feedback. I wholeheartedly appreciate it! There are still some cases that obviously don't work so I'm reopening this ticket. Let me go through the details of each comment 👀

(In reply to Nika Layzell [:nika] (ni? for response) from comment #37)

It appears that the code assumes that origin/branches/default/tip is the relevant upstream base commit [...]

The current heuristic tries to get the most accurate base branch. If your local branch tracks an upstream one, then it uses the default branch of the corresponding remote[1]. Otherwise[2], it looks at the metadata gotten during the first git clone to see if there's a default branch defined, then it asks the remote repo what its default branch is. Then, if none of that works, it guesses based on a list of hardcoded branch names, including branches/default/tip for the specific case of git-cinnabar.

I'm not sure which code path led you to origin/branches/default/tip. Although, that value looks sensible to me. Allow me to explain below 🙂

[...] to find the merge-base with, which I don't believe is correct (I have this vague memory of this not being even guaranteed to be central, so you might find the merge-base with e.g. beta? though I may be misremembering the specific issue).

So far, my testing has shown that origin/branches/default/tip matches the tip of autoland. I don't know but I wouldn't surprised if there is no guarantee of that. That said, I ran these 2 commands locally and none of them errored:

git merge-base origin/bookmarks/beta HEAD
git merge-base origin/bookmarks/esr91 HEAD

HEAD being a commit made on top of central. For the record, git merge-base finds the first common commit between 2 revisions. This means, even if origin/branches/default/tip was set to beta, it would just return the last commit before mozilla-beta diverged from mozilla-central. In order words, I believe the error comes from elsewhere and I wonder if it's related to :jld's comment.

(In reply to Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧ from comment #39)

I'm no longer able to use ./mach try chooser — it's trying to find the merge-base with origin/master, but the remote named origin is gecko-dev and I'm on a detached head based on a different remote, a cinnabar clone of hg::https://hg.mozilla.org/mozilla-central, so as far as Git can tell that's an unrelated history.

Thank you for the context. I hadn't realized people could have 2 remotes with 2 total different histories. :glandium told me earlier today that you cannot use ./mach try on gecko-dev, that's why people can have both. I guess that's your case? If so, I'd like to understand when you use one rather than the other. I'm going to spitball: would renaming origin to another name and setting origin to hg::https://hg.mozilla.org/mozilla-unified be a suitable solution in your case?

I've also noticed that during builds it's trying to resolve fork/HEAD which doesn't exist; the branch's upstream (in the Git sense) is fork/<branchname>, where fork is my GitHub fork of gecko-dev, so I guess that's where it's getting that from, but none of what's happening is helpful for finding the m-c base revision. Nothing is obviously broken yet, but maybe it's going to fail to download build tools in the future.

Interesting! So that's how the heuristic is made to behave (see above). The call to fork/HEAD when taskgraph asks the remote repo what its default branch is. I'm not sure why it doesn't exist. I just cloned your fork[3] and so far this git rev-parse --abbrev-ref fork/HEAD returns fork/master, which is the expected value. Could you tell me what it returns on your end?

:nika, would you happen to use both gecko-dev and git-cinnabar too?

[1] https://github.com/taskcluster/taskgraph/blob/3db6d85d993d05335dacffcbdcdef828bb3cd931/src/taskgraph/util/vcs.py#L352
[2] https://github.com/taskcluster/taskgraph/blob/3db6d85d993d05335dacffcbdcdef828bb3cd931/src/taskgraph/util/vcs.py#L363-L380
[3] https://github.com/jld/gecko-dev

Comment 43

[Johan Lorenzo [:jlorenzo]]

(In reply to Mathew Hodson from comment #41)

Maybe it would be useful to have an option for remote in the .mozconfig file that could be set to the upstream base branch. MozPhab already has something similar in its .moz-phab-config file.

That sounds like an interesting idea. I had a look at the git-config manual[1]. I'm not sure how we can set a remote that doesn't get changed or removed by a subsequent git command. Moreover, taskgraph aims to be generic and to support repos outside of gecko. So, if we were to make such a change, we cannot rely on ./mach bootstrap. It would have to leverage taskgraph alone, I think.

That said, there are likely things I didn't consider and I'm happy to discuss the details of this proposal 🙂

[1] https://git-scm.com/docs/git-config

Comment 44

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #42)

so far git rev-parse --abbrev-ref fork/HEAD returns fork/master

Status update: I found a way to reproduce this issue! I was testing[1] (bug 1791852) locally when I got this error:

RuntimeError: Unable to find default branch. Got: ['refs/heads/main', 'refs/remotes/origin/main', 'refs/remotes/upstream/main']

It turns out taskgraph ends up guessing the default branch[2] instead of using the cloned metadata[3]. The following command failed this way:

$ git rev-parse --abbrev-ref origin/HEAD
refs/origin/HEAD
fatal: ambiguous argument 'origin/HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

The next one is more explicit:

git rev-parse --abbrev-ref origin/HEAD --
fatal: bad revision 'origin/HEAD'

In my particular case, origin was never cloned per se, instead origin was renamed to upstream and then I created another remote called origin.

Although, the query that asks origin its default branch should work. It works fine in a separate shell but somehow exit with error 128 in Python. I'm digging deeper.

Edit: I found what's going on. In python, I get this error message:

[e-mail]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

That's because I'm using SSH and my private key is not accessible to the current process. I'm not sure what we can do about this. I'll see why origin/HEAD doesn't work in the first place.

[1] https://github.com/taskcluster/taskgraph/pull/119
[2] https://github.com/taskcluster/taskgraph/blob/3db6d85d993d05335dacffcbdcdef828bb3cd931/src/taskgraph/util/vcs.py#L380
[3] https://github.com/taskcluster/taskgraph/blob/3db6d85d993d05335dacffcbdcdef828bb3cd931/src/taskgraph/util/vcs.py#L396

Comment 45

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #44)

I'll see why origin/HEAD doesn't work in the first place.

[1] was a great read. More precisely this part:

What sets origin/HEAD?

• git clone fetches and sets it
• it would make sense if git fetch updates it like any other reference, but it doesn’t
• git remote set-head origin -a fetches and sets it
  • useful to update the local knowledge of what remote considers the “default branch”

In other words, when a user defines more than one remote repo, then git rev-parse --abbrev-ref $REMOTE_NAME/HEAD only works on the remote that was cloned from. Thus, this logic will statically fail too often. We could let taskgraph run git remote set-head $REMOTE_NAME --all but this call won't work on authenticated repos (like SSH clones) and will fail the same way as in my previous comment. We could try to let users input their password but that would be quite a strange experience when a user runs a command like mach build.

The fact that many git-cinnabar users face these taskgraph bugs is because of the way we set default parameters. Taskgraph params are meant to be set to sane default values anytime we call it. Since taskgraph 3.x, we process default_branch[2] as one of the default values. Actually, taskgraph requires a real default_branch in the context of the decision task (which is the original goal of this bug). mach commands don't care about this value. :glandium suggested we stop providing default_branch as a default value. :ahal and I just met and we agree this is the right approach. We haven't found a better way to keep an enjoyable developer experience while using this new taskgraph param just on CI/CD.

I'm on it.

[1] https://stackoverflow.com/questions/8839958/how-does-origin-head-get-set#29405808
[2] https://github.com/taskcluster/taskgraph/blob/49d407c4dd2b2bb5c335c9e24f4179d675ce5bde/src/taskgraph/parameters.py#L89

Comment 46

[Johan Lorenzo [:jlorenzo]]

Attachment: [taskgraph] Provide empty base_ref and base_rev in default parameters

Comment 47

[:gerard-majax]

I dont know why but it seems that this is completely breaking everything here, I can't build at all

 0:01.06 checking for Git version... 2.34.1
 0:01.06 checking for sparse checkout... no
 0:01.27 WARNING: Unsupported platform. Got: ('hgmo', <mozilla_repo_urls.platforms.hgmo.HgmoPlatform object at 0x7fb87b925090>). Expected: ('hgmo', 'github'). URL: https://git.mozilla.org/releases/gecko.git
 0:01.27 checking for wget... /usr/bin/wget
...
 0:05.55 ERROR: Could not find clang to generate run bindings for C/C++. Please install the necessary packages, run `mach bootstrap`, or use --with-clang-path to give the location of clang.
*** Fix above errors and then restart with "./mach build"

Comment 48

[:gerard-majax]

$ git revert --no-commit 9173c3ca848de048ceb1a400160b4df9cf307af2
$ git revert --no-commit 087e340fcd640342572f606e321f7348f7d020cc
Fusion automatique de taskcluster/gecko_taskgraph/decision.py

I can build and bootstrap again. Even nuking origin remote.

Comment 49

[Johan Lorenzo [:jlorenzo]]

(In reply to Alexandre LISSY :gerard-majax from comment #47)

<mozilla_repo_urls.platforms.hgmo.HgmoPlatform object at 0x7fb87b925090>). Expected: ('hgmo', 'github'). URL: https://git.mozilla.org/releases/gecko.git

Regarding this specific issue: per [1], git.mozilla.org was decommissioned 6 years ago (bug 1277297). Cloning it locally doesn't work anymore:

git clone https://git.mozilla.org/releases/gecko.git
Cloning into 'gecko'...
fatal: unable to update url base from redirection:
  asked for: https://git.mozilla.org/releases/gecko.git/info/refs?service=git-upload-pack
   redirect: https://wiki.mozilla.org/DeveloperServices/HistoricalVCS

[1] https://wiki.mozilla.org/DeveloperServices/HistoricalVCS

Comment 50

[Mike Hommey [:glandium]]

Taking a step back, why does taskgraph need the base revision? I get why it may need it in decision tasks on automation. But does it need it for the taskgraph usage in mach artifacts? (I don't think so) In mach taskgraph? (I'm not sure) In other things that might run locally? (are there any?)

Edit: of course I wrote this before reading comment 45...

We haven't found a better way to keep an enjoyable developer experience while using this new taskgraph param just on CI/CD.

taskcluster sets some environment variables (e.g. TASK_ID, IIRC) that won't be set outside of taskcluster.

Comment 51

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1784232 - part 3: Bump taskgraph to v3.2.1 r=jcristau

Comment 52

[:gerard-majax]

People might still need to change their origin remote to one that is supported after this lands ; my origin was set to the old git.mozilla.org and I would get failure due to it not supported. A git remote rename origin git-moz ; git remote rename unified origin fixed it for me.

mozilla_repo_urls.errors.UnsupportedPlatformError: Unsupported platform. Got: ('hgmo', <mozilla_repo_urls.platforms.hgmo.HgmoPlatform object at 0x7f3a50a1ba00>). Expected: ('hgmo', 'github'). URL: https://git.mozilla.org/releases/gecko.git

Comment 53

[Andrew Halberstadt [:ahal]]

People might still need to change their origin remote to one that is supported after this lands

Was this including the version bump from comment 51? My understanding is that should just set the default remote to an empty string when running mach locally as it wasn't technically needed for that use case anyway.

Comment 54

[Johan Lorenzo [:jlorenzo]]

(In reply to Alexandre LISSY :gerard-majax from comment #52)
(In reply to Andrew Halberstadt [:ahal] from comment #53)

Discussed with :gerard-majax in PM over Element. TL;DR: :jgilbert's patch[1] fixes this. It is included in taskgraph 3.2.1.

For more details: I got the bottom of this. It's regression caused by this change I suggested in taskgraph 3.x[2]. Since 2019[3], taskgraph looks at the local repository to get what the default base_repository and head_repository are. About a year ago, the logic was made more robust in bug 1753047, when people had no origin remote set. That is not directly related to the current bug, but that's a regression from taskgraph 3.x.

[1] https://github.com/taskcluster/taskgraph/pull/119
[2] https://github.com/taskcluster/taskgraph/pull/109#discussion_r966801770
[3] https://github.com/taskcluster/taskgraph/commit/5f5ec39b4e376575f20eb497a9ee8799d614f560

Comment 55

[Johan Lorenzo [:jlorenzo]]

I spent the last couple of hours to test comment 51. I think we're in a much better state this time. :whimboo imported the patch and immediately reported it fixed his setup.

On my end, I ran:

• mach bootstrap,
• mach configure,
• mach compile,
• mach artifact toolchain --from-build linux64-clang

All of these commands worked. I went one step further: I put a breakpoint to every single function of the GitRepository class[1] to intercept any call made to git. In each case, the calls were made to functions that:
a. existed before I started any work on this bug,
b. haven't changed since then,
c. were already called before.

In other words: I'm sure we're not trying to get a default branch or a base revision anymore. I'm going to land this patch.

[1] https://github.com/taskcluster/taskgraph/blob/58a7c503acde1021ccb044927d1b569945dd98fe/src/taskgraph/util/vcs.py#L318

Comment 56

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/50e2402e4632
part 3: Bump taskgraph to v3.2.1 r=jcristau

Comment 57

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/50e2402e4632