1730811 - unable to `hg commit`

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[Aki Sasaki (not active)]

I'm trying to commit changes to a project branch to fix bustage.
When I hg commit -m "readd m-c merge", for some reason I'm running mach to install a virtualenv and get the following messages:

 0:14.85 checking for wget... /usr/local/bin/wget
 0:14.89 Updating bootstrapped toolchain in /Users/asasaki/.mozbuild/clang
 0:23.10 ERROR!!!!!! Could not find artifacts for a toolchain build named `toolchain-macosx64-clang`. Local commits, dirty/stale files, and other changes in your checkout may cause this error. Make sure you are on a fresh, current checkout of mozilla-central. Beware that commands like `mach bootstrap` and `mach artifact` are unlikely to work on any versions of the code besides recent revisions of mozilla-central.
 4:16.16 Hint: consider reverting your local changes to the following files: <snip due to bz limits>
 4:23.26 Due to the environment variable TASKCLUSTER_ROOT_URL being set, the artifacts were expected to be found on https://firefox-ci-tc.services.mozilla.com/. If this was unintended, unset TASKCLUSTER_ROOT_URL and try again.
 4:23.31 ERROR: Command '['/bin/sh', '/Users/asasaki/src/gecko/mozilla-unified/mach', '--log-no-times', 'artifact', 'toolchain', '--from-build', 'toolchain-macosx64-clang']' returned non-zero exit status 1.
 4:23.31 ERROR: If you can't fix the above, retry with --disable-bootstrap.
Error running mach:

    ['clang-format', '-p', <snip due to bz limits>]

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file clang-format| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Process executed with non-0 exit code 1: ['/Users/asasaki/.mozbuild/_virtualenvs/mach/bin/python', '/Users/asasaki/src/gecko/mozilla-unified/configure.py']

  File "/Users/asasaki/src/gecko/mozilla-unified/python/mozbuild/mozbuild/code_analysis/mach_commands.py", line 2473, in clang_format
    rc, clang_paths = self._get_clang_tools(command_context, verbose=verbose)
  File "/Users/asasaki/src/gecko/mozilla-unified/python/mozbuild/mozbuild/code_analysis/mach_commands.py", line 2799, in _get_clang_tools
    rc, clang_paths = self._set_clang_tools_paths(command_context)
  File "/Users/asasaki/src/gecko/mozilla-unified/python/mozbuild/mozbuild/code_analysis/mach_commands.py", line 2737, in _set_clang_tools_paths
    rc, config, _ = self._get_config_environment(command_context)
  File "/Users/asasaki/src/gecko/mozilla-unified/python/mozbuild/mozbuild/code_analysis/mach_commands.py", line 2662, in _get_config_environment
    rc = builder.configure(command_context)
  File "/Users/asasaki/src/gecko/mozilla-unified/python/mozbuild/mozbuild/build_commands.py", line 261, in configure
    line_handler=line_handler,
  File "/Users/asasaki/src/gecko/mozilla-unified/python/mozbuild/mozbuild/controller/building.py", line 1529, in configure
    append_env=append_env,
  File "/Users/asasaki/src/gecko/mozilla-unified/python/mozbuild/mozbuild/base.py", line 853, in _run_command_in_objdir
    return self.run_process(cwd=self.topobjdir, **args)
  File "/Users/asasaki/src/gecko/mozilla-unified/python/mach/mach/mixin/process.py", line 175, in run_process
    "Process executed with non-0 exit code %d: %s" % (status, args)

Ah, looks like the commit finishes, just takes a long time.

• The comment about TASKCLUSTER_ROOT_URL seems a bit concerning; I usually have this set. Should we be looking at MOZ_AUTOMATION instead?
• I seem to get the error about "Could not find artifacts for a toolchain build named toolchain-macosx64-clang." even if I'm on a recent revision of m-c.
• The comment about backing out my changes seems poor, since I'm trying to commit the changes.
• How do I prevent mach from running at all when I'm running hg commands? I want to commit changes that already went through review and I'm pushing to a project branch.

Comment 1

[Mike Hommey [:glandium]]

for some reason I'm running mach to install a virtualenv

You have a commit hook that is trying to run clang-format, which runs configure. Bug 1727255 will eventually avoid the configure.

The comment about TASKCLUSTER_ROOT_URL seems a bit concerning; I usually have this set.

The comment about TASKCLUSTER_ROOT_URL is that if you have a value set that is not what is normally expected, looking at the index there may not yield the toolchains you would need. This has nothing to do with MOZ_AUTOMATION. We could change the logic to only show the message when it's set to a different value than the default.

I seem to get the error about "Could not find artifacts for a toolchain build named toolchain-macosx64-clang." even if I'm on a recent revision of m-c.

You're presumably on a recent version of a project branch. Are there toolchain changes in the project branch? Are there toolchain changes in what you're committing? (Any change that can affect index hashes, actually)

How do I prevent mach from running at all when I'm running hg commands? I want to commit changes that already went through review and I'm pushing to a project branch.

Disable your commit hooks. But I'd advise checking why it can't find the toolchain.

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

But I'd advise checking why it can't find the toolchain.

It identifies the toolchain version to download based on the contents of local files:

Local commits, dirty/stale files, and other changes in your checkout may cause this error. ... Hint: consider reverting your local changes to the following files: <snip due to bz limits>

If you hop onto a clean checkout of central, you should be able to run ./mach clang-format or ./mach configure so that your system has the toolchain pieces it needs. Then, when you unshelve your changes and try to commit again, it should use the locally-downloaded one.

Comment 3

[Connor Sheehan [:sheehan]]

(In reply to Aki Sasaki [:aki] (he/him) (UTC-6) from comment #0)

• How do I prevent mach from running at all when I'm running hg commands? I want to commit changes that already went through review and I'm pushing to a project branch.

Deactivate the js-format and clang-format hg extensions in your hgrc file. You can open the file with hg config -e.

Comment 4

[Aki Sasaki (not active)]

Awesome, thanks all!

(In reply to Mike Hommey [:glandium] from comment #1)

for some reason I'm running mach to install a virtualenv

You have a commit hook that is trying to run clang-format, which runs configure. Bug 1727255 will eventually avoid the configure.

Sounds good.

(In reply to Mitchell Hentges [:mhentges] 🦀 from comment #2)

But I'd advise checking why it can't find the toolchain.

It identifies the toolchain version to download based on the contents of local files:

Local commits, dirty/stale files, and other changes in your checkout may cause this error. ... Hint: consider reverting your local changes to the following files: <snip due to bz limits>

If you hop onto a clean checkout of central, you should be able to run ./mach clang-format or ./mach configure so that your system has the toolchain pieces it needs. Then, when you unshelve your changes and try to commit again, it should use the locally-downloaded one.

Appears to work; I think I had to ./mach bootstrap to get the right version of clang-format.

(In reply to Connor Sheehan [:sheehan] from comment #3)

(In reply to Aki Sasaki [:aki] (he/him) (UTC-6) from comment #0)

• How do I prevent mach from running at all when I'm running hg commands? I want to commit changes that already went through review and I'm pushing to a project branch.

Deactivate the js-format and clang-format hg extensions in your hgrc file. You can open the file with hg config -e.

Perfect, this was what I was hoping for. Thank you!

Resolving, though I'm open to reopening if we want to use it to track adjusting any of the above.