1669701 - mozversioncontrol.MissingVCSTool when running ./mach bootstrap

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Valentin Gosu [:valentin] (he/him) {{ AFK in July }}]

I reinstalled my system (Ubuntu 20.04.1) , did a ./mach bootstrap which failed with

./mach bootstrap
Error running mach:

    ['bootstrap']

The error occurred in mach itself. This is likely a bug in mach itself or a
fundamental problem with a loaded module.

You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file general| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

mozversioncontrol.MissingVCSTool: Unable to obtain hg path. Try running |mach bootstrap| to ensure your environment is up to date.

  File "/home/icecold/mozilla-unified/python/mach/mach/main.py", line 342, in run
    sentry = register_sentry(argv, self.settings, topsrcdir)
  File "/home/icecold/mozilla-unified/python/mach/mach/sentry.py", line 57, in register_sentry
    repo = get_repository_object(topsrcdir)
  File "/home/icecold/mozilla-unified/python/mozversioncontrol/mozversioncontrol/__init__.py", line 609, in get_repository_object
    return HgRepository(path, hg=hg)
  File "/home/icecold/mozilla-unified/python/mozversioncontrol/mozversioncontrol/__init__.py", line 285, in __init__
    super(HgRepository, self).__init__(path, tool=hg)
  File "/home/icecold/mozilla-unified/python/mozversioncontrol/mozversioncontrol/__init__.py", line 87, in __init__
    self._tool = get_tool_path(tool)
  File "/home/icecold/mozilla-unified/python/mozversioncontrol/mozversioncontrol/__init__.py", line 61, in get_tool_path
    raise MissingVCSTool('Unable to obtain %s path. Try running '

I installed mercurial:

icecold@icecold-xps:~/mozilla-unified$ hg --version
Mercurial Distributed SCM (version 4.9.1)
(see https://mercurial-scm.org for more information)

Copyright (C) 2005-2019 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
icecold@icecold-xps:~/mozilla-unified$ which hg
/usr/bin/hg

and tried doing mach bootstrap again.

Error running mach:

    ['artifact', 'toolchain', '--bootstrap', '--from-build', 'linux64-node']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file artifact| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

mozversioncontrol.MissingVCSTool: Unable to obtain /usr/local/bin/hg path. Try running |mach bootstrap| to ensure your environment is up to date.

  File "/home/icecold/mozilla-unified/python/mozbuild/mozbuild/artifact_commands.py", line 311, in artifact_toolchain
    toolchains = load_tasks_for_kind(params, 'toolchain', root_dir=root_dir)
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/generator.py", line 432, in load_tasks_for_kind
    for task in tgg.full_task_set
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/generator.py", line 154, in full_task_set
    return self._run_until('full_task_set')
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/generator.py", line 376, in _run_until
    k, v = next(self._run)
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/generator.py", line 279, in _run
    new_tasks = kind.load_tasks(
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/generator.py", line 72, in load_tasks
    tasks = [Task(self.name,
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/generator.py", line 72, in <listcomp>
    tasks = [Task(self.name,
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1988, in check_run_task_caches
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1906, in check_task_dependencies
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1894, in check_task_identifiers
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1877, in chain_of_trust
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1698, in build_task
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1689, in try_task_config_routes
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1679, in try_task_config_chemspill_prio
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1669, in try_task_config_env
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1634, in add_index_routes
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1439, in validate
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1416, in task_name_from_label
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1376, in set_defaults
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/task.py", line 1355, in set_implementation
    for task in tasks:
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/job/__init__.py", line 357, in make_task_description
    import_sibling_modules(exceptions=('common.py',))
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/util/python_path.py", line 55, in import_sibling_modules
    __import__(modpath + '.' + f[:-3])
  File "/home/icecold/mozilla-unified/build/mach_bootstrap.py", line 504, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/job/mozharness_test.py", line 22, in <module>
    from taskgraph.transforms.tests import (
  File "/home/icecold/mozilla-unified/build/mach_bootstrap.py", line 504, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/transforms/tests.py", line 48, in <module>
    from taskgraph.util.chunking import (
  File "/home/icecold/mozilla-unified/build/mach_bootstrap.py", line 504, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/icecold/mozilla-unified/taskcluster/taskgraph/util/chunking.py", line 29, in <module>
    resolver = TestResolver.from_environment(cwd=here, loader_cls=TestManifestLoader)
  File "/home/icecold/mozilla-unified/python/mozbuild/mozbuild/base.py", line 207, in from_environment
    return cls(topsrcdir, None, None, topobjdir=topobjdir,
  File "/home/icecold/mozilla-unified/testing/mozbase/moztest/moztest/resolve.py", line 502, in __init__
    self.load_tests = self._spawn(loader_cls)
  File "/home/icecold/mozilla-unified/python/mozbuild/mozbuild/base.py", line 832, in _spawn
    return cls(self.topsrcdir, self.settings, self.log_manager,
  File "/home/icecold/mozilla-unified/testing/mozbase/moztest/moztest/resolve.py", line 442, in __init__
    self.reader = self.mozbuild_reader(config_mode="empty")
  File "/home/icecold/mozilla-unified/python/mozbuild/mozbuild/base.py", line 474, in mozbuild_reader
    repo = self.repository
  File "/home/icecold/mozilla-unified/python/mozbuild/mozbuild/util.py", line 1040, in __get__
    setattr(instance, name, self.func(instance))
  File "/home/icecold/mozilla-unified/python/mozbuild/mozbuild/base.py", line 419, in repository
    return get_repository_from_build_config(self)
  File "/home/icecold/mozilla-unified/python/mozversioncontrol/mozversioncontrol/__init__.py", line 633, in get_repository_from_build_config
    return HgRepository(config.topsrcdir, hg=config.substs['HG'])
  File "/home/icecold/mozilla-unified/python/mozversioncontrol/mozversioncontrol/__init__.py", line 285, in __init__
    super(HgRepository, self).__init__(path, tool=hg)
  File "/home/icecold/mozilla-unified/python/mozversioncontrol/mozversioncontrol/__init__.py", line 87, in __init__
    self._tool = get_tool_path(tool)
  File "/home/icecold/mozilla-unified/python/mozversioncontrol/mozversioncontrol/__init__.py", line 61, in get_tool_path
    raise MissingVCSTool('Unable to obtain %s path. Try running '
Sentry is attempting to send 0 pending error messages

Eventually I just did a ln -s /usr/bin/hg /usr/local/bin/hg and that fixed it.

Comment 1

[Valentin Gosu [:valentin] (he/him) {{ AFK in July }}]

Note that without having mercurial installed, ./mach busted file general also failed.

Comment 2

[Ricky Stewart]

There are 3 issues described here: 1) mach bootstrap failed when you didn't have hg installed; 2) mach bootstrap failed again in a different way after you installed hg; and 3) ./mach busted file general fails without having hg installed.

RE: issue 1, I assume your checkout is a Mercurial checkout? mach bootstrap only works for recent checkouts of mozilla-central, and yes, that means you'll have to have your VCS installed in order to run mach bootstrap. So as stated this is a WONTFIX.

RE: issue 2, this sounds consistent with other issues we've seen where people have (unknowingly) configured a certain version of hg or git which breaks when something changes on their machine. What I assume happened is that you configured a certain version of hg before re-installing your OS, which lived at /usr/local/bin/hg, and after the re-install that stopped being true. You should make sure your mozconfig doesn't set the HG variable, and that you don't have an HG environment variable set; then, re-running mach configure (maybe after a clobber) should fix the issue.

RE: issue 3, that may or may not be a problem depending on how the issue was manifesting. I would have to see a stacktrace.

Comment 3

[Ricky Stewart]

Never mind, I spoke too soon. Looking closer at the stacktrace, there is actually a problem here: Sentry doesn't allow get_repository_object() to fail with a MissingVCSTool exception, which would explain issues (1) and (3) above. We can fix that.

Comment 4

[Ricky Stewart]

Attachment: Bug 1669701 - Catch `MissingVCSTool` when attempting to register Sentry

Without this, running ANY mach command can fail if you don't have a VCS installed, which isn't the right behavior.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/858807a24633
Catch `MissingVCSTool` when attempting to register Sentry r=firefox-build-system-reviewers,mhentges

Comment 6

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/858807a24633