1791852 - `mach bootstrap`: mozilla_repo_urls.errors.InvalidRepoUrlError: Could not parse URL: [e-mail]:gecko1

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Kelsey Gilbert [:jgilbert]]

Would you like to run a few configuration steps to ensure Git is
optimally configured? (Yn): n
git: 'cinnabar' is not a git command. See 'git --help'.
Error running mach:

    ['artifact', 'toolchain', '--bootstrap', '--from-build', 'win64-clang-tidy']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file artifact| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

mozilla_repo_urls.errors.InvalidRepoUrlError: Could not parse URL: [e-mail]:gecko1

  File "C:\dev\mozilla\gecko2\python\mozbuild\mozbuild\artifact_commands.py", line 379, in artifact_toolchain
    tasks = toolchain_task_definitions()
  File "c:\dev\mozilla\gecko2\python\mozbuild\mozbuild\toolchains.py", line 19, in toolchain_task_definitions
    toolchains = load_tasks_for_kind(params, "toolchain", root_dir=root_dir)
  File "c:\dev\mozilla\gecko2\taskcluster\gecko_taskgraph\generator.py", line 450, in load_tasks_for_kind
    for task in tgg.full_task_set
  File "c:\dev\mozilla\gecko2\taskcluster\gecko_taskgraph\generator.py", line 158, in full_task_set
    return self._run_until("full_task_set")
  File "c:\dev\mozilla\gecko2\taskcluster\gecko_taskgraph\generator.py", line 426, in _run_until
    k, v = next(self._run)
  File "c:\dev\mozilla\gecko2\taskcluster\gecko_taskgraph\generator.py", line 262, in _run
    parameters = self._parameters(graph_config)
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 353, in get_parameters
    parameters = load_parameters_file(
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 304, in load_parameters_file
    return Parameters(strict=strict, repo_root=repo_root, **overrides)
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 153, in __init__
    kwargs = Parameters._fill_defaults(repo_root=repo_root, **kwargs)
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 193, in _fill_defaults
    defaults.update(fn(repo_root))
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 83, in _get_defaults
    parsed_url = parse(repo_url)
  File "c:\dev\mozilla\gecko2\third_party\python\mozilla_repo_urls\mozilla_repo_urls\parser.py", line 22, in parse
    raise InvalidRepoUrlError(url_string)
git: 'cinnabar' is not a git command. See 'git --help'.
Error running mach:

    ['bootstrap']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file bootstrap| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '['C:\\Users\\jgilbert\\.mozbuild\\srcdirs\\gecko2-720275d95bb8\\_virtualenvs\\mach\\Scripts\\python.exe', 'C:\\dev\\mozilla\\gecko2\\mach', 'artifact', 'toolchain', '--bootstrap', '--from-build', 'win64-clang-tidy']' returned non-zero exit status 1.

  File "C:\dev\mozilla\gecko2\python\mozboot\mozboot\mach_commands.py", line 43, in bootstrap
    bootstrapper.bootstrap(command_context.settings)
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\bootstrap.py", line 383, in bootstrap
    self.maybe_install_private_packages_or_exit(application)
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\bootstrap.py", line 257, in maybe_install_private_packages_or_exit
    self.instance.ensure_clang_static_analysis_package()
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\mozillabuild.py", line 230, in ensure_clang_static_analysis_package
    self.install_toolchain_static_analysis(static_analysis.WINDOWS_CLANG_TIDY)
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\base.py", line 392, in install_toolchain_static_analysis
    self.install_toolchain_artifact_impl(clang_tools_path, toolchain_job)
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\base.py", line 429, in install_toolchain_artifact_impl
    subprocess.check_call(cmd, cwd=str(install_dir))
  File "C:\dev\mozilla\mozilla-build-4.0.1\python3\lib\subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)

Bootstrap seems to expect me to be connected directly to a mozilla remote repo, but this machine is connected through my integration server:

[e-mail] -> [e-mail] (masked as as "example.com" above) -> hg::ssh://[e-mail]/mozilla-central/

Comment 1

[Kelsey Gilbert [:jgilbert]]

I tried the workaround in bug 1791797, but it doesn't work for me:

Would you like to run a few configuration steps to ensure Git is
optimally configured? (Yn): n
error: No such remote 'origin'
fatal: HEAD does not point to a branch
fatal: ambiguous argument 'github/HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
fatal: HEAD does not point to a branch
fatal: Not a valid object name github/warn-on-convert
error: No such remote 'origin'
fatal: HEAD does not point to a branch
fatal: ambiguous argument 'github/HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
fatal: HEAD does not point to a branch
fatal: Not a valid object name github/warn-on-convert
git: 'cinnabar' is not a git command. See 'git --help'.
Error running mach:

    ['artifact', 'toolchain', '--bootstrap', '--from-build', 'win64-clang-tidy']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file artifact| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '('C:\\Program Files\\Git\\cmd\\git.EXE', 'merge-base', 'github/warn-on-convert', 'b773445f2b306dea8002648500c8e3bc6fa670e4')' returned non-zero exit status 128.

  File "C:\dev\mozilla\gecko2\python\mozbuild\mozbuild\artifact_commands.py", line 379, in artifact_toolchain
    tasks = toolchain_task_definitions()
  File "c:\dev\mozilla\gecko2\python\mozbuild\mozbuild\toolchains.py", line 19, in toolchain_task_definitions
    toolchains = load_tasks_for_kind(params, "toolchain", root_dir=root_dir)
  File "c:\dev\mozilla\gecko2\taskcluster\gecko_taskgraph\generator.py", line 450, in load_tasks_for_kind
    for task in tgg.full_task_set
  File "c:\dev\mozilla\gecko2\taskcluster\gecko_taskgraph\generator.py", line 158, in full_task_set
    return self._run_until("full_task_set")
  File "c:\dev\mozilla\gecko2\taskcluster\gecko_taskgraph\generator.py", line 426, in _run_until
    k, v = next(self._run)
  File "c:\dev\mozilla\gecko2\taskcluster\gecko_taskgraph\generator.py", line 262, in _run
    parameters = self._parameters(graph_config)
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 353, in get_parameters
    parameters = load_parameters_file(
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 304, in load_parameters_file
    return Parameters(strict=strict, repo_root=repo_root, **overrides)
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 153, in __init__
    kwargs = Parameters._fill_defaults(repo_root=repo_root, **kwargs)
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 193, in _fill_defaults
    defaults.update(fn(repo_root))
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\parameters.py", line 94, in _get_defaults
    "base_rev": repo.find_latest_common_revision(default_base_ref, repo.head_rev),
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\util\vcs.py", line 485, in find_latest_common_revision
    return self.run("merge-base", base_ref_or_rev, head_rev).strip()
  File "c:\dev\mozilla\gecko2\third_party\python\taskcluster_taskgraph\taskgraph\util\vcs.py", line 45, in run
    return subprocess.check_output(
  File "C:\dev\mozilla\mozilla-build-4.0.1\python3\lib\subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\dev\mozilla\mozilla-build-4.0.1\python3\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
git: 'cinnabar' is not a git command. See 'git --help'.
Error running mach:

    ['bootstrap']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file bootstrap| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '['C:\\Users\\jgilbert\\.mozbuild\\srcdirs\\gecko2-720275d95bb8\\_virtualenvs\\mach\\Scripts\\python.exe', 'C:\\dev\\mozilla\\gecko2\\mach', 'artifact', 'toolchain', '--bootstrap', '--from-build', 'win64-clang-tidy']' returned non-zero exit status 1.

  File "C:\dev\mozilla\gecko2\python\mozboot\mozboot\mach_commands.py", line 43, in bootstrap
    bootstrapper.bootstrap(command_context.settings)
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\bootstrap.py", line 383, in bootstrap
    self.maybe_install_private_packages_or_exit(application)
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\bootstrap.py", line 257, in maybe_install_private_packages_or_exit
    self.instance.ensure_clang_static_analysis_package()
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\mozillabuild.py", line 230, in ensure_clang_static_analysis_package
    self.install_toolchain_static_analysis(static_analysis.WINDOWS_CLANG_TIDY)
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\base.py", line 392, in install_toolchain_static_analysis
    self.install_toolchain_artifact_impl(clang_tools_path, toolchain_job)
  File "c:\dev\mozilla\gecko2\python\mozboot\mozboot\base.py", line 429, in install_toolchain_artifact_impl
    subprocess.check_call(cmd, cwd=str(install_dir))
  File "C:\dev\mozilla\mozilla-build-4.0.1\python3\lib\subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)

Very weirdly it seems to be picking up the wrong remote (my github remote) and picking a (random?) branch on it while trying to limp along maybe?

Comment 2

[Kelsey Gilbert [:jgilbert]]

This fixes it for me: https://github.com/taskcluster/taskgraph/pull/119

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1784232

:jlorenzo, since you are the author of the regressor, bug 1784232, could you take a look? Also, could you set the severity field?

For more information, please visit auto_nag documentation.

Comment 4

[Johan Lorenzo [:jlorenzo]]

(In reply to Kelsey Gilbert [:jgilbert] from comment #2)

This fixes it for me: https://github.com/taskcluster/taskgraph/pull/119

PR landed in taskgraph. We now need to cut a new release and bump the python package in mozilla-central. I'd like to fix another thing before getting this bump landed. Let's keep the bug open.

Comment 5

[Johan Lorenzo [:jlorenzo]]

This patch made the taskgraph 3.2.1 release which made autoland in bug 1784232 comment 56. I'm going to close this bug. I'll reopen it if the bump somehow doesn't make mozilla-central.