1896978 - VCS-less bootstrap does not seem to work anymore

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Malte Jürgens]

The support for running mach bootstrap without being in a VCS checkout (introduced in Bug 1798746) seems to be broken.

To reproduce:

1. Download and extract https://hg.mozilla.org/mozilla-central/archive/tip.zip
2. Run ./mach bootstrap in the extracted folder and choose bootstrap action "2. Firefox for Desktop"

Expected result:
Toolchain artifacts should be fetched, system packages should be installed, etc.

Actual result:
The bootstrap almost instantly finishes without doing anything.

While investigating I noticed that running ./mach artifact toolchain --from-build linux64-clang directly to fetch a toolchain artifact results in the following error:

TypeError: 'NoneType' object is not iterable

  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/python/mozbuild/mozbuild/artifact_commands.py", line 393, in artifact_toolchain
    tasks = toolchain_task_definitions()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/python/mozbuild/mozbuild/toolchains.py", line 18, in toolchain_task_definitions
    toolchains = load_tasks_for_kind(params, "toolchain", root_dir=root_dir)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/third_party/python/taskcluster_taskgraph/taskgraph/generator.py", line 449, in load_tasks_for_kind
    for task in tgg.full_task_set
                ^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/third_party/python/taskcluster_taskgraph/taskgraph/generator.py", line 160, in full_task_set
    return self._run_until("full_task_set")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/third_party/python/taskcluster_taskgraph/taskgraph/generator.py", line 425, in _run_until
    k, v = next(self._run)
           ^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/third_party/python/taskcluster_taskgraph/taskgraph/generator.py", line 264, in _run
    parameters = self._parameters(graph_config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/third_party/python/taskcluster_taskgraph/taskgraph/parameters.py", line 369, in get_parameters
    parameters = load_parameters_file(
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/third_party/python/taskcluster_taskgraph/taskgraph/parameters.py", line 320, in load_parameters_file
    return Parameters(strict=strict, repo_root=repo_root, **overrides)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/third_party/python/taskcluster_taskgraph/taskgraph/parameters.py", line 169, in __init__
    kwargs = Parameters._fill_defaults(repo_root=repo_root, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/third_party/python/taskcluster_taskgraph/taskgraph/parameters.py", line 209, in _fill_defaults
    defaults.update(fn(repo_root))
                    ^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/taskcluster/gecko_taskgraph/parameters.py", line 112, in get_defaults
    "files_changed": sorted(get_locally_changed_files(repo_root)),
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/python/mozbuild/mozbuild/util.py", line 902, in __call__
    self[args] = self.func(*args)
                 ^^^^^^^^^^^^^^^^
  File "/home/maltejur/Downloads/mozilla-central-fd46f1fdb469/taskcluster/gecko_taskgraph/files_changed.py", line 81, in get_locally_changed_files
    return set(vcs.get_outgoing_files("AM"))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

What seems to be happening is the get_outgoing_files stub of SrcRepository being called, and it seems the changes from Bug 1884364 seem to have caused this to happen. The easiest fix is probably just to return an empty list of "changed" and "outgoing" files for the SrcRepository instead of nothing.

Comment 1

[Malte Jürgens]

Attachment: Bug 1896978 - Return empty list of changed files instead of nothing in SrcRepository r?ahochheiden!

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1884364

Comment 3

[Liz Henry (:lizzard) (relman/hg->git project)]

Malte, do you want to request uplift to beta?

Comment 4

[Malte Jürgens]

Yes, that probably makes sense. I will request a uplift once the patch has landed.

Comment 5

[Malte Jürgens]

Matthew, your changes from Bug 1898777 conflict with my patch. Do you think it would be okay for get_outgoing_files to just return an empty array instead of an error? Because that seems to be required for ./mach artifact toolchain --from-build to function properly.

Comment 6

[Matthew Gaudet (he/him) [:mgaudet]]

Absolutely your patch is fine! I was just basing mine on the function signature, but returning empty is going to be fine for my uses too.

Comment 7

[Malte Jürgens]

Perfect, thanks

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/49ebcfb18a9c
Return empty list of changed files instead of nothing in SrcRepository r=ahochheiden

Comment 9

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/49ebcfb18a9c