1925399 - `./mach vendor python` fails when using `git` as the VCS

Status: RESOLVED FIXED

(Developer Infrastructure :: Mach Vendor & Updatebot, defect, P3)

Description

[Alex Hochheiden [:ahochheiden]]

It seems that the changes in D208968 were not tested with git. There's a missing -f flag needed when adding files in git that are captured by the .gitignore file. Since it's not there, the ./mach vendor python command fails to complete successfully and does not add the files captured by the .gitignore to VCS even though that's what we want. The early failure also prevents the uv.lock.hash file from being generated.

The following paths are ignored by one of your .gitignore files:
third_party/python/MarkupSafe/src/MarkupSafe.egg-info
third_party/python/PyYAML/lib/PyYAML.egg-info
third_party/python/aiohttp/aiohttp.egg-info
third_party/python/esprima/esprima.egg-info
third_party/python/jsmin/jsmin.egg-info
third_party/python/ply/ply.egg-info
Error running mach:

    mach vendor python -P=taskcluster_taskgraph

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke ``./mach busted`` to check if this issue is already on file. If it
isn't, please use ``./mach busted file vendor`` to report it. If ``./mach busted`` is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '('C:\\Program Files\\Git\\cmd\\git.EXE', 'add', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\aiohttp\\aiohttp.egg-info\\dependency_links.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\aiohttp\\aiohttp.egg-info\\not-zip-safe', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\aiohttp\\aiohttp.egg-info\\PKG-INFO', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\aiohttp\\aiohttp.egg-info\\requires.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\aiohttp\\aiohttp.egg-info\\SOURCES.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\aiohttp\\aiohttp.egg-info\\top_level.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\esprima\\esprima.egg-info\\dependency_links.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\esprima\\esprima.egg-info\\entry_points.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\esprima\\esprima.egg-info\\pbr.json', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\esprima\\esprima.egg-info\\PKG-INFO', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\esprima\\esprima.egg-info\\SOURCES.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\esprima\\esprima.egg-info\\top_level.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\jsmin\\jsmin.egg-info\\dependency_links.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\jsmin\\jsmin.egg-info\\PKG-INFO', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\jsmin\\jsmin.egg-info\\SOURCES.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\jsmin\\jsmin.egg-info\\top_level.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\MarkupSafe\\src\\MarkupSafe.egg-info\\dependency_links.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\MarkupSafe\\src\\MarkupSafe.egg-info\\PKG-INFO', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\MarkupSafe\\src\\MarkupSafe.egg-info\\SOURCES.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\MarkupSafe\\src\\MarkupSafe.egg-info\\top_level.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\ply\\ply.egg-info\\dependency_links.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\ply\\ply.egg-info\\PKG-INFO', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\ply\\ply.egg-info\\SOURCES.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\ply\\ply.egg-info\\top_level.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\PyYAML\\lib\\PyYAML.egg-info\\dependency_links.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\PyYAML\\lib\\PyYAML.egg-info\\PKG-INFO', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\PyYAML\\lib\\PyYAML.egg-info\\SOURCES.txt', 'D:\\mozilla-source\\mozilla-unified-git\\third_party\\python\\PyYAML\\lib\\PyYAML.egg-info\\top_level.txt')' returned non-zero exit status 1.

  File "D:\mozilla-source\mozilla-unified-git\python\mozbuild\mozbuild\vendor\mach_commands.py", line 274, in vendor_python
    vendor_command.vendor(
  File "d:\mozilla-source\mozilla-unified-git\python\mozbuild\mozbuild\vendor\vendor_python.py", line 156, in vendor
    self.repository.add_remove_files(*egg_info_files)
  File "d:\mozilla-source\mozilla-unified-git\python\mozversioncontrol\mozversioncontrol\__init__.py", line 883, in add_remove_files
    self._run(*cmd)
  File "d:\mozilla-source\mozilla-unified-git\python\mozversioncontrol\mozversioncontrol\__init__.py", line 115, in _run
    return subprocess.check_output(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\mozilla-build\python3\Lib\subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\mozilla-build\python3\Lib\subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,

Comment 1

[Alex Hochheiden [:ahochheiden]]

Attachment: Bug 1925399 - Add a `force` flag to `add_remove_files` in `mozversioncontrol` r?sheehan

Git requires a -f flag to force adding files that are captured by the
.gitignore (hg does not need this). We add the force flag to all
add_remove_files to allow us to append that flag when running the
GitRepository variant. They are unused for all other variants, but we
need to maintain a consistent function signature.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1894160

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4eb92d9c8d78
Add a `force` flag to `add_remove_files` in `mozversioncontrol` r=sheehan

Comment 4

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/4eb92d9c8d78

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:ahochheiden, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox132 to wontfix.

For more information, please visit BugBot documentation.