1723304 - ./mach configure failure for gecko 91:

Status: RESOLVED DUPLICATE of bug 1717645

(Firefox Build System :: Mach Core, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Running mach configure after mach clobber for the latest mozilla-beta (91) fails here on Windows 8.1. The same actions on mozilla-central succeed.

Beta revision is https://hg.mozilla.org/releases/mozilla-beta/rev/6f75d7405224ee9083948faa064fa225900d3b35

$ mach configure
 0:00.79 c:\Users\Username\.mozbuild\_virtualenvs\mach\Scripts\python.exe c:
/Mozilla/Coding/Code/mozilla-beta\configure.py
 0:01.03 Re-executing in the virtualenv
 0:01.27 checking for vcs source checkout... hg
 0:01.33 checking for a shell... c:/Mozilla/Coding/Buildtools/MozillaBuild/msys/
bin/sh.exe
 0:01.33 checking for host system type... x86_64-pc-mingw32
 0:01.33 checking for target system type... x86_64-pc-mingw32
 0:01.87 checking whether cross compiling... no
 0:02.04 checking for Python 3... c:/Mozilla/Coding/Code/mozilla-beta/obj-x86_64
-pc-mingw32/_virtualenvs/common/Scripts/python.exe (3.7.4)
 0:02.04 checking for hg... c:/Mozilla/Coding/Buildtools/MozillaBuild/python/Scr
ipts/hg.exe
 0:02.20 checking for Mercurial version... 5.1.2
 0:02.42 Traceback (most recent call last):
 0:02.42   File "c:/Mozilla/Coding/Code/mozilla-beta\configure.py", line 231, in
 <module>
 0:02.42     sys.exit(main(sys.argv))
 0:02.42   File "c:/Mozilla/Coding/Code/mozilla-beta\configure.py", line 55, in
main
 0:02.42     sandbox.run(os.path.join(os.path.dirname(__file__), "moz.configure"
))
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\co
nfigure\__init__.py", line 563, in run
 0:02.42     func(*args)
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\co
nfigure\__init__.py", line 614, in _value_for
 0:02.42     return self._value_for_depends(obj)
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\ut
il.py", line 1050, in method_call
 0:02.42     cache[args] = self.func(instance, *args)
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\co
nfigure\__init__.py", line 623, in _value_for_depends
 0:02.42     value = obj.result()
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\ut
il.py", line 1050, in method_call
 0:02.42     cache[args] = self.func(instance, *args)
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\co
nfigure\__init__.py", line 162, in result
 0:02.42     return self._func(*resolved_args)
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\co
nfigure\__init__.py", line 1297, in wrapped
 0:02.42     self._apply_imports(func, glob)
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\co
nfigure\__init__.py", line 958, in _apply_imports
 0:02.42     self._get_one_import(_from, _import, _as, glob)
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\co
nfigure\__init__.py", line 1095, in _get_one_import
 0:02.42     exec_(import_line, {}, glob)
 0:02.42   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\ut
il.py", line 56, in exec_
 0:02.42     exec(object, globals, locals)
 0:02.42   File "<string>", line 1, in <module>
 0:02.42 ModuleNotFoundError: No module named 'mozversioncontrol'
Error running mach:

    ['configure']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file configure| to report it. If |./mach busted
| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Process executed with non-0 exit code 1: ['c:\\Users\\Username\\.
mozbuild\\_virtualenvs\\mach\\Scripts\\python.exe', 'c:/Mozilla/Coding/Code/mozi
lla-beta\\configure.py']

  File "c:\Mozilla\Coding\Code\mozilla-beta\python/mozbuild/mozbuild/build_comma
nds.py", line 195, in configure
    line_handler=line_handler,
  File "c:\Mozilla\Coding\Code\mozilla-beta\python/mozbuild\mozbuild\controller\
building.py", line 1531, in configure
    append_env=append_env,
  File "c:\Mozilla\Coding\Code\mozilla-beta\python/mozbuild\mozbuild\base.py", l
ine 845, in _run_command_in_objdir
    return self.run_process(cwd=self.topobjdir, **args)
  File "c:\Mozilla\Coding\Code\mozilla-beta\python/mach\mach\mixin\process.py",
line 177, in run_process
    "Process executed with non-0 exit code %d: %s" % (status, args)

A previous mach configure run in a ConEmu console had failed with

$ mach configure
 0:01.15 c:\Users\Username\.mozbuild\_virtualenvs\mach\Scripts\python.exe c:/Mozilla/Coding/Code/mozilla-beta\configure.py
 0:01.45 Creating Python 3 environment
 0:03.70 created virtual environment CPython3.7.4.final.0-64 in 1516ms
 0:03.70   creator CPython3Windows(dest=C:\Mozilla\Coding\Code\mozilla-beta\obj-x86_64-pc-mingw32\_virtualenvs\common, clear=False, no_vcs_ignore=False, global=False)
 0:03.70   seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:\Users\Username\AppData\Local\pypa\virtualenv)
 0:03.70     added seed packages: pip==20.3.1, setuptools==51.0.0, wheel==0.36.1
 0:03.70   activators BashActivator,BatchActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
 0:03.98 Traceback (most recent call last):
 0:03.98   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 698, in <module>
 0:03.98     manager.populate()
 0:03.98   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 386, in populate
 0:03.98     handle_package(current_action, current_package)
 0:03.98   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 322, in handle_package
 0:03.98     submanager.populate()
 0:03.98   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 386, in populate
 0:03.98     handle_package(current_action, current_package)
 0:03.98   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 337, in handle_package
 0:03.98     handle_package(*package.split(":", maxsplit=1))
 0:03.98   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 322, in handle_package
 0:03.98     submanager.populate()
 0:03.98   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 386, in populate
 0:03.98     handle_package(current_action, current_package)
 0:03.98   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 346, in handle_package
 0:03.98     raise Exception("Unknown action: %s" % action)
 0:03.98 Exception: Unknown action: comm.pth
 0:04.00 Traceback (most recent call last):
 0:04.00   File "c:/Mozilla/Coding/Code/mozilla-beta\configure.py", line 231, in <module>
 0:04.00     sys.exit(main(sys.argv))
 0:04.00   File "c:/Mozilla/Coding/Code/mozilla-beta\configure.py", line 55, in main
 0:04.00     sandbox.run(os.path.join(os.path.dirname(__file__), "moz.configure"))
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 502, in run
 0:04.00     self.include_file(path)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 493, in include_file
 0:04.00     exec_(code, self)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\util.py", line 56, in exec_
 0:04.00     exec(object, globals, locals)
 0:04.00   File "c:/Mozilla/Coding/Code/mozilla-beta/moz.configure", line 7, in <module>
 0:04.00     include("build/moz.configure/init.configure")
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 853, in include_impl
 0:04.00     self.include_file(what)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 493, in include_file
 0:04.00     exec_(code, self)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\util.py", line 56, in exec_
 0:04.00     exec(object, globals, locals)
 0:04.00   File "c:/Mozilla/Coding/Code/mozilla-beta/build/moz.configure/init.configure", line 233, in <module>
 0:04.00     @imports(_from="six", _import="ensure_text")
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 836, in decorator
 0:04.00     depends = DependsFunction(self, func, dependencies, when=when)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 137, in __init__

 0:04.00     sandbox._value_for(self)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 614, in _value_for
 0:04.00     return self._value_for_depends(obj)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\util.py", line 1050, in method_call
 0:04.00     cache[args] = self.func(instance, *args)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 623, in _value_for_depends
 0:04.00     value = obj.result()
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\util.py", line 1050, in method_call
 0:04.00     cache[args] = self.func(instance, *args)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 162, in result
 0:04.00     return self._func(*resolved_args)
 0:04.00   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\configure\__init__.py", line 1298, in wrapped

 0:04.00     return new_func(*args, **kwargs)
 0:04.01   File "c:/Mozilla/Coding/Code/mozilla-beta/build/moz.configure/init.configure", line 345, in virtualenv_python3
 0:04.01     manager.build(python)
 0:04.01   File "c:\Mozilla\Coding\Code\mozilla-beta\python\mozbuild\mozbuild\virtualenv.py", line 462, in build
 0:04.01     raise Exception("Error populating virtualenv.")
 0:04.01 Exception: Error populating virtualenv.
Error running mach:

    ['configure']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file configure| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Process executed with non-0 exit code 1: ['c:\\Users\\Username\\.mozbuild\\_virtualenvs\\mach\\Scripts\\python.exe', 'c:/Mozilla/Coding/Code/mozilla-beta\\configure.py']

  File "c:\Mozilla\Coding\Code\mozilla-beta\python/mozbuild/mozbuild/build_commands.py", line 195, in configure
    line_handler=line_handler,
  File "c:\Mozilla\Coding\Code\mozilla-beta\python/mozbuild\mozbuild\controller\building.py", line 1531, in configure
    append_env=append_env,
  File "c:\Mozilla\Coding\Code\mozilla-beta\python/mozbuild\mozbuild\base.py", line 845, in _run_command_in_objdir
    return self.run_process(cwd=self.topobjdir, **args)
  File "c:\Mozilla\Coding\Code\mozilla-beta\python/mach\mach\mixin\process.py", line 177, in run_process
    "Process executed with non-0 exit code %d: %s" % (status, args)```

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

Workaround:

1. ./mach create-mach-environment -f
2. ./mach configure

---

This has happened because:

1. We have a bug where different source trees can "leak into" each other due to the shared "mach" virtualenv that isn't kept up-to-date. That will be fixed by this patch. In the future, we shouldn't share "mach" virtualenvs between source trees.
2. When creating the "configure/build" virtualenv, we fail because we're using your central checkout's virtualenv code, but parsing your beta checkout's virtualenv dependency file (the format recently changed)
3. Despite the virtualenv failing to be created properly, ./mach configure still attempts to run, but raises that mozversioncontrol error because it doesn't have the necessary paths in its virtualenv.

Marking as a dupe, because it'll be solved by that other bug + patch.
Please re-open if the workaround doesn't work!