1684234 - Failed to populate virtualenv on Windows when MOZ_OBJDIR is pointing to a different drive

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P3)

Description

[Xidorn Quan [:xidorn] UTC+10]

My src is in C:, but objdir is in D: (because C: doesn't have enough space, and it can't be resized easily), then I have got this:

 0:01.38 c:\Users\upsuper\.mozbuild\_virtualenvs\mach\Scripts\python.exe c:/Users/upsuper/Sources/mozilla-central\configure.py
 0:01.70 Creating Python 3 environment
 0:06.32 created virtual environment CPython3.7.4.final.0-64 in 4270ms
 0:06.32   creator CPython3Windows(dest=D:\firefox-objdir\debug\_virtualenvs\init_py3, clear=False, global=False)
 0:06.32   seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:\Users\upsuper\AppData\Local\pypa\virtualenv)
 0:06.32     added seed packages: pip==20.2.2, setuptools==49.6.0, wheel==0.35.1
 0:06.32   activators BashActivator,BatchActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
 0:06.46 Traceback (most recent call last):
 0:06.48   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\virtualenv.py", line 774, in <module>
 0:06.48     manager.populate()
 0:06.49   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\virtualenv.py", line 466, in populate
 0:06.49     handle_package(package)
 0:06.49   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\virtualenv.py", line 381, in handle_package
 0:06.49     submanager.populate(sitecustomize=sitecustomize)
 0:06.49   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\virtualenv.py", line 466, in populate
 0:06.49     handle_package(package)
 0:06.49   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\virtualenv.py", line 398, in handle_package
 0:06.50     f.write("%s\n" % os.path.relpath(path, python_lib))
 0:06.50   File "c:\mozilla-build\python3\lib\ntpath.py", line 562, in relpath
 0:06.50     path_drive, start_drive))
 0:06.50 ValueError: path is on mount 'c:', start on mount 'D:'
 0:06.50 Traceback (most recent call last):
 0:06.50   File "c:/Users/upsuper/Sources/mozilla-central\configure.py", line 231, in <module>
 0:06.52     sys.exit(main(sys.argv))
 0:06.52   File "c:/Users/upsuper/Sources/mozilla-central\configure.py", line 55, in main
 0:06.52     sandbox.run(os.path.join(os.path.dirname(__file__), "moz.configure"))
 0:06.52   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 501, in run
 0:06.52     self.include_file(path)
 0:06.52   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 492, in include_file
 0:06.52     exec_(code, self)
 0:06.52   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\util.py", line 55, in exec_
 0:06.53     exec(object, globals, locals)
 0:06.53   File "c:/Users/upsuper/Sources/mozilla-central/moz.configure", line 7, in <module>
 0:06.53     include("build/moz.configure/init.configure")
 0:06.53   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 852, in include_impl
 0:06.53     self.include_file(what)
 0:06.53   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 492, in include_file
 0:06.53     exec_(code, self)
 0:06.53   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\util.py", line 55, in exec_
 0:06.55     exec(object, globals, locals)
 0:06.55   File "c:/Users/upsuper/Sources/mozilla-central/build/moz.configure/init.configure", line 240, in <module>
 0:06.55     @imports(_from="six", _import="ensure_text")
 0:06.55   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 835, in decorator
 0:06.55     depends = DependsFunction(self, func, dependencies, when=when)
 0:06.55   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 137, in __init__
 0:06.55     sandbox._value_for(self)
 0:06.55   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 613, in _value_for
 0:06.55     return self._value_for_depends(obj)
 0:06.57   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\util.py", line 1049, in method_call
 0:06.57     cache[args] = self.func(instance, *args)
 0:06.57   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 622, in _value_for_depends
 0:06.57     value = obj.result()
 0:06.57   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\util.py", line 1049, in method_call
 0:06.57     cache[args] = self.func(instance, *args)
 0:06.57   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 162, in result
 0:06.57     return self._func(*resolved_args)
 0:06.58   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 1293, in wrapped
 0:06.58     return new_func(*args, **kwargs)
 0:06.58   File "c:/Users/upsuper/Sources/mozilla-central/build/moz.configure/init.configure", line 354, in virtualenv_python3
 0:06.59     manager.build(python)
 0:06.59   File "c:\Users\upsuper\Sources\mozilla-central\python\mozbuild\mozbuild\virtualenv.py", line 554, in build
 0:06.59     raise Exception("Error populating virtualenv.")
 0:06.59 Exception: Error populating virtualenv.

Comment 1

[Xidorn Quan [:xidorn] UTC+10]

After I moved the src dir to D: it's solved. It would probably be good to allow this or warn this gracefully?

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

It would be good to warn on this more gracefully, but it would require a bit of work - most environment validation we do happens in configure, but this error is thrown before such validation runs.
We could improve our generic virtualenv-creation error to be more clear that it doesn't work across windows partition boundaries, but the existing error arguably already hints at it?

I think that improving this is valuable, but I'm going to let this drift in the backlog for now.

Comment 3

[Alex Hochheiden [:ahochheiden]]

As far as I can tell this works. Mitch's refactoring of virtualenv.py to site.py probably fixed this as a side-effect.