1622789 - Stop using `distutils` in the ./mach bootstrap critical path (and add `python3-distutils` dependency)

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect, P2)

Description

[Kartikaya Gupta (email:[e-mail])]

I just created a VM with Ubuntu 18.04.4 LTS and it only has python3, not python2. I cloned mozilla-central and ran ./mach bootstrap but it failed like so:

Traceback (most recent call last):
  File "./mach", line 264, in <module>
    main(sys.argv[1:])
  File "./mach", line 256, in main
    mach = get_mach()
  File "./mach", line 246, in get_mach
    mach = check_and_get_mach(dir_path)
  File "./mach", line 202, in check_and_get_mach
    return load_mach(dir_path, mach_path)
  File "./mach", line 190, in load_mach
    return mach_bootstrap.bootstrap(dir_path)
  File "/home/kats/zspace/gecko-dev/build/mach_bootstrap.py", line 396, in bootstrap
    repo = resolve_repository()
  File "/home/kats/zspace/gecko-dev/build/mach_bootstrap.py", line 210, in resolve_repository
    import mozversioncontrol
  File "/home/kats/zspace/gecko-dev/build/mach_bootstrap.py", line 434, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/kats/zspace/gecko-dev/python/mozversioncontrol/mozversioncontrol/__init__.py", line 17, in <module>
    from distutils.spawn import find_executable
  File "/home/kats/zspace/gecko-dev/build/mach_bootstrap.py", line 434, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'distutils.spawn'

It would be nice if the bootstrap process worked with either python2 or python3.

Comment 1

[Kartikaya Gupta (email:[e-mail])]

Hm it failed the same way after installing the python and python2.7 packages. python --version gives me 2.7.17 so that should be the one it's picking up.

Comment 2

[Andrew Halberstadt [:ahal]]

Yeah, the problem is some Python distributions don't come with the distutils module at all. We should fix this by getting mozversioncontrol to stop using distutils.spawn.find_executable and switch over to mozfile.which (or shutil_which if we don't want to depend on mozfile).

Comment 3

[Geoff Brown [:gbrown]]

Attachment: Bug 1622789 - Remove mozversioncontrol dependency on distutils; r=

Use mozfile.which instead of distutils.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/98d7f9e4839f
Remove mozversioncontrol dependency on distutils; r=ahal

Comment 5

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/98d7f9e4839f

Comment 6

[Geoff Brown [:gbrown]]

Bug 1654915 intends to remove distutils usage from mozinfo.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

Attachment: bootstrap output

I'm spinning up a new Linux install and hit this distutils.spawn problem again.

Comment 8

[Kartikaya Gupta (email:[e-mail])]

sudo apt install python3-distutils allowed me to proceed. This is on ubuntu 20.04 LTS in case it helps.

Comment 9

[Mitchell Hentges [:mhentges] 🦀]

The core issue seems to be resolved with that patch.
Attempting to reproduce Kartikaya's issue caused ./mach bootstrap to tell me that pip needed to be installed. Installing python3-pip installed python3-distutils, so I believe that the confusing failure can no longer be reproduced.