1550956 - `python3 ./mach` throws with ModuleNotFoundError

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Kagami Rosylight [:saschanaz] (they/them) (OOO until Dec 11)]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

python3 ./mach

Actual results:

Traceback (most recent call last):
  File "mach", line 86, in <module>
    main(sys.argv[1:])
  File "mach", line 78, in main
    mach = get_mach()
  File "mach", line 68, in get_mach
    mach = check_and_get_mach(dir_path)
  File "mach", line 42, in check_and_get_mach
    return load_mach(dir_path, mach_path)
  File "mach", line 28, in load_mach
    ('.py', 'r', imp.PY_SOURCE))
  File "/usr/lib/python3.6/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.6/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/mnt/c/Users/sasch/Documents/GitHub/gecko-dev/build/mach_bootstrap.py", line 14, in <module>
    import __builtin__
ModuleNotFoundError: No module named '__builtin__'

Expected results:

Python 2.7 or above (but not Python 3) is required to run mach.
You are running Python 3.6.7

I can submit a patch as simply fixing the import __builtin__ line will solve this.

Comment 1

[Kagami Rosylight [:saschanaz] (they/them) (OOO until Dec 11)]

Attachment: Bug 1550956: Import builtins in Python 3 compatible way

Comment 2

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

:Callek, will bug 1547730 fix this as well?

Comment 3

[Justin Wood (:Callek)]

No, 1547730 will not fix mach_bootstrap.py

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fa4c37c31dfd
Import builtins in Python 3 compatible way r=firefox-build-system-reviewers,chmanchester

Comment 5

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/fa4c37c31dfd

Comment 6

[Mike de Boer [:mikedeboer]]

This patch causes the mach script to fail for each command for me locally on OSX:

mikedeboer:(hg)mozilla-central[267ddc3]/$ ./mach help
Traceback (most recent call last):
  File "./mach", line 86, in <module>
    main(sys.argv[1:])
  File "./mach", line 78, in main
    mach = get_mach()
  File "./mach", line 68, in get_mach
    mach = check_and_get_mach(dir_path)
  File "./mach", line 42, in check_and_get_mach
    return load_mach(dir_path, mach_path)
  File "./mach", line 28, in load_mach
    ('.py', 'r', imp.PY_SOURCE))
  File "/Users/mikedeboer/Projects/mozilla-central/build/mach_bootstrap.py", line 440, in <module>
    builtins.__import__ = ImportHook(builtins.__import__)
AttributeError: 'module' object has no attribute '__import__'

Comment 7

[Cristian Brindusan [:cbrindusan]]

Backed out changeset fa4c37c31dfd (Bug 1550956) as requested by mikedeboer on IRC for causing mach to stop working on Python 2.
Backout: https://hg.mozilla.org/mozilla-central/rev/5f95b3f2ea44723ba6a8c41a4b27c88032df709f

Comment 8

[Mike de Boer [:mikedeboer]]

Evaluating import builtins and builtins.__path__ in python CLI gives me:

['/usr/local/lib/python2.7/site-packages/builtins']

So that looks like there's a local package with that name conflicting here. h/t to :Pike for the hint here!

Comment 9

[Kagami Rosylight [:saschanaz] (they/them) (OOO until Dec 11)]

Any idea what builtins do on macOS? Can you access native builtins in Py3 on macOS?

Comment 10

[Mike Hommey [:glandium]]

This module apparently comes from https://pypi.org/project/future/

Comment 11

[Mike de Boer [:mikedeboer]]

Yeah I can access the native ones in Py3.

So this is what that package above, for Py2, contains:

https://gist.github.com/mikedeboer/6d80b6d12d7160e8a4a69592dcbc2403

Comment 12

[Kagami Rosylight [:saschanaz] (they/them) (OOO until Dec 11)]

Ah, I see, thanks. Looks like we should do sys.version_info thing instead.

Comment 13

[Andi [:andi]]

The same issue as #c6 I have on macOS

Comment 14

[Kagami Rosylight [:saschanaz] (they/them) (OOO until Dec 11)]

Attachment: Bug 1550956: Import builtins in Python 3 compatible way

The previous try turns out to conflict with local future package, so this now uses sys.version_info instead of ImportError.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b09e197ba5a3
Import builtins in Python 3 compatible way r=firefox-build-system-reviewers,chmanchester

Comment 16

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/b09e197ba5a3