1587525 - `ImportError: cannot import name get_context` running `mach eslint`

Status: RESOLVED INVALID

(Firefox Build System :: Lint and Formatting, defect)

Description

[Dave Townsend [:mossop] (he/him)]

Error running mach:

    ['eslint', 'intl/l10n/L10nRegistry.jsm']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ImportError: cannot import name get_context

  File "/Users/dave/mozilla/source/trunk/tools/lint/mach_commands.py", line 99, in eslint
    argv=extra_args, **kwargs)
  File "/Users/dave/mozilla/source/trunk/python/mach/mach/registrar.py", line 135, in dispatch
    if handler.parser:
  File "/Users/dave/mozilla/source/trunk/python/mach/mach/decorators.py", line 77, in parser
    self._parser = self._parser()
  File "/Users/dave/mozilla/source/trunk/tools/lint/mach_commands.py", line 34, in setup_argument_parser
    from mozlint import cli
  File "/Users/dave/mozilla/source/trunk/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/Users/dave/mozilla/source/trunk/python/mozlint/mozlint/__init__.py", line 6, in <module>
    from .roller import LintRoller
  File "/Users/dave/mozilla/source/trunk/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/Users/dave/mozilla/source/trunk/python/mozlint/mozlint/roller.py", line 17, in <module>
    from multiprocessing import cpu_count, get_context

Comment 1

[Dave Townsend [:mossop] (he/him)]

Looks like this import was added with bug 1580280. Maybe mach is using the wrong python here?

Comment 2

[Andrew Halberstadt [:ahal]]

Yes, the get_context function was added to multiprocessing in Python 3. But I'm not sure how you would see this as mach lint (and eslint) should only run with Python 3 now. Does it happen on every invocation? Was it working relatively recently?

Comment 3

[Dave Townsend [:mossop] (he/him)]

(In reply to Andrew Halberstadt [:ahal] from comment #2)

Yes, the get_context function was added to multiprocessing in Python 3. But I'm not sure how you would see this as mach lint (and eslint) should only run with Python 3 now. Does it happen on every invocation? Was it working relatively recently?

Every invocation. I haven't used the eslint command in quite some time so not sure when it was last working for me. How does lint ensure it is running with python 3?

Comment 4

[Release mgmt bot [:sylvestre / :calixte / :marco for bugbug]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 5

[Andrew Halberstadt [:ahal]]

There's a whitelist in the mach binary itself. Any command not listed there should run with Python 3:
https://searchfox.org/mozilla-central/rev/05a22d864814cb1e4352faa4004e1f975c7d2eb9/mach#13

The lint and eslint commands were removed from there in the same commit that dropped Python 2 support. Can you add:

print(sys.version_info)

just before the failing import?

Comment 6

[Andrew Halberstadt [:ahal]]

Just thought of something.. is there a chance you have mach copied somewhere on your $PATH? If so you might need to re-copy it (or symlink it if that's an option).

Comment 7

[Dave Townsend [:mossop] (he/him)]

Ah, turns out that due to past issues where selecting the correct python was broken in my build environment I run mach through a shell script that basically runs python mach ... which I imagine bypasses that whitelist entirely. I wonder if I still need that.

Comment 8

[Andrew Halberstadt [:ahal]]

Note when you run ./mach it is also a shell script that more or less just invokes python mach. So definitely try removing your shim and invoking it normally, otherwise you'll find more and more commands will stop working for you as time goes on.

If you still have troubles finding Python I can help you figure it out, but otherwise there's not much we can do for you here.

Comment 9

[Markus Stange [:mstange]]

(In reply to Andrew Halberstadt [:ahal] from comment #6)

Just thought of something.. is there a chance you have mach copied somewhere on your $PATH? If so you might need to re-copy it (or symlink it if that's an option).

This was the case for me. I deleted my copy of mach from ~/bin/ and made a new copy. Now mach lint works for me.