Closed Bug 1585734 Opened 5 years ago Closed 5 years ago

Unable to match regex against output: ModuleNotFoundError: No module named 'codespell_lib'

Categories

(Developer Infrastructure :: Lint and Formatting, defect)

defect
Not set
normal

Tracking

(firefox-esr60 unaffected, firefox-esr68 unaffected, firefox67 unaffected, firefox68 unaffected, firefox69 unaffected, firefox70 unaffected, firefox71 fixed)

RESOLVED FIXED
mozilla71
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox67 --- unaffected
firefox68 --- unaffected
firefox69 --- unaffected
firefox70 --- unaffected
firefox71 --- fixed

People

(Reporter: Mardak, Assigned: ahal)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

Split from bug 1585672 comment 4:

$ ./mach lint -l codespell browser/components/newtab/
Unable to match regex against output: Traceback (most recent call last):
Unable to match regex against output:   File "/mozilla-central/obj/_virtualenvs/init_py3/bin/codespell", line 7, in <module>
Unable to match regex against output:     from codespell_lib import _script_main
Unable to match regex against output: ModuleNotFoundError: No module named 'codespell_lib'
✖ 0 problems (0 errors, 0 warnings)
See Also: → 1585672

So this happens with or without the fix in bug 1585672? (I can't reproduce this one either)

Do you have a obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib (note change 3.7 to your version of python)? Does clobbering the _virtualenvs dir change anything?

Thanks for helping debug/fix these btw and sorry for the inconvenience.

This happens both with and without the fix in bug 1585672, and clobbering (including _virtualenvs) doesn't help:

$ rm -rf obj && ./mach lint -l codespell browser/components/newtab
Using base prefix '/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7'
/mozilla-central/third_party/python/virtualenv/virtualenv.py:1047: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /mozilla-central/obj/_virtualenvs/init_py3/bin/python3.7
Also creating executable in /mozilla-central/obj/_virtualenvs/init_py3/bin/python
Installing setuptools, pip, wheel...done.
b'running build_ext\ncopying build/lib.macosx-10.14-x86_64-3.7/psutil/_psutil_osx.cpython-37m-darwin.so -> psutil\ncopying build/lib.macosx-10.14-x86_64-3.7/psutil/_psutil_posix.cpython-37m-darwin.so -> psutil\n'
Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)
Unable to match regex against output: Traceback (most recent call last):
Unable to match regex against output:   File "/mozilla-central/obj/_virtualenvs/init_py3/bin/codespell", line 7, in <module>
Unable to match regex against output:     from codespell_lib import _script_main
Unable to match regex against output: ModuleNotFoundError: No module named 'codespell_lib'
✖ 0 problems (0 errors, 0 warnings)

codespell_lib does exist:

$ find obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/_codespell.py
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/tests
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/tests/test_dictionary.py
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/tests/test_basic.py
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/tests/__init__.py
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/tests/__pycache__
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/tests/__pycache__/test_dictionary.cpython-37.pyc
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/tests/__pycache__/test_basic.cpython-37.pyc
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/tests/__pycache__/__init__.cpython-37.pyc
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/__init__.py
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/__pycache__
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/__pycache__/_codespell.cpython-37.pyc
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/__pycache__/__init__.cpython-37.pyc
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/data
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/data/dictionary.txt
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/data/__init__.py
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/data/__pycache__
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/data/__pycache__/__init__.cpython-37.pyc
obj/_virtualenvs/init_py3/lib/python3.7/site-packages/codespell_lib/data/linux-kernel.exclude
See Also: → 1585744

Checking the sys.path, I do see mozilla.pth: and python3: entries but not objdir:

https://searchfox.org/mozilla-central/source/build/virtualenv_packages.txt

/mozilla-central/python/mach
/mozilla-central/python/mozboot
/mozilla-central/python/mozbuild
/mozilla-central/python/mozlint
/mozilla-central/python/mozrelease
/mozilla-central/python/mozterm
/mozilla-central/python/mozversioncontrol
/mozilla-central/python/l10n
/mozilla-central/third_party/python/atomicwrites
/mozilla-central/third_party/python/attrs/src
/mozilla-central/third_party/python/biplist
/mozilla-central/third_party/python/blessings
/mozilla-central/third_party/python/Click
/mozilla-central/third_party/python/compare-locales
/mozilla-central/third_party/python/configobj
/mozilla-central/third_party/python/cram
/mozilla-central/third_party/python/dlmanager
/mozilla-central/third_party/python/enum34
/mozilla-central/third_party/python/fluent
/mozilla-central/third_party/python/funcsigs
/mozilla-central/third_party/python/mohawk
/mozilla-central/third_party/python/more-itertools
/mozilla-central/third_party/python/mozilla-version
/mozilla-central/third_party/python/pathlib2
/mozilla-central/third_party/python/gyp/pylib
/mozilla-central/third_party/python/python-hglib
/mozilla-central/third_party/python/pluggy
/mozilla-central/third_party/python/jsmin
/mozilla-central/third_party/python/psutil
/mozilla-central/third_party/python/pylru
/mozilla-central/third_party/python/which
/mozilla-central/third_party/python/pystache
/mozilla-central/third_party/python/pyyaml/lib3/
/mozilla-central/third_party/python/requests
/mozilla-central/third_party/python/requests-unixsocket
/mozilla-central/third_party/python/slugid
/mozilla-central/third_party/python/taskcluster
/mozilla-central/third_party/python/taskcluster-urls
/mozilla-central/third_party/python/py
/mozilla-central/third_party/python/pytest/src
/mozilla-central/third_party/python/pytoml
/mozilla-central/third_party/python/redo
/mozilla-central/third_party/python/six
/mozilla-central/third_party/python/taskcluster-urls
/mozilla-central/third_party/python/voluptuous
/mozilla-central/third_party/python/json-e
/mozilla-central/build
/mozilla-central/build/pymake
/mozilla-central/config
/mozilla-central/config/mozunit
/mozilla-central/dom/bindings
/mozilla-central/dom/bindings/parser
/mozilla-central/layout/tools/reftest
/mozilla-central/other-licenses/ply/
/mozilla-central/taskcluster
/mozilla-central/testing
/mozilla-central/testing/firefox-ui/harness
/mozilla-central/testing/marionette/client
/mozilla-central/testing/marionette/harness
/mozilla-central/testing/marionette/harness/marionette_harness/runner/mixins/browsermob-proxy-py
/mozilla-central/testing/marionette/puppeteer/firefox
/mozilla-central/testing/raptor
/mozilla-central/testing/talos
/mozilla-central/testing/mozbase/manifestparser
/mozilla-central/testing/mozbase/mozcrash
/mozilla-central/testing/mozbase/mozdebug
/mozilla-central/testing/mozbase/mozdevice
/mozilla-central/testing/mozbase/mozfile
/mozilla-central/testing/mozbase/mozhttpd
/mozilla-central/testing/mozbase/mozinfo
/mozilla-central/testing/mozbase/mozinstall
/mozilla-central/testing/mozbase/mozleak
/mozilla-central/testing/mozbase/mozlog
/mozilla-central/testing/mozbase/moznetwork
/mozilla-central/testing/mozbase/mozpower
/mozilla-central/testing/mozbase/mozprocess
/mozilla-central/testing/mozbase/mozprofile
/mozilla-central/testing/mozbase/mozproxy
/mozilla-central/testing/mozbase/mozrunner
/mozilla-central/testing/mozbase/mozsystemmonitor
/mozilla-central/testing/mozbase/mozscreenshot
/mozilla-central/testing/mozbase/moztest
/mozilla-central/testing/mozbase/mozversion
/mozilla-central/tools
/mozilla-central/testing/web-platform
/mozilla-central/testing/web-platform/tests/tools/third_party/html5lib
/mozilla-central/testing/web-platform/tests/tools/third_party/webencodings
/mozilla-central/testing/web-platform/tests/tools/third_party/enum
/mozilla-central/testing/web-platform/tests/tools/third_party/h2
/mozilla-central/testing/web-platform/tests/tools/third_party/hpack
/mozilla-central/testing/web-platform/tests/tools/third_party/hyperframe
/mozilla-central/testing/web-platform/tests/tools/third_party/certifi
/mozilla-central/testing/web-platform/tests/tools/wptserve
/mozilla-central/testing/web-platform/tests/tools/wptrunner
/mozilla-central/testing/web-platform/tests/tools/six
/mozilla-central/testing/xpcshell
/mozilla-central/third_party/python/mock-1.0.0
/mozilla-central/xpcom/ds/tools
/mozilla-central/tools/docs
/mozilla-central/third_party/python/cbor2
/mozilla-central/third_party/python/pyasn1
/mozilla-central/third_party/python/pyasn1-modules
/mozilla-central/third_party/python/rsa
/mozilla-central/third_party/python/PyECC
/mozilla-central/toolkit/components/telemetry/tests/marionette/harness
/mozilla-central
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python37.zip
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload
/usr/local/lib/python3.7/site-packages
/usr/local/Cellar/protobuf/3.9.2/libexec/lib/python3.7/site-packages
/mozilla-central/testing/awsy

Looks like sys.exec_prefix when running ./mach lint is /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7 and sys.executable: /usr/local/opt/python/bin/python3.7

But if I manually launch ./obj/_virtualenvs/init_py3/bin/python3

sys.exec_prefix: /mozilla-central/./obj/_virtualenvs/init_py3/bin/..
sys.executable: /mozilla-central/obj/_virtualenvs/init_py3/bin/python3
sys.path includes /mozilla-central/obj/_virtualenvs/init_py3/lib/python3.7/site-packages'

Bugbug thinks this bug is a defect, but please change it back in case of error.

Type: task → defect

Fwiw it's expected that sys.executable and sys.exec_prefix won't point to the virtualenv. The reason for this is that we activate the virtualenv in-process using a special script that comes bundled with virtualenv. For whatever reason, this script doesn't mess with sys.executable (I imagine because technically the system Python was still used to start the active interpreter).

The fact that your sys.path includes that site-packages dir but still isn't found is really strange. I have a hunch that this issue is related to bug 1585672 and the global ImportHook. Try disabling it again (by commenting it out at the bottom of mach_bootstrap.py) and see if that fixes this issue.

I don't know why you were seeing modules with a __file__ value of None, but I think that might be the root of all these problems.

Oh re-reading your comment, it sounds like the objdir site-packages is not in your sys.path. That makes more sense and should be possible to debug.

The activate_this.py script is supposed to prepend the site-packages dir to sys.path. You can put some print debugging into it, it'll be at <virtualenv>/bin/activate_this.py. The exact place it should get invoked in |mach lint| is here:
https://searchfox.org/mozilla-central/source/tools/lint/mach_commands.py#63

Flags: needinfo?(edilee)

activate_this is updating sys.path, but looks like the way rstcheck is invoked, it loses the path:

https://searchfox.org/mozilla-central/rev/01d1011ca4a460f751da030d455d35c267c3e210/tools/lint/rst/__init__.py#83-88

From rst/__init__.py, sys.path has 114 entries including /mozilla-central/obj/_virtualenvs/init_py3/lib/python3.7/site-package when it's calling subprocess.Popen with cmdargs == ['/mozilla-central/obj/_virtualenvs/init_py3/bin/rstcheck', '/mozilla-central/browser/components/newtab/docs/index.rst']

Modifying /mozilla-central/obj/_virtualenvs/init_py3/bin/rstcheck to print out sys.path has only these 6 entries:

/mozilla-central/obj/_virtualenvs/init_py3/bin
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python37.zip
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload
/usr/local/lib/python3.7/site-packages
/usr/local/Cellar/protobuf/3.9.2/libexec/lib/python3.7/site-packages

Similarly, modifying /mozilla-central/obj/_virtualenvs/init_py3/bin/codespell to print sys.path has the same 6 entries as above. Although codespell uses ProcessHandlerMixin.run instead:

https://searchfox.org/mozilla-central/rev/01d1011ca4a460f751da030d455d35c267c3e210/tools/lint/spell/__init__.py#70-73

Flags: needinfo?(edilee)

Just to confirm, before switching to python 3, modifying /mozilla-central/obj/_virtualenvs/init/bin/rstcheck to print sys.path includes:

/mozilla-central/obj/_virtualenvs/init/lib/python2.7/site-packages

But why is it working for me and Shivam (and presumably everyone else who has run this in the last couple days but hasn't complained)? And why did it work before on Python 2 but start failing with Python 3? So many questions..

But this should be solvable without understanding the issue I guess. Either by calling those binaries in-process or passing sys.path into the PYTHONPATH environment variable.

Assignee: nobody → ahal
Status: NEW → ASSIGNED

Aha! I think I figured it out. ??

$ head -1 obj/_virtualenvs/init/bin/rstcheck
#!/mozilla-central/obj/_virtualenvs/init/bin/python2.7

$ head -1 obj/_virtualenvs/init_py3/bin/rstcheck
#!/usr/local/bin/python3

If I modify the python 3 version to use #!/mozilla-central/obj/_virtualenvs/init_py3/bin/python3 instead, it runs correctly with the expected sys.path !

Aha! Yeah, that would do it. My init_py3/bin/rstcheck binary points to the virtuanlenv Python.

I'm guessing that shebang gets written at install time. Rstcheck is ultimately installed here:
https://searchfox.org/mozilla-central/source/python/mozlint/mozlint/util/pip.py#13

So maybe that command is picking up your system pip instead of the one in init_py3/bin/pip. Can you confirm that init_py3/bin/pip exists on your system? Maybe we just need to call it pip3.

Hm, but rstcheck did get installed there.. so that doesn't make a ton of sense.

$ ls -1 obj/_virtualenvs/init*/bin/pip*
obj/_virtualenvs/init/bin/pip
obj/_virtualenvs/init/bin/pip2
obj/_virtualenvs/init/bin/pip2.7
obj/_virtualenvs/init_py3/bin/pip
obj/_virtualenvs/init_py3/bin/pip3
obj/_virtualenvs/init_py3/bin/pip3.7

If I do subprocess.run(['/usr/bin/which', 'pip']) from mozlint/util/pip.py, it prints out:

/mozilla-central/obj/_virtualenvs/init_py3/bin/pip

Ok, I'm overthinking this. We can change the ["rstcheck"] command to [python, "rstcheck"] (making sure to find the correct virtualenv python). Probably a similar fix needed for codespell. I'll try to get to this sometime tomorrow (unless you beat me to it).

Use distutils.spawn.find_executable to get the correct virtualenv python3.

Attachment #9098666 - Attachment description: Bug 1585734 - [lint] Make sure codespell, rst and yaml run with virtualenv Python 3 → Bug 1585734 - [lint] Make sure codespell, rst and yaml run with virtualenv python
Attachment #9098666 - Attachment description: Bug 1585734 - [lint] Make sure codespell, rst and yaml run with virtualenv python → Bug 1585734 - [lint] Make sure codespell, rst and yaml run with virtualenv Python 3
Attachment #9098666 - Attachment description: Bug 1585734 - [lint] Make sure codespell, rst and yaml run with virtualenv Python 3 → Bug 1585734 - [lint] Make sure codespell, rst and yaml run with virtualenv python
Pushed by elee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a96731900f47
[lint] Make sure codespell, rst and yaml run with virtualenv python r=ahal
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla71
Has Regression Range: --- → yes
Product: Firefox Build System → Developer Infrastructure
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: