mach doc is busted with python 3.10
Categories
(Developer Infrastructure :: Source Documentation, defect)
Tracking
(firefox110 fixed)
| Tracking | Status | |
|---|---|---|
| firefox110 | --- | fixed |
People
(Reporter: emilio, Assigned: Sylvestre)
References
(Blocks 2 open bugs)
Details
Attachments
(5 files)
Traceback (most recent call last):
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/registry.py", line 429, in load_extension
mod = import_module(extname)
File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx_js/__init__.py", line 1, in <module>
from .directives import (auto_class_directive_bound_to_app,
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx_js/directives.py", line 13, in <module>
from .renderers import AutoFunctionRenderer, AutoClassRenderer, AutoAttributeRenderer
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx_js/renderers.py", line 11, in <module>
from .parsers import PathVisitor
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx_js/parsers.py", line 3, in <module>
from parsimonious import Grammar, NodeVisitor
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/parsimonious/__init__.py", line 8, in <module>
from parsimonious.grammar import Grammar, TokenGrammar
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/parsimonious/grammar.py", line 8, in <module>
from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib64/python3.10/collections/__init__.py)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/cmd/build.py", line 276, in build_main
app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/application.py", line 237, in __init__
self.setup_extension(extension)
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/application.py", line 394, in setup_extension
self.registry.load_extension(self, extname)
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/registry.py", line 432, in load_extension
raise ExtensionError(__('Could not import extension %s') % extname,
sphinx.errors.ExtensionError: Could not import extension sphinx_js (exception: cannot import name 'Mapping' from 'collections' (/usr/lib64/python3.10/collections/__init__.py))
Extension error:
Could not import extension sphinx_js (exception: cannot import name 'Mapping' from 'collections' (/usr/lib64/python3.10/collections/__init__.py))
|
Reporter | |
Comment 1•3 years ago
|
||
Updating parsimonious should do, but I think we also need to update https://github.com/mozilla/sphinx-js to use the newer version of parsimonious.
|
||
Comment 2•3 years ago
|
||
The severity field is not set for this bug.
:Sylvestre, could you have a look please?
For more information, please visit auto_nag documentation.
|
Assignee | |
Comment 3•2 years ago
|
||
yeah, we should fix it.
I noticed that lonnen has been involved \o/
|
||
Updated•2 years ago
|
|
||
Comment 4•2 years ago
|
||
It appears rather complex to update sphinx-js to work with modern python
versions (3.10 and more recent). In the meantime, and as a work around,
developers that wish to run ./mach docs to preview regular (non-js) docs can
apply this patch.
|
|
Assignee | |
Updated•2 years ago
|
|
|
Assignee | |
Comment 5•2 years ago
•
|
||
| Comment hidden (duplicate) |
|
||
Comment 8•2 years ago
|
||
That error doesn't ring a bell specifically, but I do recall I too tried to update it once and couldn't get it working. Although I think I couldn't get the packages to resolve so looks like you got further than me!
I'm not familiar enough with Sphinx to know what's going on off the top of my head, but if we can add some print statements to that role_for_objtype function, maybe we can narrow it down. Unfortunately I won't have time for this until after the holidays.
|
||
Comment 9•2 years ago
|
||
I can take a look. If it helps, assign the bug to me and I can chase it down in the next week.
|
|
||
Comment 10•2 years ago
|
||
I have an up-to-date repository and I apply that patch. Seems like I have to recompile requirements.txt (making sure to use Python 3.9). That doesn't work initially because importlib_metadata also needs to be updated. I update that and then recompile requirements.txt.
Then when I run mach doc, it looks like it's pulling in really old versions of libraries from third_party/python/ and errors out:
sphinx 5.3.0 has requirement importlib-metadata>=4.8; python_version < "3.10", but you have importlib-metadata 1.7.0.
sphinx 5.3.0 has requirement Jinja2>=3.0, but you have jinja2 2.11.3.
sphinx 5.3.0 has requirement packaging>=21.0, but you have packaging 20.9.
sphinx-js 3.2.0 has requirement markupsafe==2.0.1, but you have markupsafe 1.1.1.
Those are the versions in third_party/python/ and not the versions specified in requirements.in.
Sylvestre: How're you testing the patch?
|
|
Assignee | |
Comment 11•2 years ago
|
||
as i am in testing mode, I did:
rm -rf
on these directories
;)
|
|
||
Comment 12•2 years ago
|
||
I don't see how either deleting everything in third_party/python/ or deleting the specific packages that are having conflicts works since it looks like packaging is needed by mach to run. I'll spend some more time trying to figure out how to set everything up in order to get the error in comment #5, but if I can't get it today, then I don't think I can help here until someone gives me steps to reproduce.
|
|
Assignee | |
Comment 13•2 years ago
|
||
well, i only deleted a couple directories, not all.
Here is the patch:
https://phabricator.services.mozilla.com/D164899 (it is ugly ;)
and with python 3.10
|
|
||
Comment 14•2 years ago
|
||
That helps a lot! I can reproduce the issue. It's this:
https://github.com/mozilla/sphinx-js/pull/178
It's one of the things that changed in sphinx-js between 3.0.1 (what firefox docs are using now) and 3.2.0 (what I just released). I'll work on fixing that in sphinx-js and do another release.
|
|
||
Comment 15•2 years ago
|
||
I pushed out sphinx-js 3.2.1 just now which fixes the issue in comment #5. I then did a mach doc using sphinx-js 3.2.1 and it now completes with no errors (but a TON of warnings--wowzers!).
|
|
Assignee | |
Comment 16•2 years ago
|
||
|
|
Assignee | |
Comment 17•2 years ago
|
||
|
||
Updated•2 years ago
|
|
|
||
Updated•2 years ago
|
|
||
Comment 18•2 years ago
|
||
|
||
Comment 19•2 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/3b6e51a87baf
https://hg.mozilla.org/mozilla-central/rev/281555ad04ca
Description
•