1719144 - Various python errors with python 3.10

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, defect)

Description

[Emilio Cobos Álvarez (:emilio)]

Lots of errors like:

AttributeError: module 'collections' has no attribute 'Mapping'

  File "/home/emilio/src/moz/gecko-2/python/mozbuild/mozbuild/artifact_commands.py", line 390, in artifact_toolchain
    tasks = toolchain_task_definitions()
  File "/home/emilio/src/moz/gecko-2/python/mozbuild/mozbuild/toolchains.py", line 11, in toolchain_task_definitions
    from taskgraph.generator import load_tasks_for_kind
  File "/home/emilio/src/moz/gecko-2/taskcluster/taskgraph/generator.py", line 13, in <module>
    from . import filter_tasks
  File "/home/emilio/src/moz/gecko-2/taskcluster/taskgraph/filter_tasks.py", line 9, in <module>
    from . import target_tasks
  File "/home/emilio/src/moz/gecko-2/taskcluster/taskgraph/target_tasks.py", line 16, in <module>
    from taskgraph.parameters import Parameters
  File "/home/emilio/src/moz/gecko-2/taskcluster/taskgraph/parameters.py", line 17, in <module>
    from taskgraph.util.schema import validate_schema
  File "/home/emilio/src/moz/gecko-2/taskcluster/taskgraph/util/schema.py", line 238, in <module>
    {voluptuous.Required("task-reference"): text_type},
  File "/home/emilio/src/moz/gecko-2/third_party/python/voluptuous/voluptuous/schema_builder.py", line 1145, in __init__
    super(Required, self).__init__(schema, msg=msg,
  File "/home/emilio/src/moz/gecko-2/third_party/python/voluptuous/voluptuous/schema_builder.py", line 978, in __init__
    self._schema = Schema(schema_)
  File "/home/emilio/src/moz/gecko-2/third_party/python/voluptuous/voluptuous/schema_builder.py", line 202, in __init__
    self._compiled = self._compile(schema)
  File "/home/emilio/src/moz/gecko-2/third_party/python/voluptuous/voluptuous/schema_builder.py", line 283, in _compile
    if isinstance(schema, collections.Mapping):

Since those live now in collections.abc rather than collections.

Comment 1

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1719144 - Fix various imports for Python 3.10. r=#build

These are enough for me to run bootstrap+configure+build.

Some touch third-party code, not sure what the right way to deal with
that is though.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::Task Configuration' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 3

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1719144 - Update voluptuous. r=#build

Depends on D119080

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d2a88c5423a0
Fix various imports for Python 3.10. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/8e4b00b9d57c
Update voluptuous. r=firefox-build-system-reviewers,mhentges

Comment 5

[Sandor Molnar[:smolnar]]

Backed out for causing build bustages

Backout link: https://hg.mozilla.org/integration/autoland/rev/61ce4881f734b578d99385f32a9d6df73582f4b0

Push with failures

Failure log

Comment 6

[Emilio Cobos Álvarez (:emilio)]

Hmm, it seems we build some stuff with versions of python < 3.3, is that expected? If so, should I just do the version check a la https://hg.mozilla.org/integration/autoland/rev/8e4b00b9d57c#l11.12? Seems quite unfortunate / ugly...

Comment 7

[Mike Hommey [:glandium]]

The error says python3.7. The problem is you don't get collections.abc unless you import it.

Python 3.7.3 (default, Jan 22 2021, 20:04:44) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import collections
>>> collections.abc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/collections/__init__.py", line 55, in __getattr__
    raise AttributeError(f'module {__name__!r} has no attribute {name!r}')
AttributeError: module 'collections' has no attribute 'abc'
>>> import collections.abc
>>> collections.abc
<module 'collections.abc' from '/usr/lib/python3.7/collections/abc.py'>

Comment 8

[Emilio Cobos Álvarez (:emilio)]

Hmmwat, ok, will fix, thanks so much Mike :)

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2418633d529c
Fix various imports for Python 3.10. r=firefox-build-system-reviewers,mhentges
https://hg.mozilla.org/integration/autoland/rev/c3b650ea51c9
Update voluptuous. r=firefox-build-system-reviewers,mhentges

Comment 10

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/2418633d529c
https://hg.mozilla.org/mozilla-central/rev/c3b650ea51c9

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6b2a1a09cd47
Keep some code python2 compatible since we still run it in some windows talos jobs.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cdf8a9a9c85a
fix linting failure. r=fix CLOSED TREE

Comment 13

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/6b2a1a09cd47
https://hg.mozilla.org/mozilla-central/rev/cdf8a9a9c85a