1751474 - Mapping is not part of Collections module

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, defect)

Description

[Stefan Zabka (:vringar)]

Steps to reproduce:

I ran mach bootstrap which then proceeded to call mach with the following arguments ['artifact', 'toolchain', '--bootstrap', '--from-build', 'linux64-clang-tidy']

Actual results:

Since python 3.3 the collections module no longer contains the Mapping class.
Instead it now exists as collections.abc.Mapping.
This caused the subprocess to crash.

Expected results:

The existing class should have been imported

Comment 1

[Stefan Zabka (:vringar)]

The stacktrace is

The details of the failure are as follows:

AttributeError: module 'collections' has no attribute 'Mapping'

  File "/run/media/stefan/SourceCode/mozilla-unified/python/mozbuild/mozbuild/artifact_commands.py", line 380, in artifact_toolchain
    tasks = toolchain_task_definitions()
  File "/run/media/stefan/SourceCode/mozilla-unified/python/mozbuild/mozbuild/toolchains.py", line 11, in toolchain_task_definitions
    from gecko_taskgraph.generator import load_tasks_for_kind
  File "/run/media/stefan/SourceCode/mozilla-unified/taskcluster/gecko_taskgraph/generator.py", line 10, in <module>
    from taskgraph.config import GraphConfig
  File "/run/media/stefan/SourceCode/mozilla-unified/third_party/python/taskcluster_taskgraph/taskgraph/config.py", line 19, in <module>
    graph_config_schema = Schema(
  File "/run/media/stefan/SourceCode/mozilla-unified/third_party/python/taskcluster_taskgraph/taskgraph/util/schema.py", line 187, in __init__
    check_schema(self)
  File "/run/media/stefan/SourceCode/mozilla-unified/third_party/python/taskcluster_taskgraph/taskgraph/util/schema.py", line 176, in check_schema
    iter("schema", schema.schema)
  File "/run/media/stefan/SourceCode/mozilla-unified/third_party/python/taskcluster_taskgraph/taskgraph/util/schema.py", line 164, in iter
    if isinstance(sch, collections.Mapping):

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

This has been fixed since last July.
Which revision/version of Firefox are you building?

Comment 3

[Stefan Zabka (:vringar)]

I am on branches/default/tip/trunk.
I'm pretty confused why it differs from what is shown on sf.

Comment 4

[Stefan Zabka (:vringar)]

ea0d2279861f02176a1d1f1080c455f8de308b41 is my hg commit hash.

Comment 5

[Stefan Zabka (:vringar)]

It seems that my local repo is just completly messed up. Sorry for the bad report.

Comment 6

[Mike Hommey [:glandium]]

Your repo is not messed up. The problem is that we have two versions of the same code, and one was fixed, and we recently switched to the one that is not fixed.

Comment 8

[Stefan Zabka (:vringar)]

Attachment: Bug 1751474: Changed import to collections.abc r=ahal

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/taskgraph/rev/ccc2f284abde
Changed import to collections.abc r=ahal,releng-reviewers

Comment 10

[Andrew Halberstadt [:ahal]]

Thanks, I just released taskgraph==1.1.2 with this fix. Don't have time to craft the Gecko update patch right now, but can likely get to it tonight.

Comment 11

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1751474 - Vendor taskcluster-taskgraph==1.1.2

This picks up a bustage fix that occurs with Python 3.10.

Comment 13

[Jens Stutte [:jstutte]]

Applying that patch I get now (maybe unrelated):

Error running mach:

    ['bootstrap']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file bootstrap| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

TypeError: ArchlinuxBootstrapper.ensure_nasm_packages() missing 2 required positional arguments: 'state_dir' and 'checkout_root'

  File "/home/jens/src/mozilla-central/python/mozboot/mozboot/mach_commands.py", line 43, in bootstrap
    bootstrapper.bootstrap(command_context.settings)
  File "/home/jens/src/mozilla-central/python/mozboot/mozboot/bootstrap.py", line 374, in bootstrap
    self.maybe_install_private_packages_or_exit(application)
  File "/home/jens/src/mozilla-central/python/mozboot/mozboot/bootstrap.py", line 241, in maybe_install_private_packages_or_exit
    self.instance.ensure_nasm_packages()

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2cb1f8cf75c4
Vendor taskcluster-taskgraph==1.1.2 r=jcristau

Comment 15

[Jens Stutte [:jstutte]]

(In reply to Jens Stutte [:jstutte] from comment #13)

Applying that patch I get now (maybe unrelated):

See bug 1751673.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/7e667b6a803d
Vendor taskcluster-taskgraph==1.1.2 r=jcristau

Comment 18

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/7e667b6a803d

Comment 20

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/2cb1f8cf75c4