1888570 - Running browser chrome tests locally with python 3.12 fails

Status: RESOLVED FIXED

(Testing :: Mochitest, defect, P2)

Description

[Robin Steuber (she/her) [:bytesized]]

When I try to run browser chrome mochitests locally on macOS, I get an error. It seems to be consistent no matter which browser chrome test I attempt to run:

> ./mach test browser/components/urlbar/tests/browser-tips/suppress-tips/browser_suppressTips.js 
Error running mach:

    mach test browser/components/urlbar/tests/browser-tips/suppress-tips/browser_suppressTips.js

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke ``./mach busted`` to check if this issue is already on file. If it
isn't, please use ``./mach busted file test`` to report it. If ``./mach busted`` is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ModuleNotFoundError: No module named 'distutils'

  File "/Users/tester/mozilla-unified/testing/mach_commands.py", line 454, in test
    res = command_context._mach_context.commands.dispatch(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tester/mozilla-unified/python/mach/mach/registrar.py", line 182, in dispatch
    if handler.parser:
       ^^^^^^^^^^^^^^
  File "/Users/tester/mozilla-unified/python/mach/mach/decorators.py", line 105, in parser
    self._parser = self._parser()
                   ^^^^^^^^^^^^^^
  File "/Users/tester/mozilla-unified/testing/mochitest/mach_commands.py", line 193, in setup_argument_parser
    load_source("mochitest", path)
  File "/Users/tester/mozilla-unified/testing/mozbase/mozfile/mozfile/mozfile.py", line 645, in load_source
    loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/Users/tester/mozilla-unified/obj-clean/_tests/testing/mochitest/runtests.py", line 37, in <module>
    from distutils import spawn

Sentry event ID: 2f9e8baae3054258bb5f0b595625b0a0

Comment 1

[Robin Steuber (she/her) [:bytesized]]

I tried this on a clean checkout of mozilla-central with a mozconfig with only this line uncommented:

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-clean

Comment 2

[Mike Hommey [:glandium]]

I doubt macOS ships with python 3.12 itself. Try uninstalling python 3.12 and use the python3 that comes with the command line tools.

Comment 3

[Paul Zühlcke [:pbz]]

Ran into this after a brew upgrade. brew unlink python@3.12 fixed the issue.

Comment 4

[Robin Steuber (she/her) [:bytesized]]

If I try to remove it, I get a warning that uninstalling it will break watchman. If I uninstall it and watchman and then rerun ./mach bootstrap it just installs python 3.12 again, presumably in order to install watchman.

Comment 5

[Robin Steuber (she/her) [:bytesized]]

(In reply to Paul Zühlcke [:pbz] from comment #3)

Ran into this after a brew upgrade. brew unlink python@3.12 fixed the issue.

This did seem to work for me.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:jmaher, could you have a look please?

For more information, please visit BugBot documentation.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

we will end up needing to support newer versions of python both from mach and in CI. For now this has a workaround.

Comment 8

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Attachment: Bug 1888570 - Replace all usages of `distutils.spawn.find_executable` with `shutil.which` within `mochitest` r?jmaher

Comment 9

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Attachment: Bug 1888570 - Replace `distutils.util.strtobool` with a custom `strtobool` within `mochitest` r?jmaher

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dee593604635
Replace all usages of `distutils.spawn.find_executable` with `shutil.which` within `mochitest` r=jmaher
https://hg.mozilla.org/integration/autoland/rev/689cadd510e1
Replace `distutils.util.strtobool` with a custom `strtobool` within `mochitest` r=jmaher

Comment 12

[Noemi Erli[:noemi_erli]]

Backed out 8 changesets (Bug 1899681, Bug 1888570, Bug 1898817) as requested by the dev CLOSED TREE

Failure Log: https://treeherder.mozilla.org/logviewer?job_id=462065796&repo=autoland&lineNumber=890

Backout: https://hg.mozilla.org/integration/autoland/rev/48183268590c481e37309299d130508c7d1dc43e

Comment 13

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Thanks!

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a7b92246bd4d
Replace all usages of `distutils.spawn.find_executable` with `shutil.which` within `mochitest` r=jmaher
https://hg.mozilla.org/integration/autoland/rev/5621ba683661
Replace `distutils.util.strtobool` with a custom `strtobool` within `mochitest` r=jmaher

Comment 15

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/a7b92246bd4d
https://hg.mozilla.org/mozilla-central/rev/5621ba683661