1953183 - `mach marionette-test` fails on python 3.13 due to `cgi` import error

Status: RESOLVED DUPLICATE of bug 1940455

(Testing :: Marionette Client and Harness, defect)

Description

[Jeremy Swinarton]

Error running mach:

    mach marionette-test browser/components/sessionstore/test/marionette/test_restore_manually_with_tab_groups.py

The error occurred in mach itself. This is likely a bug in mach itself or a
fundamental problem with a loaded module.

You can invoke ``./mach busted`` to check if this issue is already on file. If it
isn't, please use ``./mach busted file general`` to report it. If ``./mach busted`` is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ModuleNotFoundError: No module named 'cgi'

  File "[repo_root]/mozilla-unified/python/mach/mach/main.py", line 267, in run
    return self._run(argv)
           ~~~~~~~~~^^^^^^
  File "[repo_root]/mozilla-unified/python/mach/mach/main.py", line 328, in _run
    args = parser.parse_args(argv)
  File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/argparse.py", line 1889, in parse_args
    args, argv = self.parse_known_args(args, namespace)
                 ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/argparse.py", line 1899, in parse_known_args
    return self._parse_known_args2(args, namespace, intermixed=False)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/argparse.py", line 1928, in _parse_known_args2
    namespace, args = self._parse_known_args(args, namespace, intermixed)
                      ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/argparse.py", line 2183, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/argparse.py", line 2135, in consume_positionals
    take_action(action, args)
    ~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/argparse.py", line 2004, in take_action
    action(self, namespace, argument_values, option_string)
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[repo_root]/mozilla-unified/python/mach/mach/dispatcher.py", line 179, in __call__
    if handler.parser:
       ^^^^^^^^^^^^^^
  File "[repo_root]/mozilla-unified/python/mach/mach/decorators.py", line 109, in parser
    self._parser = self._parser()
                   ~~~~~~~~~~~~^^
  File "[repo_root]/mozilla-unified/testing/marionette/mach_commands.py", line 23, in create_parser_tests
    from marionette_harness.runtests import MarionetteArguments
  File "[repo_root]/mozilla-unified/testing/marionette/harness/marionette_harness/__init__.py", line 21, in <module>
    from .runner import (
    ...<10 lines>...
    )
  File "[repo_root]/mozilla-unified/testing/marionette/harness/marionette_harness/runner/__init__.py", line 5, in <module>
    from .base import (
    ...<9 lines>...
    )
  File "[repo_root]/mozilla-unified/testing/marionette/harness/marionette_harness/runner/base.py", line 30, in <module>
    from . import serve
  File "[repo_root]/mozilla-unified/testing/marionette/harness/marionette_harness/runner/serve.py", line 20, in <module>
    from . import httpd
  File "[repo_root]/mozilla-unified/testing/marionette/harness/marionette_harness/runner/httpd.py", line 19, in <module>
    from wptserve import handlers, request, server
  File "[repo_root]/mozilla-unified/testing/web-platform/tests/tools/wptserve/wptserve/__init__.py", line 1, in <module>
    from .server import WebTestHttpd, WebTestServer, Router  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[repo_root]/mozilla-unified/testing/web-platform/tests/tools/wptserve/wptserve/server.py", line 29, in <module>
    from . import routes as default_routes
  File "[repo_root]/mozilla-unified/testing/web-platform/tests/tools/wptserve/wptserve/routes.py", line 1, in <module>
    from . import handlers
  File "[repo_root]/mozilla-unified/testing/web-platform/tests/tools/wptserve/wptserve/handlers.py", line 13, in <module>
    from .request import Authentication
  File "[repo_root]/mozilla-unified/testing/web-platform/tests/tools/wptserve/wptserve/request.py", line 4, in <module>
    import cgi

I was able to get this working with python3.12 by running python3.12 ./mach marionette-test browser/components/sessionstore/test/marionette/test_restore_manually_with_tab_groups.py.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The problem here seems to be with wptserve but lets move to Marionette for now.

Over on https://github.com/web-platform-tests/wpt/pull/49970 the wpt tests were updated to work with Python 3.13 and as it looks like there was also some cli lib work done which has been downstreamed:

https://searchfox.org/mozilla-central/search?q=testing%2Fweb-platform%2Ftests%2Ftools%2Fwptserve%2Fwptserve&path=

Jeremy, you are on a latest checkout of mozilla-unified?

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Just noticed that the downstream happened about 12h ago via bug 1940455, which is after the bug got filed. So I assume this works now as expected?

Comment 3

[[:sergesanspaille]]

cgi indeed got deprecated in Python 3.11 and removed in Python 3.13. Upstream explains why in https://peps.python.org/pep-0594/#cgi

Comment 4

[Jeremy Swinarton]

Just pulled down the latest central and confirmed that the commit linked in bug 1940455 was in the stack. With that, this now works. Thanks!

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Great! Thanks for confirming.