1949718 - Upgrade to connexion 3.0+

Status: NEW

(Release Engineering :: Applications: Shipit, task, P3)

Description

[Johan Lorenzo [:jlorenzo]]

From github: https://github.com/mozilla-releng/shipit/issues/1370.

In https://github.com/mozilla-releng/shipit/pull/1369 I pinned connexion < 3.0 because it causes test failures. It turns out that connexion 3.0 is a major re-write of the library:
https://connexion.readthedocs.io/en/latest/v3.html

I didn't have time to figure out how to upgrade, so just added a pin for now. Note I didn't look into it at all, so upgrading could be quite easy, I'm not sure.

Change performed by the Move to Bugzilla add-on.

Comment 1

[Johan Lorenzo [:jlorenzo]]

On February 7, 2024, @JohanLorenzo wrote:

I spent an hour looking into this yesterday. It looks like a rabbit hole. I didn't manage to green up tox. I'll put one comment per error I encountered with tox.

Comment 2

[Johan Lorenzo [:jlorenzo]]

On February 7, 2024, @JohanLorenzo wrote:

1. Easy: Fix src/backend_common/api.py:10: ModuleNotFoundError

>   from connexion.apis.flask_api import FlaskApi
E   ModuleNotFoundError: No module named 'connexion.apis'

src/backend_common/api.py:10: ModuleNotFoundError

This patch does the trick (source: spec-first/connexion#1621):

-from connexion.apis.flask_api import FlaskApi
+from connexion.apps.flask import FlaskApi

Comment 3

[Johan Lorenzo [:jlorenzo]]

On February 7, 2024, @JohanLorenzo wrote:

2. Easy: .tox/py39/lib/python3.9/site-packages/connexion/apps/flask.py:11: ModuleNotFoundError

>   from a2wsgi import WSGIMiddleware
E   ModuleNotFoundError: No module named 'a2wsgi'

.tox/py39/lib/python3.9/site-packages/connexion/apps/flask.py:11: ModuleNotFoundError

Made it work with:

-connexion = {version = "*", extras = ["swagger-ui"]}
+connexion = {version = "*", extras = ["swagger-ui", "flask"]}

This seems to be an extra thing now because connexion 3.0 switched to an ASGI server instead of a WSGI server (source: https://connexion.readthedocs.io/en/stable/v3.html#running-the-application)

Comment 4

[Johan Lorenzo [:jlorenzo]]

On February 7, 2024, @JohanLorenzo wrote:

3. Easy: .tox/py39/lib/python3.9/site-packages/connexion/middleware/abstract.py:60: AttributeError

    def _set_base_path(self, base_path: t.Optional[str] = None) -> None:
        if base_path is not None:
            # update spec to include user-provided base_path
            self.specification.base_path = base_path
            self.base_path = base_path
        else:
>           self.base_path = self.specification.base_path
E           AttributeError: 'PosixPath' object has no attribute 'base_path'

.tox/py39/lib/python3.9/site-packages/connexion/middleware/abstract.py:60: AttributeError

Made it work with:

+from connexion.spec import Specification
[...]
-            specification=pathlib.Path(specification),
+            specification=Specification.from_file(specification),

Comment 5

[Johan Lorenzo [:jlorenzo]]

On February 7, 2024, @JohanLorenzo wrote:

4. Unsure if I got the right fix: src/backend_common/api.py:98: AttributeError

>       self.swagger_url = api.options.openapi_console_ui_path
E       AttributeError: 'FlaskApi' object has no attribute 'options'

src/backend_common/api.py:98: AttributeError

I'm not sure of the impact of:

+++ b/api/src/backend_common/api.py

-        self.swagger_url = api.options.openapi_console_ui_path

Comment 6

[Johan Lorenzo [:jlorenzo]]

On February 7, 2024, @JohanLorenzo wrote:

5. Couldn't fix it: .tox/py39/lib/python3.9/site-packages/werkzeug/local.py:508: RuntimeError

    def _get_current_object() -> T:
        try:
            obj = local.get()
        except LookupError:
>           raise RuntimeError(unbound_message) from None
E           RuntimeError: Working outside of operation context. Make sure your app is wrapped in a ContextMiddleware and you're processing a request while accessing the context.

.tox/py39/lib/python3.9/site-packages/werkzeug/local.py:508: RuntimeError

This issue seems not to be specific to connexion only. It's more of a flask thing:

• https://github.com/spec-first/connexion/issues/1824
• https://stackoverflow.com/questions/65744087/how-to-fix-runtimeerror-working-outside-of-application-context-when-creating

This week is RelEng's meet-up on taskgraph. I don't think I'll have enough time to upgrade connexion this week. That's why I posted my progress for posterity.