1660197 - All websocket mochitests fails when running locally after python env update

Status: RESOLVED DUPLICATE of bug 1660128

(Testing :: Mochitest, defect)

Description

[Julian Descottes [:jdescottes]]

After the python env changes from Bug 1656993 the test https://searchfox.org/mozilla-central/source/devtools/shared/resources/tests/browser_resources_websocket.js fails locally.

It times out after outputting could not open connection to the real server. It relies on a python helper file: https://searchfox.org/mozilla-central/source/devtools/shared/resources/tests/websocket_backend_wsh.py

from __future__ import absolute_import
from mod_pywebsocket import msgutil


def web_socket_do_extra_handshake(request):
    pass


def web_socket_transfer_data(request):
    while not request.client_terminated:
        resp = msgutil.receive_message(request)
        msgutil.send_message(request, resp)


def web_socket_passive_closing_handshake(request):
    # If we use `pass` here, the `payload` of `frameReceived` which will be happened
    # of communication of closing will be `\u0003è`. In order to make the `payload`
    # to be empty string, return code and reason explicitly.
    code = None
    reason = None
    return code, reason

Comment 1

[Julian Descottes [:jdescottes]]

Note: all the websocket netmonitor tests are also broken: https://searchfox.org/mozilla-central/search?q=devtools%2Fclient%2Fnetmonitor%2Ftest%2Fbrowser_net_ws&case=true&path=

As well as most of the tests from dom/websocket https://searchfox.org/mozilla-central/source/dom/websocket/

Comment 2

[Julian Descottes [:jdescottes]]

Joel: all the tests using pywebsocket seem to fail locally but not on try.
Any idea why the recent python env changes have regressed this?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I can reproduce this, but I cannot find where we call websocket_backend_wsh.py anywhere. Can you point me to this?

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1656993

Comment 5

[Julian Descottes [:jdescottes]]

From what I can piece together mod_pywebsocket will append _wsh.py here: https://searchfox.org/mozilla-central/rev/6cc48251bb97600fdf11a5b4c5f621bfc8606d55/testing/web-platform/tests/tools/third_party/pywebsocket3/mod_pywebsocket/dispatch.py#110-119

def _enumerate_handler_file_paths(directory):
    """Returns a generator that enumerates WebSocket Handler source file names
    in the given directory.
    """

    for root, unused_dirs, files in os.walk(directory):
        for base in files:
            path = os.path.join(root, base)
            if _SOURCE_PATH_PATTERN.search(path):
                yield path

with

_SOURCE_PATH_PATTERN = re.compile(r'(?i)_wsh\.py$')
_SOURCE_SUFFIX = '_wsh.py'

Then individual tests just create websockets via:

        webSocket = new WebSocket(
          "ws://mochi.test:8888/browser/devtools/shared/resources/tests/websocket_backend"
        );

and websocket_backend becomes websocket_backend_wsh.py. Although I don't know much about the way mod_pywebsocket works

Comment 6

[Julian Descottes [:jdescottes]]

Or I guess it's the opposite, it identifies websocket handlers by looking for files ending with "_wsh.py" and then tests just have to access them via the path without "_wsh.py"

Comment 7

[Andrew Halberstadt [:ahal]]

Is it related to the fact that we install these dependencies on the fly here?
https://searchfox.org/mozilla-central/source/testing/mochitest/mach_commands.py#372

Though this test doesn't seem like it's in the mochitest-media suite, so maybe not. Unless it was only previously working for you by accident (i.e you happened to run a mochitest-media test prior to the devtools one)?

Comment 8

[Julian Descottes [:jdescottes]]

Though this test doesn't seem like it's in the mochitest-media suite, so maybe not. Unless it was only previously working for you by accident (i.e you happened to run a mochitest-media test prior to the devtools one)?

The test was running fine in isolation before.

Also, even though we spotted the issue on this specific devtools test, it impacts all the tests using mod_pywebsocket, including the ones in dom/websocket: https://searchfox.org/mozilla-central/source/dom/websocket/

Comment 9

[Ricky Stewart]

I haven't looked closely yet, but is this a dupe of bug 1660128 (with the same fix as suggested by me in the comment)?

Comment 10

[Ricky Stewart]

This test works fine for me (doesn't time out, concludes elegantly with a 0 exit code) after the patch in bug 1660128, so I'm inclined to say this is a dupe.