1572122 - Can't run web-platform-tests with rr any more

Status: NEW

(Testing :: web-platform-tests, defect, P2)

Description

[(no longer active)]

It seems that Firefox is launched the first time, and sits there idle without doing anything, and is then terminated and launched a second time and that time the test harness can't connect to it. Reproducible 100% of the time. Here is the console output:

ehsan@teenux:~/moz/src$ ./mach wpt /cookies/http-state/comma-tests.html --debugger=rr
 0:00.02 INFO Skipping manifest download because existing file is recent
 0:02.50 mozversion INFO application_buildid: 20190807101314
 0:02.50 mozversion INFO application_display_name: Nightly
 0:02.50 mozversion INFO application_id: {ec8030f7-c20a-464f-9b0e-13a3a9e97384}
 0:02.50 mozversion INFO application_name: Firefox
 0:02.50 mozversion INFO application_remotingname: firefox
 0:02.50 mozversion INFO application_vendor: Mozilla
 0:02.50 mozversion INFO application_version: 70.0a1
 0:02.50 mozversion INFO platform_buildid: 20190807101314
 0:02.50 mozversion INFO platform_version: 70.0a1
 0:03.00 INFO Using 1 client processes
 0:03.08 INFO Starting http server on 127.0.0.1:8000
 0:03.09 INFO Starting http server on 127.0.0.1:8001
 0:03.11 INFO Starting https server on 127.0.0.1:8443
 0:03.59 SUITE_START: web-platform-test - running 1 tests
 0:03.59 INFO Running reftest tests
 0:03.60 INFO No reftest tests to run
 0:03.60 INFO Running wdspec tests
 0:03.60 INFO No wdspec tests to run
 0:03.60 INFO Running testharness tests
 0:03.63 INFO Setting up ssl
 0:03.69 certutil Full command: /home/ehsan/moz/src/obj-ff-opt/dist/bin/certutil -N -d /tmp/tmpXI4EjJ.mozrunner -f /tmp/tmpXI4EjJ.mozrunner/.crtdbpw
certutil
 0:03.76 certutil
 0:03.78 certutil
Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

web-platform-tests                                           CT,,

 0:03.78 INFO Application command: /usr/local/bin/rr /home/ehsan/moz/src/obj-ff-opt/dist/bin/firefox --marionette about:blank -profile /tmp/tmpXI4EjJ.mozrunner
 0:03.79 INFO Starting runner
 0:06.31 pid:12371 Full command: /home/ehsan/moz/src/obj-ff-opt/dist/bin/firefox --marionette about:blank -profile /tmp/tmpXI4EjJ.mozrunner
pid:12371 1565190638294 [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: mozillaAddons
 0:06.31 pid:12371 1565190638294        [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: telemetry
 0:06.31 pid:12371 1565190638295        [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: resource://pdf.js/
 0:06.31 pid:12371 1565190638295        [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: about:reader*
 0:19.28 pid:12371 1565190651263        Marionette      INFO    Listening on port 53345
 6:19.56 WARNING Failed to start protocol connection
 6:19.59 WARNING Traceback (most recent call last):
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/protocol.py", line 48, in setup
    self.connect()
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 529, in connect
    self.marionette.start_session(self.capabilities)
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/decorators.py", line 36, in _
    m._handle_socket_failure()
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/marionette.py", line 650, in _handle_socket_failure
    reraise(exc, val, tb)
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/decorators.py", line 26, in _
    return func(*args, **kwargs)
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/marionette.py", line 1116, in start_session
    resp = self._send_message("WebDriver:NewSession", capabilities)
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/decorators.py", line 36, in _
    m._handle_socket_failure()
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/marionette.py", line 650, in _handle_socket_failure
    reraise(exc, val, tb)
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/decorators.py", line 26, in _
    return func(*args, **kwargs)
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/marionette.py", line 590, in _send_message
    msg = self.client.request(name, params)
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/transport.py", line 273, in request
    return self.receive()
  File "/home/ehsan/moz/src/testing/marionette/client/marionette_driver/transport.py", line 183, in receive
    raise socket.timeout("Connection timed out after {}s".format(self.socket_timeout))
timeout: Connection timed out after 360s

 6:19.69 INFO Browser exited with return code -15
 6:19.69 INFO PROCESS LEAKS None
 6:19.69 INFO Closing logging queue
 6:19.69 INFO queue closed
 6:19.71 INFO Setting up ssl
 6:19.77 certutil
 6:19.83 certutil
 6:19.86 certutil
Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

web-platform-tests                                           CT,,

 6:19.88 INFO Application command: /usr/local/bin/rr /home/ehsan/moz/src/obj-ff-opt/dist/bin/firefox --marionette about:blank -profile /tmp/tmptyDRFK.mozrunner
 6:19.89 INFO Starting runner
 6:22.10 pid:12778 Full command: /home/ehsan/moz/src/obj-ff-opt/dist/bin/firefox --marionette about:blank -profile /tmp/tmptyDRFK.mozrunner
pid:12778 1565191014081 [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: mozillaAddons
 6:22.10 pid:12778 1565191014082        [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: telemetry
 6:22.10 pid:12778 1565191014082        [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: resource://pdf.js/
 6:22.10 pid:12778 1565191014082        [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: about:reader*
 6:34.96 pid:12778 1565191026941        Marionette      INFO    Listening on port 53345
 6:34.96 pid:12778 1565191026941        Marionette      INFO    Listening on port 53345
 6:46.44 TEST_START: /cookies/http-state/comma-tests.html
 6:58.59 WARNING Traceback (most recent call last):
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 627, in _run
    self.result = True, self.func(self.protocol, self.url, self.timeout)
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 735, in do_testharness
    timeout=10*self.timeout_multiplier)
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 226, in get_test_window
    raise Exception("unable to find test window")
Exception: unable to find test window

 6:58.59 TEST_END: ERROR, expected OK - unable to find test window
Traceback (most recent call last):
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 627, in _run
    self.result = True, self.func(self.protocol, self.url, self.timeout)
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 735, in do_testharness
    timeout=10*self.timeout_multiplier)
  File "/home/ehsan/moz/src/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 226, in get_test_window
    raise Exception("unable to find test window")
Exception: unable to find test window

 6:58.59 INFO Pausing until the browser exits
^C 7:19.04 pid:12778 1565191071024      Marionette      INFO    Stopped listening on port 53345
 7:19.73 INFO Browser exited with return code -15
 7:19.73 INFO PROCESS LEAKS None
 7:19.73 INFO Closing logging queue
 7:19.73 INFO queue closed
 7:19.73 WARNING Command left in remote_queue during cleanup: u'stop', ()
 7:19.73 INFO PROCESS LEAKS None
 7:19.73 CRITICAL Main thread got signal
 7:19.76 INFO Closing logging queue
 7:19.76 INFO queue closed
mach interrupted by signal or user action. Stopping.

Comment 1

[James Graham [:jgraham]]

Are you able to use marionette in general under rr? I don't think anything relevant has changed on the wpt side.