1265124 - "mach reftest" no longer works with --debugger=valgrind

Status: RESOLVED FIXED

(Testing :: Reftest, defect)

Description

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

./mach reftest no longer works for me with --debugger=valgrind -- there's an error from marionnette.  I can't even find the bug where we converted reftest to use this stuff to make this block that bug, although I vaguely remember reviewing patches in it.

It looks like stuff is timing out because we assume Firefox starts up within a certain time, but there isn't a way to change those assumptions.

I get:


Error running mach:

    ['reftest', '--debugger', 'valgrind', '--debugger-args=--tool=memcheck --leak-check=no --trace-children=yes --num-callers=50 --smc-check=all-non-file --vex-iropt-register-updates=allregs-at-mem-access --error-limit=no --fullpath-after=', 'layout/reftests/table-bordercollapse/borderhandling-rules-border-groups-strict.html']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

error: [Errno 111] Connection refused

  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/layout/tools/reftest/mach_commands.py", line 326, in run_reftest
    return self._run_reftest(**kwargs)
  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/layout/tools/reftest/mach_commands.py", line 374, in _run_reftest
    return reftest.run_desktop_test(**kwargs)
  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/layout/tools/reftest/mach_commands.py", line 208, in run_desktop_test
    rv = runreftest.run(**kwargs)
  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/layout/tools/reftest/runreftest.py", line 717, in run
    return reftest.runTests(options.tests, options)
  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/layout/tools/reftest/runreftest.py", line 420, in runTests
    return self.runSerialTests(manifests, options, cmdargs)
  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/layout/tools/reftest/runreftest.py", line 671, in runSerialTests
    debuggerInfo=debuggerInfo)
  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/layout/tools/reftest/runreftest.py", line 612, in runApp
    marionette.start_session()
  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/testing/marionette/client/marionette_driver/marionette.py", line 1166, in start_session
    self.protocol, _ = self.client.connect()
  File "/home/dbaron/builds/ssd/mozilla-central/mozilla/testing/marionette/client/marionette_driver/transport.py", line 226, in connect
    self.sock.connect((self.addr, self.port))
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)

Comment 1

[Andrew Halberstadt [:ahal]]

Sorry for late reply, just got back from PTO. This is similar to bug 1248056 which was filed for mochitest, the fix here should be the same.

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: MozReview Request: Bug 1265124 - [reftest] Make sure marionette waits long enough when attaching a debugger, r?chmanchester

As of bug 1245092, reftest depends on marionette. Normally marionette client has an internal timeout of around
60 seconds where it waits for the server to become available. But when using debuggers (and especially valgrind),
it can take much much longer than a minute for Firefox to start. This patch adds a couple hidden command line
args to reftest to tweak the marionette timeouts.

Review commit: https://reviewboard.mozilla.org/r/49805/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/49805/

Comment 3

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8747278 [details]
MozReview Request: Bug 1265124 - [reftest] Make sure marionette waits long enough when attaching a debugger, r?chmanchester

https://reviewboard.mozilla.org/r/49805/#review46549

::: layout/tools/reftest/reftestcommandline.py:355
(Diff revision 1)
> +            options.marionette_port_timeout = 900
> +            options.marionette_socket_timeout = 540

Is it necessary to bump both of these? It looks like we're hitting the port timeout, and the socket timeout is already pretty high. Anyway, this is safe.

Comment 4

[Andrew Halberstadt [:ahal]]

https://reviewboard.mozilla.org/r/49805/#review46549

> Is it necessary to bump both of these? It looks like we're hitting the port timeout, and the socket timeout is already pretty high. Anyway, this is safe.

These are the numbers jseward needed when testing the same problem for mochitest. I think he was actually able to hit the socket timeout on his machine as well as the port timeout. Could be mistaken, but better safe than sorry I guess.

Comment 5

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a17e8323ae15

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/a17e8323ae15