1726740 - ./mach mozregression --command "./mach mochitest --appname {binary}" is broken

Status: NEW

(Testing :: Mochitest, defect, P3)

Description

[Andreas Farre [:farre]]

The problem seems to be that the xpcshell we fire up tries to use the libxul.so from the downloaded dist, but the xpcshell we start is from the checked out code.

[farre@oberon work]$ ./mach mozregression --launch 2021-05-16 --command "./mach mochitest --appname {binary} dom/tests/browser/browser_sessionStorage_navigation.js"

---

You should use a config file. Please use the --write-config command line flag to help you create one.

---

0:02.08 INFO: Downloading build from: https://archive.mozilla.org/pub/firefox/nightly/2021/05/2021-05-16-09-17-48-mozilla-central/firefox-90.0a1.en-US.linux-x86_64.tar.bz2
===== Downloaded 100% =====
0:07.75 INFO: Running mozilla-central build for 2021-05-16
0:16.06 INFO: application_buildid: 20210516091748
0:16.06 INFO: application_changeset: 3210b5354d3ee3ba3d3ff98e8e791409909d9db7
0:16.06 INFO: application_name: Firefox
0:16.06 INFO: application_repository: https://hg.mozilla.org/mozilla-central
0:16.06 INFO: application_version: 90.0a1
0:16.06 INFO: Running test command: ./mach mochitest --appname /tmp/tmpxl3birxg/firefox/firefox dom/tests/browser/browser_sessionStorage_navigation.js
/home/farre/src/work/obj-linux-release/backend.TestManifestBackend is out of date with respect to /home/farre/src/work/obj-linux-release/_tests/testing/mochitest/runtests.py
Test configuration changed. Regenerating backend.
0:01.28 INFO Checking for ssltunnel processes...
0:01.30 INFO Checking for xpcshell processes...
0:01.32 SUITE_START: mochitest-browser - running 1 tests
0:01.32 INFO Running manifest: dom/tests/browser/browser.ini
0:01.49 pid:49086 Full command: /home/farre/src/work/obj-linux-release/dist/bin/pk12util -i /home/farre/src/work/build/pgo/certs/mochitest.client -w /tmp/tmpb1jr_hiy.mozrunner/.crtdbpw -d /tmp/tmpb1jr_hiy.mozrunner
pid:49086 pk12util: PKCS12 IMPORT SUCCESSFUL
0:01.51 INFO MochitestServer : launching ['/home/farre/src/work/obj-linux-release/dist/bin/xpcshell', '-g', '/tmp/tmpxl3birxg/firefox', '-f', '/home/farre/src/work/obj-linux-release/dist/bin/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpb1jr_hiy.mozrunner'; const _SERVER_PORT = '8888'; const _SERVER_ADDR = '127.0.0.1'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/home/farre/src/work/obj-linux-release/_tests/testing/mochitest/server.js']
0:01.51 INFO runtests.py | Server pid: 49089
/home/farre/src/work/obj-linux-release/dist/bin/xpcshell: /tmp/tmpxl3birxg/firefox/libxul.so: version `xul93a1' not found (required by /home/farre/src/work/obj-linux-release/dist/bin/xpcshell)
0:01.52 INFO runtests.py | Websocket server pid: 49092
0:01.53 INFO runtests.py | SSL tunnel pid: 49096
^C 0:14.73 INFO runtests.py | Received keyboard interrupt.

Comment 1

[Andreas Farre [:farre]]

Adding

diff --git a/testing/mochitest/runtests.py b/testing/mochitest/runtests.py
index fbedb1f37f73..8e4f1c027fc6 100644
--- a/testing/mochitest/runtests.py
+++ b/testing/mochitest/runtests.py
@@ -541,9 +541,14 @@ class MochitestServer(object):
         xpcshell = os.path.join(
             self._utilityPath, "xpcshell" + mozinfo.info["bin_suffix"]
         )
+        oldenv = env["LD_LIBRARY_PATH"]
+        env["LD_LIBRARY_PATH"] = ":".join([self._utilityPath, env["LD_LIBRARY_PATH"]])
         command = [xpcshell] + args
         self._process = mozprocess.ProcessHandler(command, cwd=SCRIPT_DIR, env=env)
         self._process.run()
+        env["LD_LIBRARY_PATH"] = oldenv;
         self._log.info("%s : launching %s" % (self.__class__.__name__, command))
         pid = self._process.pid
         self._log.info("runtests.py | Server pid: %d" % pid)

but I have no idea if this is even close to the correct solution. :ahal, does this seem like a way forward.

This prevents using mozregression with mochitest.

Comment 2

[Andrew Halberstadt [:ahal]]

Using a separate binary in mochitest is not very user friendly. I don't recall exactly what's needed, but you need to pass much more than just --appname. At least also --utility-path and --certificate-path, but probably more. You can take a look at the command from a mochitest log to get a hint here:
https://treeherder.mozilla.org/logviewer?job_id=349062562&repo=autoland&lineNumber=2183

It would be great if mochitest could autodetect everything given a standard tests.zip, but I don't think anyone has the bandwidth to implement something like that atm. If you'd like to attempt it, I'd be happy to provide guidance. Though in the context of mozregression, it would be awhile before you could use such a feature anyway :)