964102 - mach can no longer run xpcshell tests for a specific directory

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Mark Hammond [:markh] [:mhammond]]

eg, after a successful build:

"""
$ ./mach xpcshell-test services/sync/tests/unit/
From _tests: Kept 14480 existing; Added/updated 1; Removed 0 files and 0 directories.
 0:06.15 Found node at c:\src\mozilla-git\gecko-dev\testing\xpcshell\node
 0:06.15 Found moz-spdy at c:\src\mozilla-git\gecko-dev\testing\xpcshell\moz-spdy\moz-spdy.js
 0:06.15 Could not run moz-spdy server: [Error 193] %1 is not a valid Win32 application
 0:06.15 Found moz-http2 at c:\src\mozilla-git\gecko-dev\testing\xpcshell\moz-http2\moz-http2.js
 0:06.15 Could not run moz-http2 server: [Error 193] %1 is not a valid Win32 application
 0:06.21 INFO | Using at most 16 threads.
 0:06.21 TEST-INFO | test_addons_reconciler.js | Test failed or timed out, will retry.
<snip lots of test names with the same error message>
 0:06.27 Retrying tests that failed when run in parallel.
 0:06.27 INFO | Following exceptions were raised:
 0:06.27 Traceback (most recent call last):
  File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 145, in run
    self.run_test()
  File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 537, in run_test
    self.test_object['here'])
Exception: tests_root_dir is not a parent path of c:/src/mozilla-git/gecko-dev/obj-release/_tests/xpcshell/services/sync/tests/unit

 0:06.27 Traceback (most recent call last):
  File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 145, in run
    self.run_test()
  File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 537, in run_test
    self.test_object['here'])
Exception: tests_root_dir is not a parent path of c:/src/mozilla-git/gecko-dev/obj-release/_tests/xpcshell/services/sync/tests/unit

Error running mach:

    ['xpcshell-test', 'services/sync/tests/unit/']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: tests_root_dir is not a parent path of c:/src/mozilla-git/gecko-dev/obj-release/_tests/xpcshell/services/sync/tests/unit

  File "c:\src\mozilla-git\gecko-dev\testing/xpcshell/mach_commands.py", line 339, in run_xpcshell_test
    return xpcshell.run_test(**params)
  File "c:\src\mozilla-git\gecko-dev\testing/xpcshell/mach_commands.py", line 102, in run_test
    return self._run_xpcshell_harness(**args)
  File "c:\src\mozilla-git\gecko-dev\testing/xpcshell/mach_commands.py", line 187, in _run_xpcshell_harness
    result = xpcshell.runTests(**filtered_args)
  File "c:\src\mozilla-git\gecko-dev\testing/xpcshell\runxpcshelltests.py", line 1483, in runTests
    raise exceptions[0]

/c/src/mozilla-git/gecko-dev - .mozconfig-release - (fxa-sync)
$
"""

Trying to specify a single file in that directory produces the same basic error.  Trying to specify a manifest file:
"""
$ ./mach xpcshell-test services/sync/tests/unit/xpcshell.ini
From _tests: Kept 14480 existing; Added/updated 1; Removed 0 files and 0 directories.
We could not find an xpcshell test for the passed test path. Please select a path that is a test file or is a directory containing xpcshell tests.
"""

Running 'xpcshell-tests' with no arguments works as expected.  I've tried this with 2 different xpcshell test directories.

environment:

$ ./mach environment
platform:
        Windows-8-6.2.9200
python version:
        2.7.4 (default, Apr  6 2013, 19:54:46) [MSC v.1500 32 bit (Intel)]
python prefix:
        c:\mozilla-build\python
mach cwd:
        c:\src\mozilla-git\gecko-dev
os cwd:
        c:\src\mozilla-git\gecko-dev
mach directory:
        c:\src\mozilla-git\gecko-dev
state directory:
        c:/home/.mozbuild
mozconfig path:
        c:\src\mozilla-git\gecko-dev\.mozconfig-release
object directory:
        c:\src\mozilla-git\gecko-dev\obj-release
mozconfig configure args:
        --enable-application=browser
        --enable-tests
        --enable-jemalloc
        --enable-signmar
        --enable-profiling
        --enable-js-diagnostics
mozconfig extra make args:
mozconfig make flags:
        -
        j
        6
config topsrcdir:
        c:/src/mozilla-git/gecko-dev
config topobjdir:
        c:/src/mozilla-git/gecko-dev/obj-release

Comment 1

[Mark Hammond [:markh] [:mhammond]]

Attachment: 0011-Bug-964102-use-normpath-on-the-test-path-before-comp.patch

It turns out that line 534:
  self.tests_root_dir = os.path.normpath(self.tests_root_dir)

causes self.tests_root_dir to have backslashes, while self.test_object['here'] has forward slashes.  Using .normpath on the test path seems to solve the problem (as, presumably, would using posix.normpath on the root dir)

Comment 2

[Gregory Szorc [:gps]]

Comment on attachment 8365723 [details] [diff] [review]
0011-Bug-964102-use-normpath-on-the-test-path-before-comp.patch

Review of attachment 8365723 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/xpcshell/runxpcshelltests.py
@@ +531,5 @@
>          # dir and the test with path characters replaced with '.' (using Java
>          # class notation).
>          if self.tests_root_dir is not None:
>              self.tests_root_dir = os.path.normpath(self.tests_root_dir)
> +            if os.path.normpath(self.test_object['here']).find(self.tests_root_dir) != 0:

You are on the right track. We should use mozpack.mozpath.normpath for all path comparisons. (That's effectively posixpath.normpath.)

Comment 4

[Gregory Szorc [:gps]]

Comment on attachment 8365723 [details] [diff] [review]
0011-Bug-964102-use-normpath-on-the-test-path-before-comp.patch

Review of attachment 8365723 [details] [diff] [review]:
-----------------------------------------------------------------

Ignore my comment. all-tests.json doesn't store normalized paths. This is definitely a follow-up waiting to be filed. This is the correct solution given the current implementation.

Comment 5

[Gregory Szorc [:gps]]

I'd land this, but the trees are closed. Since this is developer impacting, we may want to land in all of the trees, or at least get things merged quickly.

Comment 6

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/31253b381b31

Comment 7

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/31253b381b31