./mach test can't run web-platform-tests on Windows

NEW
Unassigned

Status

defect
6 months ago
4 months ago

People

(Reporter: birtles, Unassigned)

Tracking

Version 3
x86_64
Windows 10
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Reporter

Description

6 months ago
On a fresh checkout of mozilla-central on Windows 10 I get the following:

  /c/moz/src1 (debug-opt)
  $ ./mach test testing/web-platform/tests/web-animations/
  cat: backend.TestManifestBackend.in: No such file or directory
  Build configuration changed. Regenerating backend.
  No handlers could be found for logger "mozbuild.frontend.reader"
  UNKNOWN TEST: testing/web-platform/tests/web-animations/
  I was unable to find tests from the given argument(s).

  You should specify a test directory, filename, test suite name, or
  abbreviation. If no arguments are given, there must be local file
  changes and corresponding IMPACTED_TESTS annotations in moz.build
  files relevant to those files.

  It's possible my little brain doesn't know about the type of test you are
  trying to execute. If you suspect this, please request support by filing
  a bug at
  https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=General.

However if I use `./mach wpt` instead of `./mach test` it works. On Linux both work.
Reporter

Comment 1

6 months ago
This is probably a regression since I seem to recall this used to work on Windows.
For this test, I used `./mach test testing/web-platform/tests/web-animations/timing-model/animation-effects` on both OSX and Windows10.

So in this check[1], `path` is "testing/web-platform/tests/web-animations/timing-model/animation-effects" on both OSX and Windows, `path in self._test_dirs` is False on both, and `any(p.startswith(path) for p in self._tests_by_path)` is True on OSX but False on Windows. 

The keys in self._tests_by_path on OSX look like 'testing/web-platform/tests/foo/bar/baz.html'. On Windows, those same keys look like 'testing\\web-platform\\tests\\foo\\bar\\baz.html'.

What would be the preferred way of fixing this? Converting `path` to the Windows-escaped paths that are in _tests_by_path?



1. https://searchfox.org/mozilla-central/source/testing/mozbase/moztest/moztest/resolve.py#427-428
Flags: needinfo?(james)
FWIW, `./mach test testing/web-platform/tests/web-animations` works if I add `path = path.replace("/", "\\")`before line 427, but obviously that would break non-windows.
Yeah, but path.replace("\", os.path.sep) should work. It might be more efficient to do something like

if os.path.sep != "/":
    path = path.replace("/", os.path.sep)
Flags: needinfo?(james)
Comment on attachment 9033679 [details]
Bug 1515245 - Make sure windows paths are supported for wpt via generic 'mach test' r?jgraham

This works for me on mac and windows.
Attachment #9033679 - Flags: review?(james)
Assignee: nobody → wkocher

Comment 7

6 months ago
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/1923d02bb66d
Make sure windows paths are supported for wpt via generic 'mach test' r=jgraham

Comment 9

6 months ago
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8040640ee1cd
Make sure windows paths are supported for wpt via generic 'mach test' r=jgraham
Attachment #9033679 - Flags: review?(james)
Hmm, so maybe the right fix here is to ensure that all the wpt paths are added with unix-style path seperators irrespective of the actual platform.

KWierso and jgraham:

This is really annoying bug to me. What's the status of this bug?

Flags: needinfo?(james)

I don't see myself getting back to this anytime soon. I think a workaround would be to use ./mach wpt path/to/tests rather than ./mach test path/to/tests

Assignee: wkocher → nobody
Flags: needinfo?(wkocher)

I don't have Windows installed at the moment to investigate this, but I'm about 90% sure that the problem will be in [1] where we add the list of wpt tests to the manifest used by mach test. My prediction is that something in the output of that loop has different path separator characters to the entries from other testsuites, or maybe just src_path has the wron path seperators. I think if you compare the items we add there to other items in the _tests_by_path dict then it should be obvious where the bug is.

[1] https://searchfox.org/mozilla-central/source/testing/mozbase/moztest/moztest/resolve.py#482

Flags: needinfo?(james)

(In reply to Wes Kocher (:KWierso) from comment #13)

I don't see myself getting back to this anytime soon. I think a workaround would be to use ./mach wpt path/to/tests rather than ./mach test path/to/tests

Ah, I might have met different bug... Even if I useed ./mach wpt ... on Windows, I couldn't run the test actually. However, today, I succeeded to run a WPT on my machine. So, the bug which I met has already been fixed by somebody.

(And ./mach test <a path to WPT> still doesn't work exactly same as this bug.)

You need to log in before you can comment on or make changes to this bug.