1764947 - Can't run mochitests on Windows - AssertionError - assert pathAbs.startswith(self.testRootAbs)

Status: RESOLVED FIXED

(Testing :: General, defect, P1)

Description

[:Gijs (he/him)]

$ ./mach test browser/base/content/test/general/browser_unknownContentType_title.js

produces:

Test configuration changed. Regenerating backend.
 0:10.21 INFO These variables are available in the mozinfo environment and can be used to skip tests conditionally:
 0:10.21 INFO     a11y_checks: False
 0:10.21 INFO     apple_catalina: False
 0:10.21 INFO     apple_silicon: False
 0:10.21 INFO     appname: firefox
 0:10.21 INFO     artifact: False
 0:10.21 INFO     asan: False
 0:10.21 INFO     automation: False
 0:10.21 INFO     bin_suffix: .exe
 0:10.21 INFO     bits: 64
 0:10.21 INFO     buildapp: browser
 0:10.21 INFO     buildtype_guess: opt
 0:10.21 INFO     cc_type: clang-cl
 0:10.21 INFO     ccov: False
 0:10.21 INFO     crashreporter: True
 0:10.21 INFO     datareporting: True
 0:10.21 INFO     debug: False
 0:10.21 INFO     devedition: False
 0:10.21 INFO     domstreams: True
 0:10.21 INFO     e10s: True
 0:10.21 INFO     early_beta_or_earlier: True
 0:10.21 INFO     fission: True
 0:10.21 INFO     has_sandbox: True
 0:10.21 INFO     headless: False
 0:10.21 INFO     healthreport: True
 0:10.21 INFO     isolated_process: False
 0:10.21 INFO     mozconfig: C:/dev/config/opt
 0:10.21 INFO     nightly_build: True
 0:10.21 INFO     non_native_theme: True
 0:10.21 INFO     normandy: True
 0:10.21 INFO     official: False
 0:10.21 INFO     os: win
 0:10.21 INFO     os_version: 10.0
 0:10.21 INFO     pgo: False
 0:10.21 INFO     platform_guess: win64
 0:10.21 INFO     processor: x86_64
 0:10.21 INFO     release_or_beta: False
 0:10.21 INFO     require_signing: False
 0:10.21 INFO     service_pack:
 0:10.21 INFO     serviceworker_e10s: True
 0:10.21 INFO     sessionHistoryInParent: True
 0:10.21 INFO     socketprocess_e10s: False
 0:10.21 INFO     socketprocess_networking: False
 0:10.21 INFO     stylo: True
 0:10.21 INFO     swgl: False
 0:10.21 INFO     sync: True
 0:10.21 INFO     telemetry: False
 0:10.21 INFO     tests_enabled: True
 0:10.21 INFO     toolkit: windows
 0:10.21 INFO     topsrcdir: C:/dev/mozilla-unified
 0:10.21 INFO     tsan: False
 0:10.21 INFO     ubsan: False
 0:10.21 INFO     updater: True
 0:10.21 INFO     verify: False
 0:10.21 INFO     verify_fission: False
 0:10.21 INFO     version: 10.0.19044
 0:10.21 INFO     webgl_ipc: False
 0:10.21 INFO     win10_2004: False
 0:10.21 INFO     xorigin: False
 0:10.21 INFO Checking for ssltunnel processes...
 0:10.50 INFO Checking for xpcshell processes...
Error running mach:

    ['test', 'browser/base/content/test/general/browser_unknownContentType_title.js']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file test| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

AssertionError

  File "C:\dev\mozilla-unified\testing\mach_commands.py", line 457, in test
    res = command_context._mach_context.commands.dispatch(
  File "C:\dev\mozilla-unified\python\mach\mach\registrar.py", line 185, in dispatch
    return self._run_command_handler(handler, context, **kwargs)
  File "C:\dev\mozilla-unified\python\mach\mach\registrar.py", line 118, in _run_command_handler
    result = fn(instance, **kwargs)
  File "C:\dev\mozilla-unified\testing\mochitest\mach_commands.py", line 502, in run_mochitest_general
    result = run_mochitest(
  File "C:\dev\mozilla-unified\testing\mochitest\mach_commands.py", line 147, in run_desktop_test
    result = mochitest.run_test_harness(parser, options)
  File "c:/dev/builds/opt\_tests\testing\mochitest\runtests.py", line 3739, in run_test_harness
    result = runner.runTests(options)
  File "c:/dev/builds/opt\_tests\testing\mochitest\runtests.py", line 3053, in runTests
    tests = self.getActiveTests(options)
  File "c:/dev/builds/opt\_tests\testing\mochitest\runtests.py", line 1653, in getActiveTests
    assert pathAbs.startswith(self.testRootAbs)

Sentry event ID: a25147c432a54c888d7c3c913cc0b261

This is on current central (7f6fc25d231f) but was also broken on yesterday's central. I believe it used to work. I'm supposed to be on PTO so I don't have much time to investigate further.

The same error appears when passing just a test file name (rather than a full path to a test), a manifest, or a test directory. It also doesn't work with the mach mochitest command.

I thought this used to work but can't be certain if I ran tests after updating to mozillabuild 4.0.

This appears similar to bug 1752315 except I am not using --appname and have already run a complete ./mach build. I tried clobbering, that didn't help either.

Comment 1

[:Gijs (he/him)]

Echo'ing that comparison produces:

c:\dev\builds\opt\_tests\testing\mochitest\browser\browser\base\content\test\general\browser_unknownContentType_title.js with C:\dev\builds\opt\_tests\testing\mochitest\browser

So this is yet more fallout from casing of drive letters on Windows.

I don't understand why these failures weren't caught by anyone else.

Comment 2

[:Gijs (he/him)]

Something like this:

https://paste.mozilla.org/Vvg6XfOQ

allows me to run tests locally, at least, though it's probably not the right fix.

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1764947: Normcase mochitest paths for "parent dir" comparison

MozillaBuild 4.0 changed the default drive letter case to be uppercase,
causing some havoc for in-tree code that assumes that it'll still be
lowercase.

Address this new mismatched expectation with a couple os.path.normcase
calls.

This class of issue will be solved in a cleaner, less footgun-y way by
bug 1740564.

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

(In reply to :Gijs (out, back April 27; he/him) from comment #2)

Something like this:

https://paste.mozilla.org/Vvg6XfOQ

allows me to run tests locally, at least, though it's probably not the right fix.

I think that's pretty much perfect, though I'm using os.path.normcase() in my patch to be slightly more aligned with Python conventions.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/27f07d84c825
Normcase mochitest paths for "parent dir" comparison r=ahal

Comment 6

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/27f07d84c825