1189561 - Parse error when attempt to use mach mochitest without a directory name

Status: NEW

(Testing :: Mochitest, defect)

Description

[Andrew McCreight [:mccr8]]

I'm trying to use mach to bisect where a certain failure is happening in mochitest-plain, but it produces a parse error when I do

./mach mochitest -f plain --total-chunks 5 --this-chunk 3

The error is:

Error running mach:

    ['mochitest', '-f', 'plain', '--total-chunks', '5', '--this-chunk', '3']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ParseError: could not parse: (buildapp == 'b2g' || e10s) || (toolkit == "gtk2" || toolkit == "gtk3" || (os == 'mac' && os_version != '10.6') ); variables: {u'topsrcdir': u'/home/amccreight/mc', 'e10s': False, u'toolkit': u'gtk2', u'webm': True, u'buildapp': u'browser', u'crashreporter': False, u'platform_guess': u'linux64-asan', u'appname': u'firefox', u'mozconfig': u'/home/amccreight/asanconfig', 'has_sandbox': True, u'telemetry': False, 'os_version': StringVersion (''), 'version': ' ', u'datareporting': True, u'buildtype_guess': u'opt', 'bits': 64, u'bin_suffix': u'', 'strictContentSandbox': False, u'wave': True, u'healthreport': True, u'release_build': False, u'asan': True, u'tests_enabled': True, 'linux_distro': '', u'official': False, u'pgo': False, u'tsan': False, 'nested_oop': False, u'debug': False, 'os': u'linux', 'processor': u'x86_64'}

  File "/home/amccreight/mc/testing/mochitest/mach_commands.py", line 577, in run_mochitest_general
    **harness_args)
  File "/home/amccreight/mc/testing/mochitest/mach_commands.py", line 348, in run_desktop_test
    result = mochitest.run_test_harness(options)
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 2594, in run_test_harness
    result = runner.runTests(options)
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 2093, in runTests
    testsToRun = self.getTestsToRun(options)
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 2004, in getTestsToRun
    tests = self.getActiveTests(options)
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 1950, in getActiveTests
    exists=False, disabled=disabled, filters=filters, **info)
  File "/home/amccreight/mc/testing/mozbase/manifestparser/manifestparser/manifestparser.py", line 748, in active_tests
    tests = fn(tests, values)
  File "/home/amccreight/mc/testing/mozbase/manifestparser/manifestparser/filters.py", line 170, in __call__
    tests = list(tests)
  File "/home/amccreight/mc/testing/mozbase/manifestparser/manifestparser/filters.py", line 127, in __call__
    for test in tests:
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 1901, in <genexpr>
    return (t for t in tests
  File "/home/amccreight/mc/testing/mozbase/manifestparser/manifestparser/filters.py", line 53, in fail_if
    for test in tests:
  File "/home/amccreight/mc/testing/mozbase/manifestparser/manifestparser/filters.py", line 41, in run_if
    for test in tests:
  File "/home/amccreight/mc/testing/mozbase/manifestparser/manifestparser/filters.py", line 30, in skip_if
    if tag in test and parse(test[tag], **values):
  File "/home/amccreight/mc/testing/mozbase/manifestparser/manifestparser/expression.py", line 281, in parse
    return ExpressionParser(text, values).parse()
  File "/home/amccreight/mc/testing/mozbase/manifestparser/manifestparser/expression.py", line 267, in parse
    raise ParseError("could not parse: %s; variables: %s" % (self.text, self.valuemapping))



The weird thing is, it is clearly parsing it somewhere, as if I do total-chunks 1 then it complains that the total has to be at least as large as this-chunk.

Comment 1

[Andrew McCreight [:mccr8]]

Just not having a directory doesn't seem to be the problem as this doesn't work either, which about the same error message:

./mach mochitest -f plain --e10s --start-at dom/media/webspeech/synth/test/test_indirect_service_events.html --end-at dom/tests/mochitest/dom-level1-core/test_nodeentitysetnodevalue.html dom/

Comment 2

[Gregory Szorc [:gps]]

The "ParseError: could not parse" seems to indicate a malformed test manifest. Not sure which one though. I'm kinda surprised the moz.build manifest parsing isn't catching this. Does manifestparser not validate the skip expressions when loading the file? (Perhaps it shouldn't, as this would add overhead to moz.build processing.)

Comment 3

[Andrew McCreight [:mccr8]]

I also get this with ./mach mochitest browser/base/content/test/plugins/

From _tests: Kept 41078 existing; Added/updated 0; Removed 0 files and 0 directories.

######
### Now running mochitest-browser.
######

Checking for orphan ssltunnel processes...
Checking for orphan xpcshell processes...
Error running mach:

    ['mochitest', 'browser/base/content/test/plugins/']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ParseError: could not parse: (buildapp == "mulet") || ((os == 'win' && os_version == "6.2" && e10s) ); variables: {'e10s': False, u'toolkit': u'gtk3', u'webm': True, u'buildapp': u'browser', u'crashreporter': False, u'addon_signing': True, u'platform_guess': u'linux64-asan', 'has_sandbox': True, u'appname': u'firefox', u'mozconfig': u'/home/amccreight/asanconfig', u'topsrcdir': u'/home/amccreight/mc', u'telemetry': False, 'os_version': StringVersion (''), 'version': ' ', u'datareporting': True, u'buildtype_guess': u'opt', 'bits': 64, u'bin_suffix': u'', 'strictContentSandbox': False, u'wave': True, u'healthreport': True, u'release_build': False, u'asan': True, u'tests_enabled': True, 'linux_distro': '', u'official': False, u'pgo': False, u'tsan': False, 'nested_oop': False, u'debug': False, 'os': u'linux', 'processor': u'x86_64'}

  File "/home/amccreight/mc/testing/mochitest/mach_commands.py", line 588, in run_mochitest_general
    **harness_args)
  File "/home/amccreight/mc/testing/mochitest/mach_commands.py", line 339, in run_desktop_test
    result = mochitest.run_test_harness(options)
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 2601, in run_test_harness
    result = runner.runTests(options)
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 2096, in runTests
    testsToRun = self.getTestsToRun(options)
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 2009, in getTestsToRun
    tests = self.getActiveTests(options)
  File "/home/amccreight/mc/obj-asan-opt/_tests/testing/mochitest/runtests.py", line 1946, in getActiveTests

etc. etc.

Comment 4

[Andrew McCreight [:mccr8]]

Attachment: Fix manifest

This is the line that is breaking the parser. I don't know if this is the same as the original issue.

Comment 5

[Andrew McCreight [:mccr8]]

(In reply to Gregory Szorc [:gps] from comment #2)
> The "ParseError: could not parse" seems to indicate a malformed test
> manifest. Not sure which one though. I'm kinda surprised the moz.build
> manifest parsing isn't catching this. Does manifestparser not validate the
> skip expressions when loading the file? (Perhaps it shouldn't, as this would
> add overhead to moz.build processing.)

Thanks for explaining the issue. It would be nice if at least this could fail by telling you which line was invalid, even if we don't want to validate it all of the time. I had to bisect the skip-if lines to figure out what was at fault.