1168718 - ImportError: No module named filters with mochitest-plain

Status: RESOLVED INVALID

(Firefox Build System :: Mach Core, enhancement)

Description

[Karl Tomlinson (:karlt)]

$ ./mach mochitest-plain image/test/mochitest/test_drawDiscardedImage.html
Error running mach:

    ['mochitest-plain', 'image/test/mochitest/test_drawDiscardedImage.html']

The error occurred in mach itself. This is likely a bug in mach itself or a
fundamental problem with a loaded module.

Please consider filing a bug against mach by going to the URL:

    https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&component=mach


If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ImportError: No module named filters

  File "c:\Users\Bas\Dev\mozilla-inbound\python/mach\mach\main.py", line 336, in
 run
    return self._run(argv)
  File "c:\Users\Bas\Dev\mozilla-inbound\python/mach\mach\main.py", line 382, in
 _run
    args = parser.parse_args(argv)
  File "c:\Users\Bas\Dev\mozilla-build\python\lib\argparse.py", line 1694, in pa
rse_args
    args, argv = self.parse_known_args(args, namespace)
  File "c:\Users\Bas\Dev\mozilla-build\python\lib\argparse.py", line 1726, in pa
rse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "c:\Users\Bas\Dev\mozilla-build\python\lib\argparse.py", line 1935, in _p
arse_known_args
    stop_index = consume_positionals(start_index)
  File "c:\Users\Bas\Dev\mozilla-build\python\lib\argparse.py", line 1891, in co
nsume_positionals
    take_action(action, args)
  File "c:\Users\Bas\Dev\mozilla-build\python\lib\argparse.py", line 1800, in ta
ke_action
    action(self, namespace, argument_values, option_string)
  File "c:\Users\Bas\Dev\mozilla-inbound\python/mach\mach\dispatcher.py", line 1
72, in __call__
    if handler.parser:
  File "c:\Users\Bas\Dev\mozilla-inbound\python/mach\mach\base.py", line 123, in
 parser
    self._parser = self._parser()
  File "c:\Users\Bas\Dev\mozilla-inbound\testing/mochitest/mach_commands.py", li
ne 353, in setup_argument_parser
    ('.py', 'r', imp.PY_SOURCE))
  File "c:\Users\Bas\Dev\mi-debug\_tests\testing\mochitest\runtests.py", line 49
, in <module>
    from manifestparser.filters import (

Comment 1

[Karl Tomlinson (:karlt)]

% DISPLAY=:1 ./mach mochitest-plain dom/media/mediasource/test
Error running mach:

    ['mochitest-plain', 'dom/media/mediasource/test']

The error occurred in mach itself. This is likely a bug in mach itself or a
fundamental problem with a loaded module.

Please consider filing a bug against mach by going to the URL:

    https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&component=mach


If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ImportError: No module named filters

  File "/mnt/ssd1/karl/moz/dev/python/mach/mach/main.py", line 336, in run
    return self._run(argv)
  File "/mnt/ssd1/karl/moz/dev/python/mach/mach/main.py", line 382, in _run
    args = parser.parse_args(argv)
  File "/usr/lib64/python2.7/argparse.py", line 1701, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib64/python2.7/argparse.py", line 1733, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib64/python2.7/argparse.py", line 1942, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/usr/lib64/python2.7/argparse.py", line 1898, in consume_positionals
    take_action(action, args)
  File "/usr/lib64/python2.7/argparse.py", line 1807, in take_action
    action(self, namespace, argument_values, option_string)
  File "/mnt/ssd1/karl/moz/dev/python/mach/mach/dispatcher.py", line 172, in __call__
    if handler.parser:
  File "/mnt/ssd1/karl/moz/dev/python/mach/mach/base.py", line 123, in parser
    self._parser = self._parser()
  File "/mnt/ssd1/karl/moz/dev/testing/mochitest/mach_commands.py", line 353, in setup_argument_parser
    ('.py', 'r', imp.PY_SOURCE))
  File "/mnt/sda11/karl/obj/_tests/testing/mochitest/runtests.py", line 49, in <module>
    from manifestparser.filters import (

Comment 2

[:Cykesiopka]

FWIW, advice a few days ago from #developers that might work:
> rm build/manifestparser.pyc

Comment 3

[(not currently active) Ted Mielczarek]

Fallout from bug 1123763, somehow.

Comment 4

[Andrew Halberstadt [:ahal]]

Yeah, the fix is:

    $ rm build/manifestparser.pyc

Explanation:

So manifestparser.py used to be duplicated in that directory quite a while back, but was removed sometime ago. People who have had their clones for a long time might still have a .pyc file lying around from back then. Unfortunately when running with mach, the 'build' directory is added to sys.path before the 'testing/mozbase/manifestparser' directory. So when we import manifestparser, the leftover .pyc file (which is hopelessly out of date by this point) is found before the actual module, resulting in that error. Bug 1123763 was the bug that finally added a backwards incompatible change which started triggering the error.

Comment 5

[Mike Hommey [:glandium]]

Is there something we can do to make python ignore .pyc files when there is no corresponding .py file?

Comment 6

[Gregory Szorc [:gps]]

(In reply to Mike Hommey [:glandium] from comment #5)
> Is there something we can do to make python ignore .pyc files when there is
> no corresponding .py file?

Without writing a custom importer, no. And even with Python 2.7, I'm not sure how much control we have. Python 3.3 (or maybe it was 3.2) implemented the importers in pure Python, so you are finally able to do funky things.

FWIW, I've long wanted to write a custom importer for mach and the build system so .pyc don't get written to the source directory. Stale .pyc files is one reason I've wanted this. Never had time to do it though :/ Might be worth reevaluating that.

Comment 10

[Tanvi Vyas[:tanvi]]

Why not have mach check for the file and remove it if exists or is stale?