Closed Bug 1574462 Opened 2 months ago Closed 2 months ago

Error running mach: ['try', 'fuzzy']

Categories

(Firefox Build System :: General, defect, blocker)

defect
Not set
blocker

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: mayhemer, Unassigned)

References

(Blocks 1 open bug)

Details

$ ./mach try fuzzy
Error running mach:

    ['try', 'fuzzy']

The error occurred in mach itself. This is likely a bug in mach itself or a
fundamental problem with a loaded module.

You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ImportError: cannot import name import_sibling_modules

  File "c:\Mozilla\src\mozilla-central4\python/mach\mach\main.py", line 360, in run
    return self._run(argv)
  File "c:\Mozilla\src\mozilla-central4\python/mach\mach\main.py", line 417, in _run
    args = parser.parse_args(argv)
  File "c:\Mozilla\mozilla-build\python\lib\argparse.py", line 1701, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "c:\Mozilla\mozilla-build\python\lib\argparse.py", line 1733, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "c:\Mozilla\mozilla-build\python\lib\argparse.py", line 1942, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "c:\Mozilla\mozilla-build\python\lib\argparse.py", line 1898, in consume_positionals
    take_action(action, args)
  File "c:\Mozilla\mozilla-build\python\lib\argparse.py", line 1807, in take_action
    action(self, namespace, argument_values, option_string)
  File "c:\Mozilla\src\mozilla-central4\python/mach\mach\dispatcher.py", line 173, in __call__
    if handler.parser:
  File "c:\Mozilla\src\mozilla-central4\python/mach\mach\decorators.py", line 82, in parser
    self._parser = self._parser()
  File "c:\Mozilla\src\mozilla-central4\tools/tryselect/mach_commands.py", line 37, in __call__
    mod = importlib.import_module('tryselect.selectors.{}'.format(self.selector))
  File "c:\Mozilla\mozilla-build\python\lib\importlib\__init__.py", line 37, in import_module
    __import__(name)
  File "c:\Mozilla\src\mozilla-central4\build/mach_bootstrap.py", line 404, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "c:\Mozilla\src\mozilla-central4\tools\tryselect\selectors\fuzzy.py", line 17, in <module>
    from ..cli import BaseTryParser
  File "c:\Mozilla\src\mozilla-central4\build/mach_bootstrap.py", line 404, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "c:\Mozilla\src\mozilla-central4\tools\tryselect\cli.py", line 12, in <module>
    from .templates import all_templates
  File "c:\Mozilla\src\mozilla-central4\build/mach_bootstrap.py", line 404, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "c:\Mozilla\src\mozilla-central4\tools\tryselect\templates.py", line 20, in <module>
    from .tasks import resolve_tests_by_suite
  File "c:\Mozilla\src\mozilla-central4\build/mach_bootstrap.py", line 404, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "c:\Mozilla\src\mozilla-central4\tools\tryselect\tasks.py", line 21, in <module>
    from taskgraph.generator import TaskGraphGenerator
  File "c:\Mozilla\src\mozilla-central4\build/mach_bootstrap.py", line 404, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "c:\Mozilla\src\mozilla-central4\taskcluster\taskgraph\generator.py", line 15, in <module>
    from .optimize import optimize_task_graph
  File "c:\Mozilla\src\mozilla-central4\build/mach_bootstrap.py", line 404, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "c:\Mozilla\src\mozilla-central4\taskcluster\taskgraph\optimize\__init__.py", line 24, in <module>

fresh checkout of mozilla-central.

BTW, this fails for any ./mach try form, so it's NOT isolated to fuzzy.

If you update m-c again to get the backout of https://bugzilla.mozilla.org/show_bug.cgi?id=1568277 are you fixed?

Flags: needinfo?(honzab.moz)

I'm on 5d4cbfe103bb, which is a child of 8da8443e0bcb7a6d9766d179332443660c926d8b (the backout cset).

Flags: needinfo?(honzab.moz)

Honza, can you please remove *.pyc files and try again?

https://bugzilla.mozilla.org/show_bug.cgi?id=1568277#c16

Flags: needinfo?(honzab.moz)

I'll try ASAP. Justin, please also see https://bugzilla.mozilla.org/show_bug.cgi?id=1574451 and all of its comments (it's short)

(In reply to Honza Bambas (:mayhemer) from comment #5)

I'll try ASAP. Justin, please also see https://bugzilla.mozilla.org/show_bug.cgi?id=1574451 and all of its comments (it's short)

Yea the largest confusion to me, in both bugs, is that "c:\Mozilla\src\mozilla-central4\taskcluster\taskgraph\optimize\__init__.py" should not exist if you're on the latest rev of m-c.

I suspect there is a *.pyc file shadowing in there and is what could be causing python to think it sees that.

Yes this is happening because bug 1568277 landed and then got backed out. Run ./mach clobber python to delete all *.pyc files in tree and it'll start working again.

Duplicate of this bug: 1574451

(In reply to Andrew Halberstadt [:ahal] from comment #7)

Yes this is happening because bug 1568277 landed and then got backed out. Run ./mach clobber python to delete all *.pyc files in tree and it'll start working again.

Helped! This is not the first time in-tree *.pyc files caused trouble. Is it a python bug?

Thanks.

(Not sure how to resolve this bug...)

Flags: needinfo?(honzab.moz)

Python will check the timestamp of the .pyc and use it if it is newer than the timestamp of the source file (otherwise recompile). When backing out (or even just updating to an older revision), I guess the timestamp of the source file also gets reverted so Python doesn't realize the .pyc is actually stale.

Tbh, I'm not really sure why this happens.. when checking out older versions of a file, the mtime should still get updated so one would think that Python wouldn't use the corresponding .pyc file.

Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED

The subtle problem is that before the backout there was taskcluster/taskgraph/optimize/__init__.py and after the backout there wasn't, and there was taskcluster/taskgraph/optimize.py instead. Python happily uses a .pyc when a .py doesn't exist, so when it found the stale taskcluster/taskgraph/optimize/__init__.pyc, it used it, rather than taskcluster/taskgraph/optimize.{py,pyc}.

You need to log in before you can comment on or make changes to this bug.