1573845 - ./mach python-test python/mozbuild/mozbuild/test/backend/test_build.py fails locally

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[Andrew Halberstadt [:ahal]]

STR

$ ./mach python-test python/mozbuild/mozbuild/test/backend/test_build.py

Result

running build_ext
copying build/lib.linux-x86_64-2.7/psutil/_psutil_linux.so -> psutil
copying build/lib.linux-x86_64-2.7/psutil/_psutil_posix.so -> psutil

Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)
 /home/ahal/dev/mozilla-central/objdirs/opt/backend.TestManifestBackend is out of date with respect to /home/ahal/dev/mozilla-central/browser/base/content/test/general/browser.ini
Test configuration changed. Regenerating backend.
 0:09.83 /home/ahal/dev/mozilla-central/python/mozbuild/mozbuild/test/backend/test_build.py
 0:09.83 ============================= test session starts ==============================
 0:09.83 platform linux2 -- Python 2.7.16, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- /home/ahal/dev/mozilla-central/objdirs/opt/_virtualenvs/mozilla-central-c6uB0mvl-2.7/bin/python
 0:09.83 rootdir: /home/ahal/dev/mozilla-central, inifile: /home/ahal/dev/mozilla-central/config/mozunit/mozunit/pytest.ini
 0:09.83 collecting ... collected 3 items
 0:09.83 
 0:09.83 python/mozbuild/mozbuild/test/backend/test_build.py::TestBuild::test_faster_make TEST-UNEXPECTED-FAIL
 0:09.83 python/mozbuild/mozbuild/test/backend/test_build.py::TestBuild::test_faster_recursive_make TEST-UNEXPECTED-FAIL
 0:09.83 python/mozbuild/mozbuild/test/backend/test_build.py::TestBuild::test_recursive_make TEST-UNEXPECTED-FAIL
 0:09.83 
 0:09.83 =================================== FAILURES ===================================
 0:09.83 __________________________ TestBuild.test_faster_make __________________________
 0:09.83 
 0:09.83 self = <mozbuild.test.backend.test_build.TestBuild testMethod=test_faster_make>
 0:09.83 
 0:09.83     def test_faster_make(self):
 0:09.83         substs = list(BASE_SUBSTS) + [
 0:09.83             ('MOZ_BUILD_APP', 'dummy_app'),
 0:09.83             ('MOZ_WIDGET_TOOLKIT', 'dummy_widget'),
 0:09.83         ]
 0:09.83         with self.do_test_backend(RecursiveMakeBackend, FasterMakeBackend,
 0:09.83                                   substs=substs) as config:
 0:09.83             buildid = mozpath.join(config.topobjdir, 'config', 'buildid')
 0:09.84             ensureParentDir(buildid)
 0:09.84             with open(buildid, 'w') as fh:
 0:09.84                 fh.write('20100101012345\n')
 0:09.84     
 0:09.84             build = MozbuildObject(config.topsrcdir, None, None,
 0:09.84                                    config.topobjdir)
 0:09.84             overrides = [
 0:09.84                 'TEST_MOZBUILD=1',
 0:09.84             ]
 0:09.84             with self.line_handler() as handle_make_line:
 0:09.84                 build._run_make(directory=mozpath.join(config.topobjdir,
 0:09.84                                                        'faster'),
 0:09.84                                 target=overrides, silent=False,
 0:09.84 >                               line_handler=handle_make_line)
 0:09.84 
 0:09.84 python/mozbuild/mozbuild/test/backend/test_build.py:145:
 0:09.84 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
 0:09.84 python/mozbuild/mozbuild/base.py:677: in _run_make
 0:09.84     if num_jobs == 0 and self.mozconfig['make_flags']:
 0:09.84 python/mozbuild/mozbuild/base.py:321: in mozconfig
 0:09.84     return self.mozconfig_and_target[0]
 0:09.84 python/mozbuild/mozbuild/base.py:313: in mozconfig_and_target
 0:09.84     self.topsrcdir, self._mozconfig, os.environ.get('MOZCONFIG'))
 0:09.84 python/mozbuild/mozbuild/util.py:952: in __call__
 0:09.84     self[args] = self.func(*args)
 0:09.84 python/mozbuild/mozbuild/base.py:301: in get_mozconfig_and_target
 0:09.84     sandbox._value_for(sandbox['mozconfig_options'])
 0:09.84 python/mozbuild/mozbuild/configure/__init__.py:538: in _value_for
 0:09.84     return self._value_for_depends(self._depends[obj])
 0:09.84 python/mozbuild/mozbuild/util.py:961: in method_call
 0:09.84     cache[args] = self.func(instance, *args)
 0:09.84 python/mozbuild/mozbuild/configure/__init__.py:550: in _value_for_depends
 0:09.84     value = obj.result()
 0:09.84 python/mozbuild/mozbuild/util.py:961: in method_call
 0:09.84     cache[args] = self.func(instance, *args)
 0:09.84 python/mozbuild/mozbuild/configure/__init__.py:155: in result
 0:09.84     for d in self.dependencies]
 0:09.84 python/mozbuild/mozbuild/configure/__init__.py:541: in _value_for
 0:09.84     return self._value_for_depends(obj)
 0:09.84 python/mozbuild/mozbuild/util.py:961: in method_call
 0:09.84     cache[args] = self.func(instance, *args)
 0:09.84 python/mozbuild/mozbuild/configure/__init__.py:550: in _value_for_depends
 0:09.84     value = obj.result()
 0:09.84 python/mozbuild/mozbuild/util.py:961: in method_call
 0:09.84     cache[args] = self.func(instance, *args)
 0:09.84 python/mozbuild/mozbuild/configure/__init__.py:156: in result
 0:09.84     return self._func(*resolved_args)
 0:09.84 python/mozbuild/mozbuild/configure/__init__.py:1125: in wrapped
 0:09.84     return new_func(*args, **kwargs)
 0:09.84 build/moz.configure/init.configure:191: in mozconfig
 0:09.84     mozconfig = loader.read_mozconfig(mozconfig)
 0:09.84 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
 0:09.84 
 0:09.84 self = <mozbuild.mozconfig.MozconfigLoader object at 0x7f1139bdf510>
 0:09.84 path = '/home/ahal/dev/mozilla-central/mozconfig'
 0:09.84 
 0:09.84     def read_mozconfig(self, path=None):
 0:09.84         """Read the contents of a mozconfig into a data structure.
 0:09.84     
 0:09.84             This takes the path to a mozconfig to load. If the given path is
 0:09.84             AUTODETECT, will try to find a mozconfig from the environment using
 0:09.84             find_mozconfig().
 0:09.84     
 0:09.84             mozconfig files are shell scripts. So, we can't just parse them.
 0:09.84             Instead, we run the shell script in a wrapper which allows us to record
 0:09.84             state from execution. Thus, the output from a mozconfig is a friendly
 0:09.84             static data structure.
 0:09.84             """
 0:09.85         if path is self.AUTODETECT:
 0:09.85             path = self.find_mozconfig()
 0:09.85     
 0:09.85         result = {
 0:09.85             'path': path,
 0:09.85             'topobjdir': None,
 0:09.85             'configure_args': None,
 0:09.85             'make_flags': None,
 0:09.85             'make_extra': None,
 0:09.85             'env': None,
 0:09.85             'vars': None,
 0:09.85         }
 0:09.85     
 0:09.85         if path is None:
 0:09.85             return result
 0:09.85     
 0:09.85         path = mozpath.normsep(path)
 0:09.85     
 0:09.85         result['configure_args'] = []
 0:09.85         result['make_extra'] = []
 0:09.85         result['make_flags'] = []
 0:09.85     
 0:09.85         env = dict(os.environ)
 0:09.85     
 0:09.85         # Since mozconfig_loader is a shell script, running it "normally"
 0:09.85         # actually leads to two shell executions on Windows. Avoid this by
 0:09.85         # directly calling sh mozconfig_loader.
 0:09.85         shell = 'sh'
 0:09.85         if 'MOZILLABUILD' in os.environ:
 0:09.85             shell = os.environ['MOZILLABUILD'] + '/msys/bin/sh'
 0:09.85         if sys.platform == 'win32':
 0:09.85             shell = shell + '.exe'
 0:09.85     
 0:09.85         command = [shell, mozpath.normsep(self._loader_script),
 0:09.85                    mozpath.normsep(self.topsrcdir), path, sys.executable,
 0:09.85                    mozpath.join(mozpath.dirname(self._loader_script),
 0:09.85                                 'action', 'dump_env.py')]
 0:09.85     
 0:09.85         try:
 0:09.85             # We need to capture stderr because that's where the shell sends
 0:09.85             # errors if execution fails.
 0:09.85             output = subprocess.check_output(command, stderr=subprocess.STDOUT,
 0:09.85                                              cwd=self.topsrcdir, env=env)
 0:09.85         except subprocess.CalledProcessError as e:
 0:09.85             lines = e.output.splitlines()
 0:09.85     
 0:09.85             # Output before actual execution shouldn't be relevant.
 0:09.85             try:
 0:09.85                 index = lines.index('------END_BEFORE_SOURCE')
 0:09.85                 lines = lines[index + 1:]
 0:09.85             except ValueError:
 0:09.85                 pass
 0:09.85     
 0:09.85 >           raise MozconfigLoadException(path, MOZCONFIG_BAD_EXIT_CODE, lines)
 0:09.85 E           MozconfigLoadException: Evaluation of your mozconfig exited with an error. This could be triggered
 0:09.85 E           by a command inside your mozconfig failing. Please change your mozconfig
 0:09.85 E           to not error and/or to catch errors in executed commands.
 0:09.85 
 0:09.85 python/mozbuild/mozbuild/mozconfig.py:256: MozconfigLoadException
 0:09.85 ----------------------------- Captured stdout call -----------------------------
 0:09.85 
 0:09.85 ------------------------------ Captured log call -------------------------------
 0:09.85 __init__.py                382 INFO     checking for vcs source checkout...
 0:09.85 __init__.py                382 INFO     hg

The other two tests in this file fail with the same error.

Comment 1

[Chris Manchester (limited bugmail, email directly)]

I wasn't able to reproduce this, do you have anything interesting in your actual mozconfig?

Comment 2

[Andrew Halberstadt [:ahal]]

Looks like I can't reproduce anymore either. I'm not particularly inclined to bother bisecting (also entirely possible I was in some weird state), so I guess WFM.