Closed Bug 1648506 Opened 7 months ago Closed 7 months ago

test_moz_configure.py::TestTargetLinux::test_target fails "Please use the *system* python to run this script" on OSX and --python=python2 only, when 'mach python-test' runs in python 3

Categories

(Firefox Build System :: Mach Core, defect, P3)

defect

Tracking

(firefox79 fixed)

RESOLVED FIXED
mozilla79
Tracking Status
firefox79 --- fixed

People

(Reporter: gbrown, Assigned: mhentges)

References

Details

(Keywords: in-triage)

Attachments

(1 file)

In bug 1594914, we are trying to run 'mach python-test' with python3: remove it from the python2 exceptions at https://searchfox.org/mozilla-central/rev/a87a1c3b543475276e6d57a7a80cb02f3e42b6ed/mach#35. With a few other changes, all is well locally and on try, except for source-test-python-mozbuild-macosx1014-64/opt-py2:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b6a57f0cf677067b7a21fdb5cd358bf7c1dddea8

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=307405609&repo=try&lineNumber=723

[task 2020-06-24T17:23:53.146Z]  0:56.24 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestTargetLinux::test_target TEST-UNEXPECTED-FAIL
[task 2020-06-24T17:23:53.146Z]  0:56.24 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestTargetWindows::test_target TEST-UNEXPECTED-FAIL
[task 2020-06-24T17:23:53.146Z]  0:56.24 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestTargetAndroid::test_target TEST-UNEXPECTED-FAIL
[task 2020-06-24T17:23:53.146Z]  0:56.24 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestTargetOpenBSD::test_target TEST-UNEXPECTED-FAIL
[task 2020-06-24T17:23:53.146Z]  0:56.24 python/mozbuild/mozbuild/test/configure/test_moz_configure.py::TestMozConfigure::test_nsis_version TEST-UNEXPECTED-FAIL
[task 2020-06-24T17:23:53.146Z]  0:56.24 
[task 2020-06-24T17:23:53.146Z]  0:56.24 =================================== FAILURES ===================================
[task 2020-06-24T17:23:53.146Z]  0:56.24 _________________________ TestTargetLinux.test_target __________________________
[task 2020-06-24T17:23:53.146Z]  0:56.24 
[task 2020-06-24T17:23:53.146Z]  0:56.24 self = <test_moz_configure.TestTargetLinux testMethod=test_target>
[task 2020-06-24T17:23:53.146Z]  0:56.24 
[task 2020-06-24T17:23:53.149Z]  0:56.24     def test_target(self):
[task 2020-06-24T17:23:53.149Z]  0:56.24 >       self.assertEqual(self.get_target([]), self.HOST)
...
[task 2020-06-24T17:23:53.157Z]  0:56.25         if result:
[task 2020-06-24T17:23:53.157Z]  0:56.25             raise Exception(
[task 2020-06-24T17:23:53.157Z]  0:56.25                 'Failed to create virtualenv: %s (virtualenv.py retcode: %s)' % (
[task 2020-06-24T17:23:53.157Z]  0:56.25 >                   self.virtualenv_root, result))
[task 2020-06-24T17:23:53.157Z]  0:56.25 E           Exception: Failed to create virtualenv: /Users/cltbld/tasks/task_1593018941/checkouts/gecko/obj-x86_64-apple-darwin18.6.0/_virtualenvs/init_py3 (virtualenv.py retcode: 1)
[task 2020-06-24T17:23:53.157Z]  0:56.25 
[task 2020-06-24T17:23:53.157Z]  0:56.25 python/mozbuild/mozbuild/virtualenv.py:251: Exception
[task 2020-06-24T17:23:53.157Z]  0:56.25 ------------------------------ Captured log call -------------------------------
[task 2020-06-24T17:23:53.157Z]  0:56.25 __init__.py                383 DEBUG    python3: running with pid 2069
[task 2020-06-24T17:23:53.157Z]  0:56.25 __init__.py                383 DEBUG    python3: sys.executable: '/Users/cltbld/tasks/task_1593018941/checkouts/gecko/obj-x86_64-apple-darwin18.6.0/_virtualenvs/gecko-T77fYFED-2/bin/python'
[task 2020-06-24T17:23:53.157Z]  0:56.25 __init__.py                383 DEBUG    python3: executable from configuration: None
[task 2020-06-24T17:23:53.157Z]  0:56.25 __init__.py                383 DEBUG    python3: found executable: u'/Users/cltbld/tasks/task_1593018941/checkouts/gecko/obj-x86_64-apple-darwin18.6.0/_virtualenvs/init_py3/bin/python3'
[task 2020-06-24T17:23:53.157Z]  0:56.25 __init__.py                383 INFO     Creating Python 3 environment
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO     Please use the *system* python to run this script
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO     Traceback (most recent call last):
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO       File "/Users/cltbld/tasks/task_1593018941/checkouts/gecko/third_party/python/virtualenv/virtualenv.py", line 2634, in <module>
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO         main()
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO       File "/Users/cltbld/tasks/task_1593018941/checkouts/gecko/third_party/python/virtualenv/virtualenv.py", line 870, in main
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO         symlink=options.symlink,
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO       File "/Users/cltbld/tasks/task_1593018941/checkouts/gecko/third_party/python/virtualenv/virtualenv.py", line 1162, in create_environment
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO         install_python(home_dir, lib_dir, inc_dir, bin_dir, site_packages=site_packages, clear=clear, symlink=symlink)
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO       File "/Users/cltbld/tasks/task_1593018941/checkouts/gecko/obj-x86_64-apple-darwin18.6.0/_virtualenvs/init_py3/bin/../lib/python3.7/posixpath.py", line 378, in abspath
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO         path = os.fspath(path)
[task 2020-06-24T17:23:53.158Z]  0:56.25 __init__.py                383 INFO     TypeError: expected str, bytes or os.PathLike object, not NoneType
[task 2020-06-24T17:23:53.158Z]  0:56.25 ________________________ TestTargetWindows.test_target _________________________

I think that get_target() call will sometimes trigger venv creation, but I'm not clear on the conditions for that.

I see that virtualenv.py is failing because of "Please use the system python to run this script", but I don't understand how to fix that.

:rstewart - Can you help out here, help us understand these tests, or give us tips for debugging?

Flags: needinfo?(rstewart)
Severity: -- → S3
Flags: needinfo?(rstewart)
Keywords: in-triage
Priority: -- → P3
Assignee: nobody → mhentges

How about not running configure tests with python2?

skip-if = python == 2 && os == "mac"
would be good enough...if no one objects?

SGTM. Unfortunately taskgraph still ends up directly or indirectly executing a lot of configure stuff in Python 2 (that prevented me from landing bug 1645507), so continuing to run those tests with Python 2 on some operating system for the forseeable future is probably still necessary and sufficient.

These tests fail on py2 on osx when 'mach python-test' runs with python 3, blocking
bug 1594914.

Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/650b8d5618ae
Disable some python tests on py2 on mac; r=mhentges
Status: NEW → RESOLVED
Closed: 7 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla79
Blocks: 1654961
You need to log in before you can comment on or make changes to this bug.