1590197 - [conditioned-profile] Fix Raptor unittests newly-added conditioned-profile depedency install (Python 2)

Status: RESOLVED INVALID

(Testing :: Raptor, defect, P1)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

I need to fix Raptor's unittests, to unblock my patch in bug 1537944.

Currently, under Python 2 (they are being switched to Python 3 in bug 1520463, and might need a different fix, once that lands), they fail like so (again, this is with the otherwise working code from bug 1537944):

./mach python-test --python 2 --subsuite raptor:
<snip>
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not: conditioned-profile~=0.1 from https://files.pythonhosted.org/packages/79/8c/5821811b07acb3cbed2fcf5b97aa4db124bc34e35a098a086c7fd5474641/conditioned-profile-0.1.tar.gz#sha256=51c0326837ae3ee7f405466afde6ecf1c7e857dbd7c6884545ce05b5395b00ff (from -r /Users/sdonner/mozilla-source/mozilla-unified/testing/raptor/requirements.txt (line 1))
<snip>
Error running mach:

['python-test', '--python', '2', '--subsuite', 'raptor']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

CalledProcessError: Command '[u'/Users/sdonner/mozilla-source/mozilla-unified/obj-x86_64-apple-darwin18.7.0/_virtualenvs/mozilla-unified-evYUFdVP-2/bin/pip', u'install', u'--requirement', '/Users/sdonner/mozilla-source/mozilla-unified/testing/raptor/requirements.txt', u'--require-hashes', u'--quiet']' returned non-zero exit status 1

File "/Users/sdonner/mozilla-source/mozilla-unified/python/mach_commands.py", line 107, in python_test
return self.run_python_tests(*args, **kwargs)
File "/Users/sdonner/mozilla-source/mozilla-unified/python/mach_commands.py", line 198, in run_python_tests
return_code = on_test_finished(future.result())
File "/Users/sdonner/mozilla-source/mozilla-unified/third_party/python/futures/concurrent/futures/_base.py", line 398, in result
return self.__get_result()
File "/Users/sdonner/mozilla-source/mozilla-unified/third_party/python/futures/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/sdonner/mozilla-source/mozilla-unified/python/mach_commands.py", line 219, in _run_python_test
self.virtualenv_manager.install_pip_requirements(test['requirements'], quiet=True)
File "/Users/sdonner/mozilla-source/mozilla-unified/python/mozbuild/mozbuild/virtualenv.py", line 570, in install_pip_requirements
return self._run_pip(args)
File "/Users/sdonner/mozilla-source/mozilla-unified/python/mozbuild/mozbuild/virtualenv.py", line 582, in _run_pip
universal_newlines=PY3)
File "/usr/local/Cellar/python@2/2.7.16_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 190, in check_call
raise CalledProcessError(retcode, cmd)

Comment 1

[Robert Wood [:rwood]]

Hey, I'm a bit confused by this bug, the raptor unit tests are running green and passing on python 2 in production:

i.e. https://treeherder.mozilla.org/#/jobs?repo=autoland&tier=1%2C2%2C3&searchStr=raptor%2Csource&revision=92b79faea252e47cbad3e2be29cdbd1f6949811c

Locally they are passing fine also (for me on Py2 and Py3 with my Py3 work), by running:

./mach python-test testing/raptor/test/

Comment 2

[Stephen Donner [:stephend] Not actively reading bugmail]

Sorry, Rob, I meant to make it clearer that these failures only occur with my new patch, in https://phabricator.services.mozilla.com/D49970?id=179555

Would you have further suggestions for how to fix these unittests?

1. In my first attempt, we had to sprinkle (with added import sys):
   paths = [here]
condprofile_dir = os.path.join(os.path.dirname(os.path.dirname(here)), "condprofile")
sys.path.insert(0, condprofile_dir)
   which "fixed" it.
2. This was before bug 1589538, which we now have. The suggestion from the last comment in https://phabricator.services.mozilla.com/D47716#1501065 indicated this should work once we have that package, and remove the path hacks, but now, with this new error, I'm at a loss as to which approach to try next (but will keep looking, all the same).

The actual test passes, itself (https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&author=sdonner%40mozilla.com&selectedJob=272223467), so it's "just" these unittests failing [0][1] now, seemingly.

[0] linux: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&author=sdonner%40mozilla.com&selectedJob=272220849
[1] windows 10: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&author=sdonner%40mozilla.com&selectedJob=272220850

Comment 3

[Stephen Donner [:stephend] Not actively reading bugmail]

I've looked around, and found:

re: hashes, from https://firefox-source-docs.mozilla.org/python/python/index.html#using-a-python-package-index - we MUST specify hashes, which matches the errors in https://gist.github.com/stephendonner/b5646c8e9370b5b64118fb7252970614. If we

1. add conditioned-profile ~= 0.1 in testing/raptor/requirements.txt, and then 2) adding requirements = testing/raptor/requirements.txt to /testing/raptor/test/python.ini, that triggers this hash check, and thus forces it for all packages specified there (and appears to fix the "no module named condprof.client/condprof.util" import issues.

Fixing that would require adding sha256 hashes to all packages (using hashin? https://davidwalsh.name/hashin), a la https://searchfox.org/mozilla-central/rev/088e2cf29c59d733d57af43903eb0267dbf72e2a/tools/lint/python/flake8_requirements.txt

@tarek (@sparky, if you also have ideas?), is this the preferred route to get the unittests happily passing/module-importing, again, or is there another avenue I should be exploring? And if so, would you have further pointers? Thanks!

Comment 4

[Stephen Donner [:stephend] Not actively reading bugmail]

I'm working on vendoring conditioned-profile==0.1 into /third_party/python, after some questions from :sparky about my approaches thus far, helped me finally link most of the remaining pieces together.

Comment 5

[Tarek Ziadé (:tarek)]

No this is not the solution, I posted the solution here https://bugzilla.mozilla.org/attachment.cgi?id=9103867&action=edit

please do not vendor the conditioned profile. To use it in your project you need just a few lines. see my patch

Comment 6

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Tarek Ziadé (:tarek) from comment #5)

No this is not the solution, I posted the solution here https://bugzilla.mozilla.org/attachment.cgi?id=9103867&action=edit

please do not vendor the conditioned profile. To use it in your project you need just a few lines. see my patch

Yup, thank you, Tarek - this works just fine now; closing out as INVALID (since we didn't need to do anything to actually fix the tests).