Closed Bug 1323439 Opened 8 years ago Closed 3 years ago

Need functioning pyOpenSSL on testers

Categories

(Release Engineering :: General, defect)

Product:
Release Engineering
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: bwc, Unassigned)

References

(Blocks 1 open bug)

Details

In order to test bug 1056934, our test TURN server needs to be able to serve TLS. Twisted (which this test TURN server uses) needs pyOpenSSL to do this. The copy of pyOpenSSL we have on http://pypi.pub.build.mozilla.org/pub does not appear to be functional on windows, linux, or OS X: Windows: 16:46:04 INFO - Running setup.py (path:C:\slave\test\build\venv\build\pyOpenSSL\setup.py) egg_info for package pyOpenSSL 16:46:05 INFO - error: could not find 'ssleay32.dll' 16:46:05 INFO - Complete output from command python setup.py egg_info: 16:46:05 INFO - running egg_info 16:46:05 INFO - creating pip-egg-info\pyOpenSSL.egg-info 16:46:05 INFO - writing pip-egg-info\pyOpenSSL.egg-info\PKG-INFO 16:46:05 INFO - writing top-level names to pip-egg-info\pyOpenSSL.egg-info\top_level.txt 16:46:05 INFO - writing dependency_links to pip-egg-info\pyOpenSSL.egg-info\dependency_links.txt 16:46:05 INFO - writing manifest file 'pip-egg-info\pyOpenSSL.egg-info\SOURCES.txt' 16:46:05 INFO - warning: manifest_maker: standard file '-c' not found 16:46:05 INFO - error: could not find 'ssleay32.dll' 16:46:05 INFO - ---------------------------------------- 16:46:05 INFO - Cleaning up... Linux: 15:18:49 INFO - ICE Server: File "iceserver/iceserver.py", line 17, in <module> 15:18:49 INFO - ICE Server: 15:18:49 INFO - ICE Server: from twisted.internet import reactor, protocol, ssl 15:18:49 INFO - ICE Server: File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/twisted/internet/ssl.py", line 46, in <module> 15:18:49 INFO - ICE Server: 15:18:49 INFO - ICE Server: from OpenSSL import SSL 15:18:49 INFO - ICE Server: File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/OpenSSL/__init__.py", line 11, in <module> 15:18:49 INFO - ICE Server: 15:18:49 INFO - ICE Server: import rand, crypto, SSL, tsafe 15:18:49 INFO - ICE Server: ImportError 15:18:49 INFO - ICE Server: : 15:18:49 INFO - ICE Server: /builds/slave/test/build/venv/local/lib/python2.7/site-packages/OpenSSL/SSL.so: undefined symbol: SSLv2_method 15:18:49 INFO - ICE Server: 15:18:49 INFO - ICE Server: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ProcessTerminated'>: A process has ended with a probable error condition: process ended with exit code 1. 15:18:49 INFO - ICE Server: ] 15:18:49 INFO - ICE Server websocket closed OS X: 16:24:28 INFO - ICE Server: File "iceserver/iceserver.py", line 764, in <module> 16:24:28 INFO - ICE Server: tls_context_factory = ssl.DefaultOpenSSLContextFactory(KEY_FILE, CERT_FILE, SSL.TLSv1_2_METHOD) 16:24:28 INFO - ICE Server: AttributeError: 'module' object has no attribute 'TLSv1_2_METHOD' 16:24:28 INFO - ICE Server websocket closed
Summary: Need newer pyOpenSSL on builders → Need functioning pyOpenSSL on testers
I actually see two version of pyOpenSSL in http://pypi.pub.build.mozilla.org/pub: pyOpenSSL-0.10.tar.gz pyOpenSSL-16.2.0.tar.gz The 0.10 version is from 2009, so that would not be surprising if that does not work. :catlee pointed out to me that we should at least be able to make progress on this for Linux by modifying the Docker scripts for the Linux test machines. And the docker scripts appear to be this: http://searchfox.org/mozilla-central/source/taskcluster/docker/desktop-test or http://searchfox.org/mozilla-central/source/taskcluster/docker/desktop1604-test Not sure how pipi and the Docker things interact here though.
Those are the right directories to look at the dockerfiles for both images. These dockerfiles make use of some in-tree magic that can grab files from a directory other than the working image directory and import them into the image. Such as here: https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/desktop1604-test/Dockerfile#26 Both images have a setup shell script that they run that configures system packages, etc as can be seen here [1]. In this example peep is used to install python packages. You can find the setup script for desktop1604-test here [2]. If you change these any files that make up the build for the image, when you push to a branch, such as try, the images will be rebuilt with those changes and used for the tasks scheduled for that push. That's an easy way to try your changes out prior to getting review and pushing elsewhere. [1] https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/recipes/ubuntu1204-test-system-setup.sh#149 [2] https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/recipes/ubuntu1604-test-system-setup.sh
Component: General Automation → General

confirmed pyopenssl-19.1.0 includes openssl for macos (the wheels includes all but libc). So we do not expect problems updating on the macos workers:

[dhouse@t-mojave-r7-121.test.releng.mdc2.mozilla.com ~]$ /usr/local/bin/pip3 install pyopenssl
Collecting pyopenssl
  Downloading https://files.pythonhosted.org/packages/9e/de/f8342b68fa9e981d348039954657bdf681b2ab93de27443be51865ffa310/pyOpenSSL-19.1.0-py2.py3-none-any.whl (53kB)
     |████████████████████████████████| 61kB 6.2MB/s
Collecting cryptography>=2.8
  Downloading https://files.pythonhosted.org/packages/6b/4a/ce93178469d4460d6b3a5e648fc1a2f426030f3d30a12d7ed4df73d044de/cryptography-2.8-cp34-abi3-macosx_10_6_intel.whl (1.6MB)
     |████████████████████████████████| 1.6MB 27.8MB/s
Collecting six>=1.5.2
  Downloading https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8
  Downloading https://files.pythonhosted.org/packages/d5/61/32b1aa5ef1bf60be4ef679c4aae082a7ceef98517e0e0fde68072c6ef8b6/cffi-1.13.2-cp37-cp37m-macosx_10_6_intel.whl (258kB)
     |████████████████████████████████| 266kB 43.7MB/s
Collecting pycparser
  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
     |████████████████████████████████| 163kB 48.0MB/s
Building wheels for collected packages: pycparser
  Building wheel for pycparser (setup.py) ... done
  Created wheel for pycparser: filename=pycparser-2.19-py2.py3-none-any.whl size=111029 sha256=c642f717ddc476bb655ddb51f215b1e5bfd615bb21558f90e03b67a03b786ba5
  Stored in directory: /Users/dhouse/Library/Caches/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511
Successfully built pycparser
Installing collected packages: six, pycparser, cffi, cryptography, pyopenssl
Successfully installed cffi-1.13.2 cryptography-2.8 pycparser-2.19 pyopenssl-19.1.0 six-1.14.0
[dhouse@t-mojave-r7-121.test.releng.mdc2.mozilla.com ~]$ brew upgrade openssl
Warning: openssl 1.1.1d already installed
[dhouse@t-mojave-r7-121.test.releng.mdc2.mozilla.com ~]$ otool -L /usr/local/opt/openssl@1.1/bin/openssl
/usr/local/opt/openssl@1.1/bin/openssl:
	/usr/local/Cellar/openssl@1.1/1.1.1d/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
	/usr/local/Cellar/openssl@1.1/1.1.1d/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
[dhouse@t-mojave-r7-121.test.releng.mdc2.mozilla.com ~]$ find /usr/local/lib/python3.7/site-packages/cryptography/ -name "*.so" | xargs -I{} otool -L {}
/usr/local/lib/python3.7/site-packages/cryptography//hazmat/bindings/_padding.abi3.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
/usr/local/lib/python3.7/site-packages/cryptography//hazmat/bindings/_constant_time.abi3.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
/usr/local/lib/python3.7/site-packages/cryptography//hazmat/bindings/_openssl.abi3.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
See Also: → 1340020
See Also: → 1702323

Is the only thing needed here uploading a package to internal pypi? Not working today but can tackle next week, please need info if so.

It might be enough? We also need the service_identity package now. Just having this working for one platform would be a huge improvement, if that helps expedite.

Looks like pyOpenSSL==19.1.0 is already on the internal pypi. I can also upload version 20.0.1 if desired.

I uploaded service_identity==18.1.0 while I was there.

Just tried this out, we seem to be failing on a dependency (setuptools-rust):

https://treeherder.mozilla.org/jobs?repo=try&revision=dc666cd26f677abfeda14627dda29069cb4f1acf

Flags: needinfo?(ahal)

Also added setuptools-rust==0.12.1.

(And thanks for the needinfo, please needinfo again if there's even more missing or I'll likely miss this again)

Flags: needinfo?(ahal)
Blocks: 1710634

Looks like we need yet more deps. Is there a way to run pip against our pypi from a developer machine?

Latest failure on try seems to indicate we need pyasn1-modules. We might also need setuptools>=46.1 and setuptools_scm[toml]>=3.4.3, but it is hard to tell because an intermediate dependency might be a different version than what we have on our pypi. We probably need semantic-version (>=2.6.0 is what pip wants for me), but again it is hard to tell because of differences in version in intermediate dependencies.

Flags: needinfo?(ahal)

Yeah, you should be able to use pip install --no-index --find-links https://pypi.pub.build.mozilla.org/pub <package>. It will fail on the first package that is missing, but then you could do a normal pip install of that package (ideally in a virtualenv), and then run the first command again to see the next failure. After a few iterations of that you can generate the full list of packages you need.

Flags: needinfo?(ahal)

So, after experimenting with a venv, it looks like the only thing we're missing now is pyasn1-modules. The try push in comment 12 ought to be able to pick that up if it is added to the pypi and retriggered, could we give that a go?

Flags: needinfo?(jmaher)

I have uploaded the module:
https://pypi.pub.build.mozilla.org/pub/pyasn1_modules-0.2.8-py2.py3-none-any.whl

I have also retriggered a few jobs on various platforms to see if this works.

Flags: needinfo?(jmaher)
Blocks: 1731302

Thanks! Seems to be working!

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.