Closed
Bug 1240530
Opened 6 years ago
Closed 6 years ago
No attribute PROTOCOL_SSLv3 error for artifact builds on Linux
Categories
(Firefox Build System :: General, defect)
Tracking
(firefox46 fixed)
RESOLVED
FIXED
mozilla46
Tracking | Status | |
---|---|---|
firefox46 | --- | fixed |
People
(Reporter: ato, Assigned: chmanchester)
References
(Blocks 1 open bug)
Details
Attachments
(1 file)
Trying to compile artifact build on Linux yields an error: Output: % ./mach build 0:00.15 Running |mach artifact install|. 0:00.15 /home/ato/1/gecko/src/mach artifact install Error running mach: ['artifact', 'install'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: AttributeError: 'module' object has no attribute 'PROTOCOL_SSLv3' File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/mach_commands.py", line 1498, in artifact_install artifacts = self._make_artifacts(tree=tree, job=job) File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/mach_commands.py", line 1465, in _make_artifacts from mozbuild.artifacts import Artifacts File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/artifacts.py", line 59, in <module> import taskcluster File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/lib/python2.7/site-packages/taskcluster/__init__.py", line 5, in <module> import client File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/lib/python2.7/site-packages/taskcluster/client.py", line 15, in <module> import requests File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/lib/python2.7/site-packages/requests/__init__.py", line 53, in <module> from .packages.urllib3.contrib import pyopenssl File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/lib/python2.7/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 73, in <module> ssl.PROTOCOL_SSLv3: OpenSSL.SSL.SSLv3_METHOD, Error running mach: ['build'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: Exception: Process executed with non-0 exit code 1: [u'/home/ato/1/gecko/src/mach', u'artifact', u'install'] File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/mach_commands.py", line 411, in build self._run_mach_artifact_install() File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/mach_commands.py", line 638, in _run_mach_artifact_install pass_thru=True, ensure_exit_code=True) File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/base.py", line 642, in _run_command_in_srcdir return self.run_process(cwd=self.topsrcdir, **args) File "/home/ato/1/gecko/src/python/mach/mach/mixin/process.py", line 147, in run_process raise Exception('Process executed with non-0 exit code %d: %s' % (status, args)) % python --version Python 2.7.9 mozconfig: mk_add_options AUTOCLOBBER=1 ac_add_options --with-ccache ac_add_options --enable-rust ac_add_options --enable-application=browser ac_add_options --enable-artifact-builds mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../build/artifact-desktop-release
Comment 1•6 years ago
|
||
This might be an upstream bug in requests/urllib3. Another workaround is to install Python 2.7.9+ to get a Python distribution with proper security support. I recommend https://github.com/yyuu/pyenv for installing Python in userland.
Comment 2•6 years ago
|
||
Just noticed from my output of an artifact build that we're installing requests-2.4.3. This is an ancient version that has known security vulnerabilities. It is also almost certainly not compatible with Python < 2.7.9 and the source of this bug.
Component: mach → Build Config
Comment 3•6 years ago
|
||
We have requests 2.5.1 vendored in m-c under python/requests. It should be upgraded to latest and `mach artifact` should use it.
Comment 4•6 years ago
|
||
We might just bump the taskcluster client version, which may have advanced. I haven't checked in a long time.
Assignee | ||
Comment 5•6 years ago
|
||
(In reply to Nick Alexander :nalexander from comment #4) > We might just bump the taskcluster client version, which may have advanced. > I haven't checked in a long time. This fixes the issue for me locally (as does |./mach python ./mach build|) by picking up the in-tree requests, which is apparently recent enough to avoid this issue.
Assignee | ||
Comment 6•6 years ago
|
||
Currently |./mach artifact| installs an old version of the taskcluster client, which installs an old version of requests that's incompatible with commonly installed python versions. This bumps to a version of taskcluster client that accepts and requests version < 3, so we pick up the in-tree version instead of installing 2.4.3. Review commit: https://reviewboard.mozilla.org/r/31525/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/31525/
Attachment #8709722 -
Flags: review?(nalexander)
Assignee | ||
Comment 7•6 years ago
|
||
ato, can you confirm the fix in comment 6 is effective?
Flags: needinfo?(ato)
Comment 8•6 years ago
|
||
Comment on attachment 8709722 [details] MozReview Request: Bug 1240530 - Bump the taskcluster version installed by |./mach artifact| to avoid installing an out-dated requests. r=nalexander https://reviewboard.mozilla.org/r/31525/#review28217 Code is fine. Ship it if you're confident with your testing.
Attachment #8709722 -
Flags: review?(nalexander) → review+
Reporter | ||
Comment 9•6 years ago
|
||
With the change applied, it hangs with the following stacktrace: % ./mach build 0:00.15 Running |mach artifact install|. 0:00.15 /home/ato/1/gecko/src/mach artifact install New python executable in /home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/bin/python2.7 Not overwriting existing python script /home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/bin/python (you must use /home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/bin/python2.7) Installing setuptools, pip...done. running build_ext copying build/lib.linux-x86_64-2.7/psutil/_psutil_linux.so -> psutil copying build/lib.linux-x86_64-2.7/psutil/_psutil_posix.so -> psutil Error running mach: ['artifact', 'install'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: AttributeError: 'module' object has no attribute 'PROTOCOL_SSLv3' File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/mach_commands.py", line 1498, in artifact_install artifacts = self._make_artifacts(tree=tree, job=job) File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/mach_commands.py", line 1465, in _make_artifacts from mozbuild.artifacts import Artifacts File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/artifacts.py", line 59, in <module> import taskcluster File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/lib/python2.7/site-packages/taskcluster/__init__.py", line 5, in <module> import client File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/lib/python2.7/site-packages/taskcluster/client.py", line 15, in <module> import requests File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/lib/python2.7/site-packages/requests/__init__.py", line 53, in <module> from .packages.urllib3.contrib import pyopenssl File "/home/ato/1/gecko/src/build/mach_bootstrap.py", line 340, in __call__ module = self._original_import(name, globals, locals, fromlist, level) File "/home/ato/1/gecko/build/artifact-desktop-release/_virtualenv/lib/python2.7/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 73, in <module> ssl.PROTOCOL_SSLv3: OpenSSL.SSL.SSLv3_METHOD, Error running mach: ['build'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: Exception: Process executed with non-0 exit code 1: [u'/home/ato/1/gecko/src/mach', u'artifact', u'install'] File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/mach_commands.py", line 411, in build self._run_mach_artifact_install() File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/mach_commands.py", line 638, in _run_mach_artifact_install pass_thru=True, ensure_exit_code=True) File "/home/ato/1/gecko/src/python/mozbuild/mozbuild/base.py", line 642, in _run_command_in_srcdir return self.run_process(cwd=self.topsrcdir, **args) File "/home/ato/1/gecko/src/python/mach/mach/mixin/process.py", line 147, in run_process raise Exception('Process executed with non-0 exit code %d: %s' % (status, args)) And when I interrupt it with SIGINT, I get the following: ^?Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs Process Process-1: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/home/ato/1/gecko/src/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 86, in _collect while not pipe.poll(sleep_interval): KeyboardInterrupt func(*targs, **kargs) File "/usr/lib/python2.7/multiprocessing/util.py", line 325, in _exit_function p.join() File "/usr/lib/python2.7/multiprocessing/process.py", line 145, in join res = self._popen.wait(timeout) File "/usr/lib/python2.7/multiprocessing/forking.py", line 154, in wait return self.poll(0) File "/usr/lib/python2.7/multiprocessing/forking.py", line 135, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt Error in sys.exitfunc: Traceback (most recent call last): File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/usr/lib/python2.7/multiprocessing/util.py", line 325, in _exit_function p.join() File "/usr/lib/python2.7/multiprocessing/process.py", line 145, in join res = self._popen.wait(timeout) File "/usr/lib/python2.7/multiprocessing/forking.py", line 154, in wait return self.poll(0) File "/usr/lib/python2.7/multiprocessing/forking.py", line 135, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt
Assignee: nobody → cmanchester
Status: NEW → ASSIGNED
Flags: needinfo?(ato)
Reporter | ||
Comment 10•6 years ago
|
||
Of course I should have done a clobber first. After the clobber, I get a different error that I’m not sure is related to this bug? https://gist.github.com/andreastt/742335c6b460b8211dde
Flags: needinfo?(cmanchester)
Comment 11•6 years ago
|
||
(In reply to Andreas Tolfsen (:ato) from comment #10) > Of course I should have done a clobber first. After the clobber, I get a > different error that I’m not sure is related to this bug? > > https://gist.github.com/andreastt/742335c6b460b8211dde https://gist.github.com/andreastt/742335c6b460b8211dde#file-gistfile1-txt-L271 is pretty clear: 0:14.32 Try running |hg pushlogsync|; see https://developer.mozilla.org/en-US/docs/Artifact_builds It looks like something has gone wrong with your configuration, can you follow these instructions and make sure you're locally configured correctly?
Assignee | ||
Comment 12•6 years ago
|
||
Definitely try `hg pushlogsync` before proceeding (I recall needing to do this). Note also this isn't going to work as expected unless you're developing against fx-team (bug 1240667 tracks removing that requirement).
Flags: needinfo?(cmanchester)
Reporter | ||
Comment 14•6 years ago
|
||
nalexander, chmanchester: I should have read the earlier error message more carefully. Running `hg pushlogsync` as suggested first resolves that issue. With this change I’m able to do artifact-based builds on Linux, which is great news! Thanks!
Comment 15•6 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/a1c1d5cf87ce
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
status-firefox46:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla46
Updated•4 years ago
|
Product: Core → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•