Closed
Bug 869609
Opened 11 years ago
Closed 11 years ago
mail: Virtualenv failing on multi-arch Debian-based distros
Categories
(Thunderbird :: Testing Infrastructure, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
Thunderbird 24.0
People
(Reporter: tessarakt, Assigned: tessarakt)
References
Details
Attachments
(1 file, 2 obsolete files)
205.85 KB,
patch
|
standard8
:
review+
|
Details | Diff | Splinter Review |
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:22.0) Gecko/20130505 Firefox/22.0 Build ID: 20130505004014 Steps to reproduce: Tried to build recent comm-central. Actual results: Build failed with the following log entries: rm -rf ../../../mozilla/_tests/mozmill/../mozmill-virtualenv && \ mkdir ../../../mozilla/_tests/mozmill/../mozmill-virtualenv && \ unset MACOSX_DEPLOYMENT_TARGET && \ /home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_virtualenv/bin/python ../../../mozilla/_tests/mozmill/resources/installmozmill.py ../../../mozilla/_tests/mozmill/../mozmill-virtualenv Python: 2.7.4 (default, Apr 19 2013, 18:28:01) [GCC 4.7.3] Using real prefix '/usr' New python executable in /home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_tests/mozmill-virtualenv/bin/python Installing setuptools................................... Complete output from command /home/jens/devel/moz...irtualenv/bin/python -c "#!python \"\"\"Bootstra...sys.argv[1:]) " /home/jens/devel/moz...ols-0.6c11-py2.7.egg: Traceback (most recent call last): File "<string>", line 279, in <module> File "<string>", line 240, in main File "/home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_tests/mozmill/resources/virtualenv/virtualenv_support/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 1712, in main File "/home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_tests/mozmill/resources/virtualenv/virtualenv_support/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 1700, in with_ei_usage File "/home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_tests/mozmill/resources/virtualenv/virtualenv_support/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 1716, in <lambda> File "/usr/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/usr/lib/python2.7/distutils/dist.py", line 971, in run_command cmd_obj.ensure_finalized() File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_tests/mozmill/resources/virtualenv/virtualenv_support/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 125, in finalize_options File "/home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_tests/mozmill/resources/virtualenv/virtualenv_support/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 1121, in _expand File "/usr/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command cmd_obj.ensure_finalized() File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_tests/mozmill/resources/virtualenv/virtualenv_support/setuptools-0.6c11-py2.7.egg/setuptools/command/install.py", line 32, in finalize_options File "/usr/lib/python2.7/distutils/command/install.py", line 321, in finalize_options (prefix, exec_prefix) = get_config_vars('prefix', 'exec_prefix') File "/home/jens/devel/mozilla-comm-central/obj-mail/mozilla/_tests/mozmill-virtualenv/lib/python2.7/distutils/__init__.py", line 88, in sysconfig_get_config_vars real_vars = old_get_config_vars(*args) File "/usr/lib/python2.7/distutils/sysconfig.py", line 489, in get_config_vars func() File "/usr/lib/python2.7/distutils/sysconfig.py", line 433, in _init_posix from _sysconfigdata import build_time_vars File "/usr/lib/python2.7/_sysconfigdata.py", line 6, in <module> from _sysconfigdata_nd import * ImportError: No module named _sysconfigdata_nd ---------------------------------------- ...Installing setuptools...done. Traceback (most recent call last): File "virtualenv/virtualenv.py", line 2098, in <module> main() File "virtualenv/virtualenv.py", line 928, in main never_download=options.never_download) File "virtualenv/virtualenv.py", line 1040, in create_environment search_dirs=search_dirs, never_download=never_download) File "virtualenv/virtualenv.py", line 593, in install_setuptools search_dirs=search_dirs, never_download=never_download) File "virtualenv/virtualenv.py", line 567, in _install_req cwd=cwd) File "virtualenv/virtualenv.py", line 1006, in call_subprocess % (cmd_desc, proc.returncode)) OSError: Command /home/jens/devel/moz...irtualenv/bin/python -c "#!python \"\"\"Bootstra...sys.argv[1:]) " /home/jens/devel/moz...ols-0.6c11-py2.7.egg failed with error code 1 Failure to install virtualenv Expected results: Build should have been successful.
Assignee | ||
Comment 1•11 years ago
|
||
I replaced the contents of mail/test/resources/virtualenv with those of mozilla/python/virtualenv. Apparently this works now. Here the relevant part of the logfile: /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/config/nsinstall -R /home/jens/devel/mozilla-comm-central/mailnews/test/resources/* ../../../mozilla/_tests/mozmill/resources /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/config/nsinstall -R /home/jens/devel/mozilla-comm-central/mailnews/test/fakeserver/* ../../../mozilla/_tests/mozmill/resources /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/config/nsinstall -R /home/jens/devel/mozilla-comm-central/mail/base/test/unit/resources/* ../../../mozilla/_tests/mozmill/resources /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/config/nsinstall -t /home/jens/devel/mozilla-comm-central/mail/test/resources ../../../mozilla/_tests/mozmill rm -rf ../../../mozilla/_tests/mozmill/../mozmill-virtualenv && \ mkdir ../../../mozilla/_tests/mozmill/../mozmill-virtualenv && \ unset MACOSX_DEPLOYMENT_TARGET && \ /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_virtualenv/bin/python ../../../mozilla/_tests/mozmill/resources/installmozmill.py ../../../mozilla/_tests/mozmill/../mozmill-virtualenv Python: 2.7.4 (default, Apr 19 2013, 18:28:01) [GCC 4.7.3] Using real prefix '/usr' New python executable in /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin/python Overwriting /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/lib/python2.7/distutils/__init__.py with new content Installing setuptools............done. Installing pip...............done. Unpacking ./ManifestDestiny Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/ManifestDestiny Unpacking ./simplejson-2.1.6 Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/simplejson-2.1.6 Unpacking ./mozrunner Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/mozrunner Unpacking ./jsbridge Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/jsbridge warning: no files found matching '*' under directory 'jsbridge/xpi' Unpacking ./mozmill Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/mozmill Installing collected packages: ManifestDestiny, simplejson, mozrunner, jsbridge, mozmill Running setup.py install for ManifestDestiny Installing manifestparser script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Running setup.py install for simplejson building 'simplejson._speedups' extension x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c simplejson/_speedups.c -o build/temp.linux-x86_64-2.7/simplejson/_speedups.o x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/simplejson/_speedups.o -o build/lib.linux-x86_64-2.7/simplejson/_speedups.so Running setup.py install for mozrunner Installing mozrunner script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Installing addon_id script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Running setup.py install for jsbridge warning: no files found matching '*' under directory 'jsbridge/xpi' Installing jsbridge script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Running setup.py install for mozmill Installing mozmill-thunderbird script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Installing mozmill-restart script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Installing mozmill script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Successfully installed ManifestDestiny simplejson mozrunner jsbridge mozmill Cleaning up... make[5]: Leaving directory '/home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mail/test/mozmill' Is this ok? Or is there anything else I have to care about?
Assignee | ||
Comment 2•11 years ago
|
||
Assignee | ||
Updated•11 years ago
|
Attachment #751641 -
Flags: review?(bugspam.Callek)
Comment 3•11 years ago
|
||
Wouldn't it be better to just use the code from m-c?
Assignee | ||
Comment 4•11 years ago
|
||
Probably. That needs to be done by someone who understands the build system.
Assignee | ||
Updated•11 years ago
|
Attachment #751641 -
Flags: review?(bugspam.Callek)
Assignee | ||
Comment 5•11 years ago
|
||
Attachment #751641 -
Attachment is obsolete: true
Assignee | ||
Updated•11 years ago
|
Attachment #751650 -
Flags: review?(bugspam.Callek)
Assignee | ||
Comment 6•11 years ago
|
||
(In reply to Jens Müller from comment #4) > Probably. That needs to be done by someone who understands the build system. I guess in mail/resources/installmozmill.py the following bits need to be changed: # check for existence of necessary files if not os.path.exists('virtualenv'): print "File not found: virtualenv" sys.exit(1) and returncode = call([sys.executable, os.path.join('virtualenv', 'virtualenv.py'), destination], env=env) Is "os.path.join('..', '..', '..', 'mozilla', 'python', 'virtualenv')" the correct syntax?
Assignee | ||
Comment 7•11 years ago
|
||
(In reply to Jens Müller from comment #6) > Is "os.path.join('..', '..', '..', 'mozilla', 'python', 'virtualenv')" the > correct syntax? Apparently, it is more complex than that: The script is executed in the obj directory. And the mozilla/python/virtualenv is not copied to the obj dir.
Assignee | ||
Comment 8•11 years ago
|
||
OK, should not be too difficult. Apparently, the following block in mail/test/mozmill/Makefile.in needs to be changed. # Copy MozMill and its dependencies over, and set up a virtualenv. The # virtualenv directory is outside because we don't want to bundle it up during # stage-package. VIRTUALENV_DIR = $(_DEST_DIR)/../mozmill-virtualenv mozmill-virtualenv: NSDISTMODE=copy mozmill-virtualenv: $(INSTALL) $(topsrcdir)/mail/test/resources $(_DEST_DIR) rm -rf $(VIRTUALENV_DIR) && \ mkdir $(VIRTUALENV_DIR) && \ unset MACOSX_DEPLOYMENT_TARGET && \ $(PYTHON) $(_DEST_DIR)/resources/installmozmill.py $(VIRTUALENV_DIR) I will try that later on.
Assignee | ||
Updated•11 years ago
|
Attachment #751650 -
Flags: review?(bugspam.Callek)
Assignee | ||
Comment 9•11 years ago
|
||
OK, that one worked as well. I'll prepare the patch now. /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/config/nsinstall -t /home/jens/devel/mozilla-comm-central/mail/test/resources ../../../mozilla/_tests/mozmill rm -rf ../../../mozilla/_tests/mozmill/../mozmill-virtualenv && \ mkdir ../../../mozilla/_tests/mozmill/../mozmill-virtualenv && \ unset MACOSX_DEPLOYMENT_TARGET && \ /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_virtualenv/bin/python ../../../mozilla/_tests/mozmill/resources/installmozmill.py ../../../mozilla/_tests/mozmill/../mozmill-virtualenv Python: 2.7.4 (default, Apr 19 2013, 18:28:01) [GCC 4.7.3] Using real prefix '/usr' New python executable in /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin/python Installing setuptools............done. Installing pip...............done. Unpacking ./ManifestDestiny Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/ManifestDestiny Unpacking ./simplejson-2.1.6 Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/simplejson-2.1.6 Unpacking ./mozrunner Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/mozrunner Unpacking ./jsbridge Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/jsbridge warning: no files found matching '*' under directory 'jsbridge/xpi' Unpacking ./mozmill Running setup.py egg_info for package from file:///home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill/resources/mozmill Installing collected packages: ManifestDestiny, simplejson, mozrunner, jsbridge, mozmill Running setup.py install for ManifestDestiny Installing manifestparser script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Running setup.py install for simplejson building 'simplejson._speedups' extension x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c simplejson/_speedups.c -o build/temp.linux-x86_64-2.7/simplejson/_speedups.o x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/simplejson/_speedups.o -o build/lib.linux-x86_64-2.7/simplejson/_speedups.so Running setup.py install for mozrunner Installing mozrunner script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Installing addon_id script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Running setup.py install for jsbridge warning: no files found matching '*' under directory 'jsbridge/xpi' Installing jsbridge script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Running setup.py install for mozmill Installing mozmill-thunderbird script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Installing mozmill-restart script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Installing mozmill script to /home/jens/devel/mozilla-comm-central/obj-x86_64-unknown-linux-gnu/mozilla/_tests/mozmill-virtualenv/bin Successfully installed ManifestDestiny simplejson mozrunner jsbridge mozmill Cleaning up...
Assignee | ||
Comment 10•11 years ago
|
||
Attachment #751650 -
Attachment is obsolete: true
Attachment #751670 -
Flags: review?(bugspam.Callek)
Assignee | ||
Updated•11 years ago
|
Attachment #751670 -
Flags: review?(bugspam.Callek)
Assignee | ||
Comment 11•11 years ago
|
||
Comment on attachment 751670 [details] [diff] [review] Use virtualenv from Mozilla Do you prefer the added line with or without a / at the end?
Assignee | ||
Updated•11 years ago
|
Attachment #751670 -
Flags: review?(mbanner)
Assignee | ||
Updated•11 years ago
|
OS: All → Linux
Updated•11 years ago
|
Assignee: nobody → blog
Comment 12•11 years ago
|
||
Comment on attachment 751670 [details] [diff] [review] Use virtualenv from Mozilla This looks fine to me, I did have an idea about building mozmill into the actual virtualenv that m-c hooks up, but that's definitely a different bug, and re-using the m-c virtualenv is a big win. Thanks.
Attachment #751670 -
Flags: review?(mbanner) → review+
Updated•11 years ago
|
Status: UNCONFIRMED → NEW
Ever confirmed: true
Assignee | ||
Updated•11 years ago
|
Keywords: checkin-needed
Assignee | ||
Updated•11 years ago
|
Component: Build Config → Testing Infrastructure
Product: MailNews Core → Thunderbird
Comment 13•11 years ago
|
||
https://hg.mozilla.org/comm-central/rev/36be1bd38b3c
Status: NEW → RESOLVED
Closed: 11 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 24.0
You need to log in
before you can comment on or make changes to this bug.
Description
•