801973 - Running setup_development.py from tests.zip tries to install mozinfo from pypi

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Jonathan Griffin (:jgriffin)]

If you run 'setup_development.py' from the tests/mozbase folder of a tests.zip package, it will attempt to install mozinfo using easy_install, rather than from the mozinfo package at tests/mozbase/mozinfo.

This impacts the ability to use setup_development.py from within a mozharness script.  Currently attempting to do so will generate:

18:02:05     INFO - Running command: ['/home/jgriffin/mozharness/scripts/build/venv/bin/python', '/home/jgriffin/mozharness/scripts/build/tests/mozbase/setup_development.py'] in /home/jgriffin/mozharness/scripts/build/venv
18:02:05     INFO - Copy/paste: /home/jgriffin/mozharness/scripts/build/venv/bin/python /home/jgriffin/mozharness/scripts/build/tests/mozbase/setup_development.py
18:02:06     INFO -  error: can't create or remove files in install directory
18:02:06     INFO -  The following error occurred while trying to add or remove files in the
18:02:06     INFO -  installation directory:
18:02:06     INFO -      [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-31602.pth'
18:02:06     INFO -  The installation directory you specified (via --install-dir, --prefix, or
18:02:06     INFO -  the distutils default setting) was:
18:02:06     INFO -      /usr/local/lib/python2.7/dist-packages/
18:02:06     INFO -  Perhaps your account does not have write access to this directory?  If the
18:02:06     INFO -  installation directory is a system-owned directory, you may need to sign in
18:02:06     INFO -  as the administrator or "root" account.  If you do not have administrative
18:02:06     INFO -  access to this machine, you may wish to choose a different installation
18:02:06     INFO -  directory, preferably one that is listed in your PYTHONPATH environment
18:02:06     INFO -  variable.
18:02:06     INFO -  For information on other options, you may wish to consult the
18:02:06     INFO -  documentation at:
18:02:06     INFO -    http://packages.python.org/distribute/easy_install.html
18:02:06     INFO -  Please make the appropriate changes for your system and try again.
18:02:06     INFO -  Traceback (most recent call last):
18:02:06     INFO -    File "/home/jgriffin/mozharness/scripts/build/tests/mozbase/setup_development.py", line 236, in <module>
18:02:06     INFO -      main()
18:02:06     INFO -    File "/home/jgriffin/mozharness/scripts/build/tests/mozbase/setup_development.py", line 228, in main
18:02:06     INFO -      call(['easy_install', version])
18:02:06     INFO -    File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
18:02:06     INFO -      raise CalledProcessError(retcode, cmd)
18:02:06     INFO -  subprocess.CalledProcessError: Command '['easy_install', 'mozinfo']' returned non-zero exit status 1
18:02:06    ERROR - Return code: 1

Comment 1

[Jonathan Griffin (:jgriffin)]

Adding some debugging statements to setup_development.py confirms that mozinfo is the only package that it attempts to install using easy_install in this scenario.

Comment 2

[Aki Sasaki (not active)]

Our current workaround here is to specify a pypi_url, e.g. http://hg.mozilla.org/build/mozharness/file/ee49beb18cbe/configs/talos/linux_config.py#l12

That will prevent trying to hit pypi iirc.

Comment 3

[Jeff Hammel]

My guess is that some dependency in mozbase is not pegged to the version of mozinfo in-tree.  running `setup_development.py --info` should display what is going to be installed in the order in which it will be.  :jgriffin, do you happen to know which package it is on when trying to install mozinfo? I will try to figure out what is going on here as soon as I can

Comment 4

[Jonathan Griffin (:jgriffin)]

setup_development.py -d:

ManifestDestiny: []
mozfile: []
mozhttpd: []
mozinfo: []
mozInstall: ['mozinfo == 0.4', 'mozfile']
mozlog: []
mozprocess: ['mozinfo']
mozprofile: ['ManifestDestiny >= 0.5.4']
mozrunner: ['mozinfo == 0.4', 'mozprocess == 0.7', 'mozprofile == 0.4']

setup_development.py --list:

mozhttpd
mozfile
mozlog
mozprocess
ManifestDestiny
mozprofile
mozinfo
mozInstall
mozrunner

Fixing this is not super high priority, just nice to have so we can get away from a flat package list in the Marionette and B2G mozharness scripts.

Comment 5

[Jeff Hammel]

Fixing this is actually a high priority for me, since this bug means the mozbase tree is currently in a state of insanity ;)

Comment 6

[Jonathan Griffin (:jgriffin)]

Attachment: Fix setup_development.py's handling of versionless deps

Comment 7

[Jeff Hammel]

Comment on attachment 671937 [details] [diff] [review]
Fix setup_development.py's handling of versionless deps

Oh me and my awful misspullungs :(  Thanks for the fix, jgriffin!

Comment 8

[Jonathan Griffin (:jgriffin)]

https://github.com/mozilla/mozbase/commit/aa50f7cf0f67b5762277bbafe5eecdd65440ed4b