804886 - Add more verbose information which packages are getting installed from which location

Status: RESOLVED WORKSFORME

(Testing :: Mozbase, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Right now setup_development.py has to be run from the same folder. To make it easier for everyone in setting up a virtual environment we should allow to get this run from whatever the CWD for the user is.

The attached patch will fix the little mistake, which went probably in with the implementation.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Patch

Pointer to Github pull-request

Comment 2

[Jeff Hammel]

I'm not actually sure what the problem is or how this fixes.  how does specifying the cwd for easy_install fix anything?

Also, please make patches not pull requests as per https://wiki.mozilla.org/Auto-tools/Projects/MozBase

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Jeff Hammel [:jhammel] from comment #2)
> I'm not actually sure what the problem is or how this fixes.  how does
> specifying the cwd for easy_install fix anything?

When you check all the other calls to call() you will notice that all of them are specifying the CWD. Only the one for easy_install is missing it which causes a failure to actually install the packages.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Patch v1

Comment 5

[Jeff Hammel]

So setup.py has a "bug"/(feature) that you are required to be in its CWD when you run e.g. `python setup.py develop` .  `easy_install` in `setup_development.py` should only be used to install dependencies from pypi.  To my knowledge, e.g. `easy_install simplejson` should not require a particular CWD.

Could you please paste the failure so that I can see what is failing here?

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

$ python /data/code/mozbase/setup_development.py
Version: mozinfo
Searching for mozinfo
Reading http://pypi.python.org/simple/mozinfo/
Download error: [Errno 8] nodename nor servname provided, or not known -- Some packages may not be found!
Couldn't find index page for 'mozinfo' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://pypi.python.org/simple/
Download error: [Errno 8] nodename nor servname provided, or not known -- Some packages may not be found!
No local packages or download links found for mozinfo
Best match: None
Traceback (most recent call last):
  File "/Volumes/data/envs/aa/bin/easy_install", line 8, in <module>
    load_entry_point('setuptools==0.6c11', 'console_scripts', 'easy_install')()
  File "/Volumes/data/envs/aa/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 1712, in main
  File "/Volumes/data/envs/aa/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 1700, in with_ei_usage
  File "/Volumes/data/envs/aa/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 1716, in <lambda>
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Volumes/data/envs/aa/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 211, in run
  File "/Volumes/data/envs/aa/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 434, in easy_install
  File "/Volumes/data/envs/aa/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/package_index.py", line 475, in fetch_distribution
AttributeError: 'NoneType' object has no attribute 'clone'
Traceback (most recent call last):
  File "/data/code/mozbase/setup_development.py", line 237, in <module>
    main()
  File "/data/code/mozbase/setup_development.py", line 229, in main
    call(['easy_install', version])
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['easy_install', 'mozinfo']' returned non-zero exit status 1

Comment 7

[Jeff Hammel]

So there are a few things going on here.  This is failing (directly) because pypi isn't resolving:

'Download error: [Errno 8] nodename nor servname provided, or not known -- Some packages may not be found!'

The messages afterward are mostly known but annoying setuptools eccentricities (e.g. "Download error: [Errno 8] nodename nor servname provided, or not known -- Some packages may not be found!")

However, a deeper problem is why is mozinfo being installed from pypi and easy_install at all!  This seems like a deeper bug.  setup_development.py should install all packages actually in mozbase from the local files, including mozinfo.

We should add more diagnoses to setup_development so we can at least know what package is being installed which triggers mozinfo to be installed from pypi

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I can't reproduce it anymore. Not sure what happened when I was running the command while I was in offline mode. Lets update the summary so we can have a better patch for making it more verbose.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Closing as WFM given that it is no longer reproducible.