1082297 - app-validator's Python dependencies are not getting downloaded

Status: RESOLVED WONTFIX

(Marketplace Graveyard :: Validation, defect, P5)

Description

[Christopher Van Wiemeersch [:cvan]]

Am I doing something wrong?

> % workon validator                                  /opt/app-validator (master)
> ERROR: Environment 'validator' does not exist. Create it with 'mkvirtualenv validator'.
> % mkvirtualenv validator --distribute               /opt/app-validator (master)
> New python executable in validator/bin/python2.7
> Also creating executable in validator/bin/python
> Installing distribute.................................................................................................................................................................done.
> Installing pip......................done.
> % workon validator                                  /opt/app-validator (master)
> (validator)% pip install -r requirements.txt        /opt/app-validator (master)
> Downloading/unpacking cssutils==0.9.7 (from -r requirements.txt (line 1))
>   Downloading cssutils-0.9.7.zip (447kB): 447kB downloaded
>   Running setup.py egg_info for package cssutils
>     
> Downloading/unpacking Fabric==0.9.0 (from -r requirements.txt (line 2))
>   Downloading Fabric-0.9.0.tar.gz (179kB): 179kB downloaded
>   Running setup.py egg_info for package Fabric
>     
>     warning: no previously-included files matching '*' found under directory 'docs/_build'
> Downloading/unpacking nose==1.0.0 (from -r requirements.txt (line 3))
>   Downloading nose-1.0.0.tar.gz (718kB): 718kB downloaded
>   Running setup.py egg_info for package nose
>     
>     no previously-included directories found matching 'doc/.build'
> Downloading/unpacking simplejson==2.3.0 (from -r requirements.txt (line 4))
>   Downloading simplejson-2.3.0.tar.gz (50kB): 50kB downloaded
>   Running setup.py egg_info for package simplejson
>     
> Downloading/unpacking argparse==1.1 (from -r requirements.txt (line 5))
>   Downloading argparse-1.1.zip (151kB): 151kB downloaded
>   Running setup.py egg_info for package argparse
>     
> Obtaining fastchardet from git+git://github.com/mattbasta/fastchardet#egg=fastchardet (from -r requirements.txt (line 6))
>   Cloning git://github.com/mattbasta/fastchardet to /Users/chris/.virtualenvs/validator/src/fastchardet
>   Running setup.py egg_info for package fastchardet
>     
> Downloading/unpacking mock==1.0b1 (from -r requirements.txt (line 7))
>   Downloading mock-1.0b1.tar.gz (806kB): 806kB downloaded
>   Running setup.py egg_info for package mock
>     
>     warning: no files found matching '*.png' under directory 'docs'
>     warning: no files found matching '*.css' under directory 'docs'
>     warning: no files found matching '*.html' under directory 'docs'
>     warning: no files found matching '*.js' under directory 'docs'
> Downloading/unpacking ndg-httpsclient==0.3.2 (from -r requirements.txt (line 8))
>   Downloading ndg_httpsclient-0.3.2.tar.gz
>   Running setup.py egg_info for package ndg-httpsclient
>     
> Downloading/unpacking pyasn1==0.1.7 (from -r requirements.txt (line 9))
>   Downloading pyasn1-0.1.7.tar.gz (68kB): 68kB downloaded
>   Running setup.py egg_info for package pyasn1
>     
> Downloading/unpacking pyOpenSSL==0.13.1 (from -r requirements.txt (line 10))
>   Downloading pyOpenSSL-0.13.1.tar.gz (254kB): 254kB downloaded
>   Running setup.py egg_info for package pyOpenSSL
>     
>     warning: no previously-included files matching '*.pyc' found anywhere in distribution
> Downloading/unpacking requests==2.0.0 (from -r requirements.txt (line 11))
>   Downloading requests-2.0.0.tar.gz (362kB): 362kB downloaded
>   Running setup.py egg_info for package requests
>     
> Downloading/unpacking Pillow==2.3.0 (from -r requirements.txt (line 12))
>   Downloading Pillow-2.3.0.zip (2.4MB): 2.4MB downloaded
>   Running setup.py egg_info for package Pillow
>     
> Downloading/unpacking pycrypto>=1.9 (from Fabric==0.9.0->-r requirements.txt (line 2))
>   Downloading pycrypto-2.6.1.tar.gz (446kB): 446kB downloaded
>   Running setup.py egg_info for package pycrypto
>     
> Installing collected packages: cssutils, Fabric, nose, simplejson, argparse, fastchardet, mock, ndg-httpsclient, pyasn1, pyOpenSSL, requests, Pillow, pycrypto
>   Running setup.py install for cssutils
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     warning: install_lib: byte-compiling is disabled, skipping.
>     
>     
>     Installing csscombine script to /Users/chris/.virtualenvs/validator/bin
>     Installing cssparse script to /Users/chris/.virtualenvs/validator/bin
>     Installing csscapture script to /Users/chris/.virtualenvs/validator/bin
>   Running setup.py install for Fabric
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     warning: install_lib: byte-compiling is disabled, skipping.
>     
>     
>     warning: no previously-included files matching '*' found under directory 'docs/_build'
>     Installing fab script to /Users/chris/.virtualenvs/validator/bin
>   Running setup.py install for nose
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     warning: install_lib: byte-compiling is disabled, skipping.
>     
>     
>     no previously-included directories found matching 'doc/.build'
>     Installing nosetests script to /Users/chris/.virtualenvs/validator/bin
>     Installing nosetests-2.7 script to /Users/chris/.virtualenvs/validator/bin
>   Running setup.py install for simplejson
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     building 'simplejson._speedups' extension
>     cc -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -I/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c simplejson/_speedups.c -o build/temp.macosx-10.8-intel-2.7/simplejson/_speedups.o
>     cc -bundle -undefined dynamic_lookup -L/usr/local/lib -arch i386 -arch x86_64 build/temp.macosx-10.8-intel-2.7/simplejson/_speedups.o -o build/lib.macosx-10.8-intel-2.7/simplejson/_speedups.so
>     warning: install_lib: byte-compiling is disabled, skipping.
>     
>     
>   Running setup.py install for argparse
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     warning: install_lib: byte-compiling is disabled, skipping.
>     
>     
>   Running setup.py develop for fastchardet
>     
>     Creating /Users/chris/.virtualenvs/validator/lib/python2.7/site-packages/fastchardet.egg-link (link to .)
>     Adding fastchardet 0.2.0 to easy-install.pth file
>     
>     Installed /Users/chris/.virtualenvs/validator/src/fastchardet
>   Running setup.py install for mock
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     warning: install_lib: byte-compiling is disabled, skipping.
>     
>     
>     warning: no files found matching '*.png' under directory 'docs'
>     warning: no files found matching '*.css' under directory 'docs'
>     warning: no files found matching '*.html' under directory 'docs'
>     warning: no files found matching '*.js' under directory 'docs'
>   Running setup.py install for ndg-httpsclient
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     Skipping installation of /Users/chris/.virtualenvs/validator/lib/python2.7/site-packages/ndg/__init__.py (namespace package)
>     warning: install_lib: byte-compiling is disabled, skipping.
>     
>     
>     Installing /Users/chris/.virtualenvs/validator/lib/python2.7/site-packages/ndg_httpsclient-0.3.2-py2.7-nspkg.pth
>     Installing ndg_httpclient script to /Users/chris/.virtualenvs/validator/bin
>   Running setup.py install for pyasn1
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     warning: install_lib: byte-compiling is disabled, skipping.
>     
>     
>   Running setup.py install for pyOpenSSL
>     warning: build_py: byte-compiling is disabled, skipping.
>     
>     building 'OpenSSL.crypto' extension
>     cc -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -I/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c OpenSSL/crypto/crypto.c -o build/temp.macosx-10.8-intel-2.7/OpenSSL/crypto/crypto.o
>     ...
>     
> Successfully installed cssutils Fabric nose simplejson argparse fastchardet mock ndg-httpsclient pyasn1 pyOpenSSL requests Pillow pycrypto
> Cleaning up...
> (validator)% workon validator                       /opt/app-validator (master)
> (validator)% nosetests                              /opt/app-validator (master)
> /usr/local/lib/python2.7/site-packages/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin django_statsd = django_statsd:NoseStatsd: Requested setting STATSD_PATCHES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
>   RuntimeWarning)
> EEEEEEEEEEEEEEEEEEEEEEEEEE.....EEEEEEEEEEEEEE
> ======================================================================
> ERROR: Failure: ImportError (No module named simplejson)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/site-packages/nose/loader.py", line 414, in loadTestsFromName
>     addr.filename, addr.module)
>   File "/usr/local/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
>     return self.importFromDir(dir_path, fqname)
>   File "/usr/local/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
>     mod = load_module(part_fqname, fh, filename, desc)
>   File "/opt/app-validator/appvalidator/__init__.py", line 1, in <module>
>     from validate import validate_app, validate_packaged_app
>   File "/opt/app-validator/appvalidator/validate.py", line 4, in <module>
>     import loader
>   File "/opt/app-validator/appvalidator/loader.py", line 1, in <module>
>     import testcases.content
>   File "/opt/app-validator/appvalidator/testcases/content.py", line 5, in <module>
>     import scripting as testendpoint_js
>   File "/opt/app-validator/appvalidator/testcases/scripting.py", line 2, in <module>
>     import javascript.acorn as acorn
>   File "/opt/app-validator/appvalidator/testcases/javascript/acorn.py", line 4, in <module>
>     import simplejson as json
> ImportError: No module named simplejson
> 
> ======================================================================
> ERROR: Failure: ImportError (No module named simplejson)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/site-packages/nose/loader.py", line 414, in loadTestsFromName
>     addr.filename, addr.module)
>   File "/usr/local/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
>     return self.importFromDir(dir_path, fqname)
>   File "/usr/local/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
>     mod = load_module(part_fqname, fh, filename, desc)
>   File "/opt/app-validator/tests/js/test_arrays.py", line 1, in <module>
>     from js_helper import TestCase
>   File "/opt/app-validator/tests/js/js_helper.py", line 8, in <module>
>     from .. import helper
>   File "/opt/app-validator/tests/helper.py", line 7, in <module>
>     from appvalidator.zip import ZipPackage
>   File "/opt/app-validator/appvalidator/__init__.py", line 1, in <module>
>     from validate import validate_app, validate_packaged_app
>   File "/opt/app-validator/appvalidator/validate.py", line 4, in <module>
>     import loader
>   File "/opt/app-validator/appvalidator/loader.py", line 1, in <module>
>     import testcases.content
>   File "/opt/app-validator/appvalidator/testcases/content.py", line 5, in <module>
>     import scripting as testendpoint_js
>   File "/opt/app-validator/appvalidator/testcases/scripting.py", line 2, in <module>
>     import javascript.acorn as acorn
>   File "/opt/app-validator/appvalidator/testcases/javascript/acorn.py", line 4, in <module>
>     import simplejson as json
> ImportError: No module named simplejson
> 
> ...

Comment 1

[Mathieu Pillard [:mat]]

I can't reproduce. Ran 687 tests in 10.391s with a freshly created virtualenv.

Full log is https://gist.github.com/diox/9a2fd7b72132ea9da752

Comment 2

[Andy McKay]

There's this cool docker thing that does this all for you.

Comment 3

[Wil Clouser [:clouserw]]

I tried this on linux.  Fresh virtualenv and install.  Didn't get any simplejson errors, but did get tons of errors:

> Ran 688 tests in 13.119s
> FAILED (errors=137, failures=130)

Comment 4

[Mathieu Pillard [:mat]]

Wil: that's probably because your 'js' binary doesn't point to spidermonkey. See https://bugzilla.mozilla.org/show_bug.cgi?id=971104

Comment 5

[Christopher Van Wiemeersch [:cvan]]

(In reply to Andy McKay [:andym] from comment #2)
> There's this cool docker thing that does this all for you.

This isn't the first time I've heard this from folks. I love that we're using Docker now, but should Docker be considered an alternative to code working out of the box? Am I wrong in thinking this?

I've not seen any other OSS project say, "our code works only in Docker! godspeed!"