885342 - Improve mozbase resolution of test dependencies

Status: RESOLVED INACTIVE

(Testing :: Mozbase, defect)

Description

[Jeff Hammel]

Some mozbase packages have tests that depend on mozbase packages
or 3rd party packages not in the package's under test
'setup.py'. These probably should be listed in the package's
'tests_require' setup keyword:
http://pythonhosted.org/distribute/setuptools.html#id8 , if only as an
indicator to the installer that these packages are necessary.
Ideally (IMHO), the tests should be runnable with `python setup.py
test` ( http://peak.telecommunity.com/DevCenter/setuptools#test ), but
this requires that `test_suite` be set for normal operation to the
string of a dot-resolvable module or that `--test-suite` be
specified.  But since our tests all live in the 'test/' setup.py-level
subdirectory (and not a normally importable module, basically, if
you'll accept my hand-waviness), this won't work.

A few alternatives:

- setup.py adds the test module, conditionally, to e.g. sys.path

- the test_loader keyword,
  http://peak.telecommunity.com/DevCenter/setuptools#test-loader ,
  could be conceivably used with a custom solution....somehow.  For
  instance, we could build a mini module just for mozbase testing.
  Or we could make the meta-package mozbase that would include this
  sort of thing and (also e.g.) be a place to consolidate things (such
  as setup_development.py, etc) and do project-wide sorts of things,
  typically automation for the project.  Other ideas are also
  possible.

- use an off-the-shelf solution and add that to `tests_require`.  Both
  `nose` and `unittest2` can do this (Examples:
  
  https://github.com/msabramo/logan/blob/5f88c65842a003adefb79d240b7e3c9eaff81806/setup.py
  and
  http://parijatmishra.wordpress.com/2008/10/08/python-packaging-setuptools-and-eggs/
  ).

This is probably all low priority.

Ref:
https://bugzilla.mozilla.org/show_bug.cgi?id=884344#c6
http://www.python.org/dev/peps/pep-0426/

Comment 1

[Jeff Hammel]

See also: https://wiki.mozilla.org/Auto-tools/Projects/Mozbase#Test_Python_Dependencies

The goal is to make things simpler for the cases we care about, so at worst we shouldn't make things worse wrt m-c testing, etc.  There are three steps (m-c, travis CI, setup_development.py) currently....we don't need more, and preferably less.

Comment 2

[Julien Pagès (:parkouss)]

Hi,

Putting current tests dir as a package (eg mozfile/tests, with an __init__.py file inside) may be a good idea I think. You just have then to add "test_suite='mozfile.tests'" in the setup.py file to be able to run tests with "python setup.py test", without needing unittest2 or nose.

As a bonus, it will facilitate python 3 support (if one day python 2 and 3 needs to be supported) as the directive "use_2to3" in setup.py file will then handle tests module automagically.

See https://pythonhosted.org/setuptools/python3.html for example.

I tried this with success:

cd testing/mozbase/mozfile
hg move tests mozfile
touch mozfile/tests/__init__.py

rm tests # just to be sure

# edit setup.py to add "test_suite='mozfile.tests'"

python setup.py test

# ...
Ran 22 tests in 2.330s

What do you think ?