916772 - Importing objects from project folder while running gaiatests results in a import error

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Florin Strugariu [:Bebe]]

I get a strange import error while trying to write tests using gaiatest pypi package.

Repo:
https://github.com/bebef1987/marketplace-tests-gaia

to install gaia tests run:
pip install -Ur requierments.txt

to run tests:
gaiatest --address=localhost:2828 --type=b2g-xfail --restart --testvars=testvars.json tests/manifest.ini 


When I run tests I get the flowing error:


(mk)florinstrugariu@P5171:~/marketplace-tests-gaia$ gaiatest --address=localhost:2828 --type=b2g-xfail --restart --testvars=testvars.json tests/manifest.ini 
starting httpd
running webserver on http://192.168.77.232:60940/
TEST-START test_marketplace_login.py
Traceback (most recent call last):
  File "/home/florinstrugariu/.virtualenvs/mk/bin/gaiatest", line 9, in <module>
    load_entry_point('gaiatest==0.14', 'console_scripts', 'gaiatest')()
  File "/home/florinstrugariu/.virtualenvs/mk/local/lib/python2.7/site-packages/gaiatest-0.14-py2.7.egg/gaiatest/runtests.py", line 280, in main
    cli(runner_class=GaiaTestRunner, parser_class=GaiaTestOptions)
  File "/home/florinstrugariu/.virtualenvs/mk/local/lib/python2.7/site-packages/marionette/runtests.py", line 786, in cli
    runner = startTestRunner(runner_class, options, tests)
  File "/home/florinstrugariu/.virtualenvs/mk/local/lib/python2.7/site-packages/marionette/runtests.py", line 778, in startTestRunner
    runner.run_tests(tests)
  File "/home/florinstrugariu/.virtualenvs/mk/local/lib/python2.7/site-packages/gaiatest-0.14-py2.7.egg/gaiatest/runtests.py", line 144, in run_tests
    MarionetteTestRunner.run_tests(self, tests)
  File "/home/florinstrugariu/.virtualenvs/mk/local/lib/python2.7/site-packages/marionette/runtests.py", line 423, in run_tests
    self.run_test(test)
  File "/home/florinstrugariu/.virtualenvs/mk/local/lib/python2.7/site-packages/marionette/runtests.py", line 511, in run_test
    self.run_test(i["path"])
  File "/home/florinstrugariu/.virtualenvs/mk/local/lib/python2.7/site-packages/marionette/runtests.py", line 526, in run_test
    **self.test_kwargs)
  File "/home/florinstrugariu/.virtualenvs/mk/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 286, in add_tests_to_suite
    test_mod = imp.load_source(mod_name, filepath)
  File "/home/florinstrugariu/marketplace-tests-gaia/tests/test_marketplace_login.py", line 8, in <module>
    from apps.marketplace.app import Marketplace
ImportError: No module named apps.marketplace.app

Comment 1

[Florin Strugariu [:Bebe]]

jgriffing can you take a look over it please. davehunt said you will certainly have some thoughts

Comment 2

[Florin Strugariu [:Bebe]]

The issue here is that the sys.path doesn’t contain ./src and ./test directories so basically python doesn't know where to search for the imports.

I tried to do something like:

sys.path.append(os.getcwd()) 

somewhere before we run the tests:
https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/runtests.py#L177

the issue here is if we run the tests from a different place (not the repo folder) then we will have issues

Ideally we need to get the test folder and src folder and add it to sys.path

Comment 3

[Jonathan Griffin (:jgriffin)]

If you intend to use gaiatest from other than the repo path and you want to import something from it, you must specify the package name.

E.g., if you want apps.marketplace.app, "apps" only means something if there's a package at the cwd that is named "apps", or there's a package that's been installed by that name.  Neither is true in your case.

Instead, you must include the package it belongs to, which _has_ been installed:

from gaiatest.apps.marketplace.app import Marketplace

Comment 4

[Florin Strugariu [:Bebe]]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/12250#attch-to-bugzilla

Pointer to Github pull-request

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 805841 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/12250#attch-to-bugzilla

Reading through the good practices of py.test (which we're essentially emulating here) I think the best approach is to create a setup.py for our classes supporting the tests (app/mock objects, etc).

> * putting tests into an extra directory outside your actual application
>   code, useful if you have many functional tests or want to keep
>   tests separate from actual application code::
> 
>     mypkg/
>         __init__.py
>         appmodule.py
>     tests/
>         test_app.py
>         ...
> 
> In both cases you usually need to make sure that ``mypkg`` is importable, for example by using the setuptools ``python setup.py develop`` method.

From py.test good practices: https://bitbucket.org/hpk42/pytest/src/5239f31ebf82eee712fc9f54f414e34afdfc1ddf/doc/en/goodpractises.txt?at=default#cl-145

I've convinced myself this is a bad idea, and that we're probably not following good practice in our Selenium tests. I know creating a setup.py and having to install the package is overhead, but it is minimal.

I think I already know how Jonathan feels about the hack in this patch, but would be good to clarify.

Comment 6

[Jonathan Griffin (:jgriffin)]

I agree with Dave.  Making your packages importable modules is the standard method for achieving your goal in Python; pytest-style hacks may be appealing, but I'm afraid would lead to confusion with other people who aren't familiar with them and expect things to behave in the standard Pythonic way.

Given that we want these tests and all their objects to be very accessible to others (developers and partners, etc), I think following Python best practices is a good idea unless there's a compelling reason not to.

Also, having a setup.py allows your package to track dependencies correctly, which may be something you'll want, sooner or later.

Comment 7

[Florin Strugariu [:Bebe]]

Thanks for the feedback all!!

As we all agree that setup.py is the way to go I will close this bug as wontfix