Open Bug 1301177 Opened 8 years ago Updated 1 year ago

Marionette harness tests run locally with mach should have same logs as in automation


(Testing :: Marionette Client and Harness, defect, P5)



(Not tracked)


(Reporter: vakila, Unassigned)



(Keywords: pi-marionette-harness-tests)

Once Bug 1285299 lands, the logs for the Marionette harness tests job (Mn-h) on Taskcluster will be in Mozlog's TBPL format, thanks to the new pytest_mozlog Pytest plugin which is becoming a part of the Mozlog package.

However, to use this plugin when running the tests locally with `mach python-test`, we need to manually register the pytest_mozlog plugin with Pytest, e.g. via extra command-line arguments in a pytest.ini file or the `if __name__` blocks of each test module, or by adding a `pytest_plugins` line to [1]. 

This is because of the way mach loads Python packages into the virtualenv [2], i.e. copying the modules into the path instead of installing them using their files. When the Mozlog package is installed with, as is the case in automation or if we populate a local virtualenv by hand to run the tests locally with `py.test`, Pytest finds the entry point defined in Mozlog's and registers the plugin. 

This creates a problem for mach compatibility, because if we try to register the plugin manually in the command-line args or, Pytest complains that the plugin is already registered. This means we essentially need to have two different configurations for running the tests, one for automation/py.test and one for mach, which is not idea.

We should change the way the pytest_mozlog plugin is registered when running the tests with mach so that it does not require command-line arguments or `pytest_plugins`. Possible approaches include:
- Change the way mach installs the Mozlog package to use "" instead of "python.pth" [3]
- Use pytest hook(s) (e.g. `pytest_configure`) in the harness tests' to register the pytest_mozlog plugin if not present (e.g. [4])




Depends on: 1285299
[mass update] Setting priority
Priority: -- → P5
Severity: normal → S3
Product: Testing → Remote Protocol
Component: Marionette → Marionette Client and Harness
Product: Remote Protocol → Testing
You need to log in before you can comment on or make changes to this bug.