Open Bug 1025051 Opened 10 years ago Updated 2 years ago

Mozrunner's device runner shouldn't add test harness specific handlers

Categories

(Testing :: Mozbase, defect)

Product:
Testing
Component:
Mozbase
Type:
defect

Tracking

(Not tracked)

People

(Reporter: ahal, Unassigned)

References

Details

Currently mozrunner's device runner adds an output handler that retrieves the last run test. It also adds handlers for test timeouts. This is a problem because the assumption that the runner is even being used to run tests in the first place is invalid. This logic should either live in the various test harnesses themselves, or some kind of shared testing module in mozrunner itself. If we do the latter, I think it would be reasonable to assume the harness is using the standard structured logging protocol instead of relying on regex.
Specifically this bug is talking about the on_timeout and on_output methods here: http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozrunner/mozrunner/base/device.py#79 Ahmed, cc'ing you as this bug is about moving those methods somewhere else (like in the mochitest harness) and it may impact your structured logging work. For now I think it is ok for you to ignore them though.
As a counterpoint, I would like to move more things like this into mozrunner, although I am fine with having them on a MozRunner subclass or whatever. There's lots of shared logic between our various test harnesses, and MozRunner feels like a good place to put it. (If people object, we could invent a moztestrunner module and stick it there.)
I always thought stuff like this could go into moztest [1]. Though I think no one (myself included) really understands what the intention/scope of moztest is or was. Maybe we should have a discussion on how mozrunner/moztest/harness' fit in with one another. I do agree that there is value in keeping these handlers and other stuff like them in a shared location. Whether that's a mozrunner subclass, moztest or somewhere else are all fine with me. [1] http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/moztest/moztest/
That seems as good a place as any.
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.