807123 - Write new testrunner for Marionette

Status: RESOLVED WONTFIX

(Remote Protocol :: Marionette, defect)

Description

[Jonathan Griffin (:jgriffin)]

Currently Marionette uses Python unittest's TextTestRunner as its testrunner.  Unfortunately, there are a lot of tweaks we've needed to make to this, and others we'd like to make, but unittest.TextTestRunner isn't very easy to tweak.

It's possible to override methods (such as we do at http://mxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/runtests.py#117), but it isn't possible to do this without a lot of copy and pasting, since the methods are not structured to be easily overridden.  Additionally, the methods are not consistent between Python 2.6 and 2.7, and we need to support both for now.

Because of these things, I think we'd be better off writing an entirely new testrunner.

Possibly, this could eventually grow into a moztestrunner component, but for now I'm primarily interested in getting something more usable for Marionette.

Comment 1

[mathieu bultel:mbu]

Hi Jonathan,

I would like to focus on this bug. Could you give more explanation of what you really need by refactoring the testrunner ?

Comment 2

[Jonathan Griffin (:jgriffin)]

Since I've filed this bug, Marionette has evolved quite a bit, and I think we're stuck with Python's unittest, since we have a lot of consumers of it, and changing would be disruptive.

Instead of tackling this, then, would you be interested at looking at bugs that deal with improving the way we use unittest?  I'm thinking of things like bug 874599, and bug 807095.

Comment 3

[mathieu bultel:mbu]

thanks you Jonathan for your comment.
I'll take a look on this two issues (bug 874599, and bug 807095).