777841 - Write mozharness script for desktop Marionette tests

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Jonathan Griffin (:jgriffin)]

This is needed to get Marionette tests running in TBPL for the debug desktop Firefox builds.

Comment 1

[Jonathan Griffin (:jgriffin)]

Attachment: patch v0.1

This patch provides a mozharness script for running Marionette tests on desktop Firefox.  It works OK locally.

I still need to add some log parsing and more detailed status output, probably similar to that used for unittests in https://bug650880.bugzilla.mozilla.org/attachment.cgi?id=642861.  Is there an ETA for bug 650880 to land, or should I roll my own code and not wait for that?

Comment 2

[Aki Sasaki (not active)]

Comment on attachment 647282 [details] [diff] [review]
patch v0.1

> +from mozharness.base.log import DEBUG, INFO, WARNING, ERROR

Nit: WARNING is unused.

> +        [["--marionette_address"],

Style nit: --marionette-address

> +        {'regex': re.compile(r'''^INFO '''), 'level': DEBUG},

The above will, by default, hide all lines that start with INFO.
Is that the desired behavior?

> +            all_actions=['clobber',
> +                         'pull',

I think you're missing a pull method.

We'll probably need config files at some point.

I haven't run this, and I may be overlooking something, but overall this looks like a very clean and well-written mozharness script.


> Is there
> an ETA for bug 650880 to land, or should I roll my own code and not wait for
> that?

Jordan is still refactoring some code and hasn't gotten an r+ yet.
I'd suggest either a) working with Jordan to get something stable shared; or b) if it's still in flux and you need something sooner, maybe parallel solutions that merge later.

Comment 3

[Jonathan Griffin (:jgriffin)]

Attachment: patch v0.2

Addressed aki's comments.  Will work on some TBPL/logging code, and merge with Jordan's patch later as appropriate.

Comment 4

[Jonathan Griffin (:jgriffin)]

Attachment: patch v0.3

This patch adds a TinderboxPrint line to the output, so TBPL can figure out the status.  It does this by creating an OutputParser subclass, and then refactoring OutputParser and run_command just a bit so that you can use such a class.

I like this solution better than requiring that the output be iterated through multiple times (once by OutputParser via run_command, and again by my own parsing code); it seems more efficient.

Comment 5

[Aki Sasaki (not active)]

Comment on attachment 647732 [details] [diff] [review]
patch v0.3

I still haven't run this, but this looks good.
The run_command/OutputParser changes should be helpful in the future, too; thanks for those.

Comment 6

[Jonathan Griffin (:jgriffin)]

http://hg.mozilla.org/build/mozharness/rev/b77e44a6fcce

I'll file a separate bug for Marionette mozharness config files.