921145 - Use a more human-friendly reporter for travis

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Gareth Aye [:gaye] (back from PTO)]

The gaia-ui-tests use the same tbpl reporter they use on TBPL on travis. It's very spammy and difficult for me to parse. Would you all mind turning on something like mocha's dot or spec reporters for when the tests run on travis?

Comment 1

[James Lal [:lightsofapollo]]

I have no idea if there even are other reports for the runner?

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Can you give an example of what's difficult to parse and how you would suggest presenting it? We do have a --html-output option but that would require us to set up artifacts (http://about.travis-ci.org/blog/2012-12-18-travis-artifacts/) to see the ones generated by Travis. Here's an example of a HTML report: http://people.mozilla.org/~dhunt/gaiatest_results.html

Comment 3

[Zac C (:zac)]

Yes I thought the Travis reporting was very readable myself:

TEST-START test_settings_app_permissions.py
test_open_app_permissions (test_settings_app_permissions.TestAppPermissions) ... ok
----------------------------------------------------------------------
Ran 1 test in 12.372s


TBPL on the other hand is definitely an explosion of mostly useless noise!

Comment 4

[Gareth Aye [:gaye] (back from PTO)]

Wow the html output one is beautiful! I was looking for something more like the one we currently have turned on for test-agent or marionette js on travis.

Comment 5

[Gareth Aye [:gaye] (back from PTO)]

I would be overjoyed though if you set up a way for me to look at the html ones on every travis build though!

Comment 6

[James Lal [:lightsofapollo]]

We would need to setup this: http://about.travis-ci.org/blog/2012-12-18-travis-artifacts/

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Raised bug 925266 for Travis CI artifacts.

Comment 8

[David Burns :automatedtester]

Dependent bug was implemented so closing this one