Closed Bug 802742 Opened 7 years ago Closed 6 years ago
Marionette should list all failed tests at the end of the testrun
Right now you only get a very brief summary for the test results at the end of a testrun. If there are failing tests it's very hard to figure out which of those are failing. You would have to scrape the whole console output: SUMMARY ------- passed: 108 failed: 2 todo: 0 It would be nice if Marionette could print a list of those failed tests at the end.
For anyone interested in this bug, please follow https://developer.mozilla.org/en-US/docs/Marionette/Developer_setup first to get your dev environment set up. You'll need to get it set up to work with the client, not the server. Generally speaking, to solve this bug, we'll need to modify (mozilla-central)/testing/marionette/client/marionette/runtests.py so that when the MarionetteTestRunner runs each test, it will gather all failed tests, and when it is done the suite, it should print out the failed tests. MarionetteTestRunner creates MarionetteTextTestRunner objects to run the tests, so you'll need to modify MarionetteTextTestRunner to gather the information we need.
Whiteboard: [good first bug][lang=python][mentor=mdas]
Hello, The bug seems very insteresting and I would like to fix it,This'll be my first bug, so I'd need some guidance.Thanks:)
Bebe wants to work on this, and asked if this should be default behaviour or triggered by an option. I think having an option, say --list-failed or maybe --verbose, would be ideal. It would list the failed tests. I'd like this to list the failure messages as well, so instead of having them interlaced with the test output, it can instead be printed at the end. This could be in addition to listing the failed tests if this is too difficult for a first bug.
Assignee: nobody → florin.strugariu
Did anyone respond to Mike Ling from comment 2, who also expressed an interest in working on this bug?
(In reply to Dave Hunt (:davehunt) from comment #4) > Did anyone respond to Mike Ling from comment 2, who also expressed an > interest in working on this bug? As I talked with mdas last night MikeLing is woking on some other bugs and this issue is free. MikeLing if you you are still working on this I'm more than happy to leave to you
This is what I did for now. mdas can you review it and check if I'm on the right track? After that I can add the other options
(In reply to Florin Strugariu [:Bebe] from comment #6) > Created attachment 771580 [details] [diff] [review] > Bug 802742 - Marionette should list all failed tests at the end of the > testrun > > This is what I did for now. > mdas can you review it and check if I'm on the right track? After that I can > add the other options This looks good, and actually, I spoke with jgriffin and we can just have the failed tests be listed by default. We can ignore the command line requirement, and ignore the error message printing. As such, your patch's complexity is greatly reduced:)
Comment on attachment 773179 [details] [diff] [review] Bug 802742 - Marionette should list all failed tests at the end of the testrun Review of attachment 773179 [details] [diff] [review]: ----------------------------------------------------------------- great, thanks! tested with python2.6 as well. I'll push this out to m-i soon.
Attachment #773179 - Flags: review?(mdas) → review+
pushed to inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/69e7468ac58f
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla25
You need to log in before you can comment on or make changes to this bug.