mach mochitest should exit with non-0 status code if tests fail

RESOLVED FIXED in mozilla24

Status

Testing
Mochitest
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: gps, Assigned: gps)

Tracking

(Blocks: 1 bug)

Trunk
mozilla24
Points:
---
Bug Flags:
in-testsuite -

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [mach])

Attachments

(1 attachment)

mach currently exits with the status code returned by the test runner. This is the status code returned by the app. testsuite-targets.mk greps over process output and exits with 1 if there was a test failure. mach should do the same.
Created attachment 753342 [details] [diff] [review]
mach mochitest-* now emits proper exit code on test failure

Not sure if it's obvious from the diff context, but test_output is the
filtered output of all unexpected messages. If there is a test failure,
test_output will be defined.
Attachment #753342 - Flags: review?(Ms2ger)
Assignee: nobody → gps
We could (and maybe should) push this down into the Mochitest harness code, so that runTests returns the proper status for success or failure, and then mach could just exit with that.
Comment on attachment 753342 [details] [diff] [review]
mach mochitest-* now emits proper exit code on test failure

Review of attachment 753342 [details] [diff] [review]:
-----------------------------------------------------------------

This should eventually be in runtests.py, along with the UNEXPECTED grepping, but this works for now.
Attachment #753342 - Flags: review?(Ms2ger) → review+
Verified this stops now on a unexpected failure
Blocks: 875388
https://hg.mozilla.org/integration/mozilla-inbound/rev/43886dce099a
Status: NEW → ASSIGNED
Flags: in-testsuite-
Target Milestone: --- → mozilla24
https://hg.mozilla.org/mozilla-central/rev/43886dce099a
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.