Closed Bug 1191378 Opened 6 years ago Closed 6 years ago

Marionette job shows as success but contains failed steps

Categories

(Testing :: Marionette, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 1156070

People

(Reporter: jfrench, Unassigned)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Split off from unrelated TC bug 1190523, the Treeherder and Logviewer view of this Marionette test looks like a normal successful job, except it contains a 'show successful steps' - which contains failed steps.
https://treeherder.allizom.org/logviewer.html#?job_id=9175861&repo=mozilla-inbound

Ed suggested there might be a small adjustment possible on the harness side, to produce a blob that would include a suitable property (eg. testfailed) instead of 'success' in this case.
Attached image marionetteUnitTests
Per screen grab.
The job is marked as successful even though the log contains:

07:00:02 ERROR - TEST-START | test_errors.py TestAllExceptions.test_properties
07:00:02 ERROR - TEST-PASS | test_errors.py TestAllExceptions.test_properties | took 158ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_cause_string
07:00:02 ERROR - TEST-PASS | test_errors.py TestExceptionType.test_cause_string | took 166ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_cause_tuple
07:00:02 ERROR - TEST-PASS | test_errors.py TestExceptionType.test_cause_tuple | took 178ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_construction
07:00:02 ERROR - TEST-PASS | test_errors.py TestExceptionType.test_construction | took 214ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_defaults
07:00:02 ERROR - TEST-PASS | test_errors.py TestExceptionType.test_defaults | took 192ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_str 

Treeherder displays the job result according to what buildbot tells it, and buildbot gets that from mozharness/marionette/...
:philor diagnosed this correctly in bug 1156070
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1156070
You need to log in before you can comment on or make changes to this bug.