Marionette job shows as success but contains failed steps

RESOLVED DUPLICATE of bug 1156070

Status

Testing
Marionette
RESOLVED DUPLICATE of bug 1156070
3 years ago
3 years ago

People

(Reporter: jfrench, Unassigned)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
Split off from unrelated TC bug 1190523, the Treeherder and Logviewer view of this Marionette test looks like a normal successful job, except it contains a 'show successful steps' - which contains failed steps.
https://treeherder.allizom.org/logviewer.html#?job_id=9175861&repo=mozilla-inbound

Ed suggested there might be a small adjustment possible on the harness side, to produce a blob that would include a suitable property (eg. testfailed) instead of 'success' in this case.
(Reporter)

Comment 1

3 years ago
Created attachment 8643767 [details]
marionetteUnitTests

Per screen grab.

Comment 2

3 years ago
The job is marked as successful even though the log contains:

07:00:02 ERROR - TEST-START | test_errors.py TestAllExceptions.test_properties
07:00:02 ERROR - TEST-PASS | test_errors.py TestAllExceptions.test_properties | took 158ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_cause_string
07:00:02 ERROR - TEST-PASS | test_errors.py TestExceptionType.test_cause_string | took 166ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_cause_tuple
07:00:02 ERROR - TEST-PASS | test_errors.py TestExceptionType.test_cause_tuple | took 178ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_construction
07:00:02 ERROR - TEST-PASS | test_errors.py TestExceptionType.test_construction | took 214ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_defaults
07:00:02 ERROR - TEST-PASS | test_errors.py TestExceptionType.test_defaults | took 192ms
07:00:02 ERROR - TEST-START | test_errors.py TestExceptionType.test_str 

Treeherder displays the job result according to what buildbot tells it, and buildbot gets that from mozharness/marionette/...
:philor diagnosed this correctly in bug 1156070
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1156070
You need to log in before you can comment on or make changes to this bug.