792173 - Marionette should output TEST-UNEXPECTED-FAIL for all failures

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Jonathan Griffin (:jgriffin)]

TBPL provides links to failures, based on the string TEST-UNEXPECTED-FAIL.  We currently emit this for failing JS assertions, but not for failed Python tests.  We should add this.

Comment 1

[Jonathan Griffin (:jgriffin)]

I'm trying out a patch for this on try, which includes a deliberate test failure: 

https://tbpl.mozilla.org/?tree=Try&rev=257e2e949f34

Comment 2

[Jonathan Griffin (:jgriffin)]

Attachment: Add TEST-UNEXPECTED-FAIL output to Marionette,

This patch allows TBPL to correctly recognize failures in Marionette Python tests (it already handles them for JS tests).  It works as this try submission shows:  https://tbpl.mozilla.org/?tree=Try&rev=09060e7768e5&noignore=1.  The failure is a deliberate failure I added to that submission just to verify this code

Comment 3

[Malini Das [:mdas] - Away, not checking bugmail]

Comment on attachment 663131 [details] [diff] [review]
Add TEST-UNEXPECTED-FAIL output to Marionette,

Review of attachment 663131 [details] [diff] [review]:
-----------------------------------------------------------------

Mmm, flavoured error logs.

Comment 4

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/mozilla-central/rev/5dde4b86acfb