758835 - Add ability to flag tests as expected failures in the manifest

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Jonathan Griffin (:jgriffin)]

We should support an expectedfailure = true flag in the test manifest, which would cause the flagged test to be run as an expected failure, i.e., to fail if the test passes.

This would be the exact analog to the @unittest.expectedFailure decorator that is available to Python tests.

Comment 1

[Jonathan Griffin (:jgriffin)]

Whoops, wrong component.

Comment 2

[Jonathan Griffin (:jgriffin)]

Attachment: Add ability to mark tests as expected failures in the manifest,

Works well locally; will run through try shortly.

Comment 3

[Jonathan Griffin (:jgriffin)]

https://tbpl.mozilla.org/?tree=Try&rev=b7c415f938f2

Comment 4

[David Burns :automatedtester]

Comment on attachment 818159 [details] [diff] [review]
Add ability to mark tests as expected failures in the manifest,

Review of attachment 818159 [details] [diff] [review]:
-----------------------------------------------------------------

looks good

Comment 5

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/91993a2ea998

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/91993a2ea998

Comment 8

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/11c08b521d97