Currently when a test unexpectedly passes, the following is written to the xml file (this is an example): <testcase classname="test_cards_view_with_three_apps.TestCardsViewThreeApps" name="test_cards_view" time="0"> <failure message="test failure">N/A</failure> </testcase> It would be preferable to record the failure as: <failure message="test failure">Unexpected Pass</failure> This would make reviewing the failures on Jenkins more straighforward. Dave Hunt pointed at two different components which come into play: http://hg.mozilla.org/mozilla-central/file/default/testing/marionette/client/marionette/runtests.py#l726 https://github.com/mozilla/mozbase/blob/57a1c6dd00cc945e3274b0b17aba93fb3000cc43/moztest/moztest/results.py#L316
I think the recent change to structured logging will have resolved this. Ahal, is that true? If so can you close this bug?
Yes, I believe the built-in marionette formatters have been replaced by the mozlog equivalents, in this case: http://mozbase.readthedocs.org/en/latest/mozlog_structured.html#mozlog.structured.formatters.xunit.XUnitFormatter That being said, I've never used that formatter, so maybe it has the same problem. I think if it does we can open a new bug for it.