907923 - Fix mozharness parsing of cppunittest output

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Dan Minor [:dminor]]

Currently the output parser in desktop_unittest.py decides that the cpp unittests fail collectively despite all of the tests passing individually. I think this is due to the word 'WARNING' showing up in the test output, but it might be something else.

The rununittest.py script itself returns 0 for success and 1 otherwise, but I don't want to ignore the parser completely and lose the ability to correctly recognize crashes and other problems.

Comment 1

[Dan Minor [:dminor]]

Updating summary as the changes to the parser required are in unittest.py and errors.py, not desktop_unittest.py.

Comment 2

[Dan Minor [:dminor]]

Attachment: Patch to handle parsing cppunittest output

I've added a regex for cppunittests and also changed the pass/fail count logic to handle test harnesses that do not keep track of these numbers.

Comment 3

[Aki Sasaki (not active)]

Comment on attachment 794033 [details] [diff] [review]
Patch to handle parsing cppunittest output

Discussed in IRC:
I think a preferable solution would be to get the cpp test harness to behave similarly to the others.  If we hit issues with that, we still have this approach as a second option.

Comment 4

[Dan Minor [:dminor]]

Attachment: Patch to add support to parse cppunittest output.

Bug 908644 adds the summary to cppunittest we parse with this patch.

Comment 5

[Aki Sasaki (not active)]

Comment on attachment 794788 [details] [diff] [review]
Patch to add support to parse cppunittest output.

Thanks!

Comment 6

[Aki Sasaki (not active)]

Comment on attachment 794788 [details] [diff] [review]
Patch to add support to parse cppunittest output.

https://hg.mozilla.org/build/mozharness/rev/008b723bdd3a

Comment 7

[John Hopkins (:jhopkins)]

In production