1071227 - Modifications to mochitest for structured logging support in mozharness

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

Getting rid of the regex based output parser in mozharness is the end of the line for unstructured failure output from mochitest.

Comment 1

[Chris Manchester (limited bugmail, email directly)]

Attachment: Modifications to mochitests to use structured logging in mozharness

This is what I've found. Needs the mozharness piece to test. I'm skeptical an error from manifestLibrary.js would propagate correclty.

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Bug 1097278 highlights that we'll need an "end of test run" marker from javascript in place of the old summary so we don't end up with a green in these situations.

Comment 3

[Andrew Halberstadt [:ahal]]

This is definitely something we should do, but the StructuredOutputParser in mozharness should also have the ability to fallback to regexes to find errors in unstructured output. It kind of already does:
https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/structuredlog.py#49

If we still have a regex based fallback for unstructured output, we don't need to be super paranoid about errors slipping through the cracks.

Comment 4

[Paul Yang [:pyang] (away)]

//note to myself
working on poc for individual script to trigger mochitest with structuredLogOutputParser

Comment 5

[Paul Yang [:pyang] (away)]

while mochitest emits structured log, this will be solved once bug 1261194 fixed.
chmanchester, do you think we need patch for this bug?

Comment 6

[Chris Manchester (limited bugmail, email directly)]

So long as we're using the regex fallback, the patch in this bug isn't strictly necessary, but skimming it now I think it has some reasonable cleanups in it.

Comment 7

[Chris Manchester (limited bugmail, email directly)]

I'm not sure what the status of everything that was originally in this change, but it looks like most of this was subsumed by other work in the meantime.