1055087 - Mozlog should log a smaller set of statuses for compatibility with tbpl

Status: RESOLVED INCOMPLETE

(Testing :: Mozbase, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

See https://bugzilla.mozilla.org/show_bug.cgi?id=1050177#c7

I have something that implements this in a patch for bug 1055070. The requirement as I understand it is to log a smaller set of statuses (tbpl knows about "PASS" and "FAIL", pretty much), and to propagate a failing status from subtests to the test_end message.

Comment 1

[Ahmed Kachkach [:akachkach]]

Isn't the part about propagating a failing status from subtests to test_end already done by the test harnesses currently?

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Attachment: Log a smaller set of statuses to conform to legacy output for tbpl.

I think this can be simpler, but here it is on try with a timeout and failure:

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=38f9e397b11f

Comment 3

[Chris Manchester (limited bugmail, email directly)]

The failures are in a different chunk. Here:

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=7bd73cdd89c5

Comment 4

[Chris Manchester (limited bugmail, email directly)]

(In reply to Ahmed Kachkach [:akachkach] from comment #1)
> Isn't the part about propagating a failing status from subtests to test_end
> already done by the test harnesses currently?

I mean only logging "TEST-PASS" and "TEST-UNEXPECTED-FAIL" in test_end.

Comment 5

[James Graham [:jgraham]]

Hmm, why do we want to propagate the status from subtests to the test_end? I'm not thrilled about that.

Comment 6

[James Graham [:jgraham]]

(but I can probably be convinced that it's OK to do this for mochitest for legacy reasons, if it's something we can remove once mozharness uses the raw logs to set the job status, which should be a high priority).

Comment 7

[Chris Manchester (limited bugmail, email directly)]

I guess this isn't about mozharness. It's a requirement of the logs as they are today that they preserve the meaning of mozharness' parsers for all the relevant cases. Anywhere they aren't should be fixed in an urgent manner. This is more about keeping the logs looking similar to the way they did before, and possibly a precautionary measure with respect to the log parsers we haven't yet discovered. 

Making things look like they did before in a way that reduces the amount of information they contain is pretty lame, but it's just a formatter.

Comment 8

[Ahmed Kachkach [:akachkach]]

We didn't have any TEST-UNEXPECTED-FAIL on the subtest level before? If it's the case then this change makes sense since it would give the "right" number of errors in mozharness (before we finally get to refactor it)

Comment 9

[James Graham [:jgraham]]

I'm guessing this bug isn't relevant any more.