1046242 - mozlog.structured's mach formatter should print a list of failed tests at the end of a run

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

Something like what was implemented in bug 1041944 will probably make sense for other users of the mach formatter.

Comment 1

[James Graham [:jgraham]]

Attachment: Provide a run summary in the structuredlog mach formatter.

Not very well tested yet, but worth taking for a spin.

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8465493 [details] [diff] [review]
Provide a run summary in the structuredlog mach formatter.

Review of attachment 8465493 [details] [diff] [review]:
-----------------------------------------------------------------

This looks good. I'd love to see unit tests for the various scenarios for runs with/without subtests.

::: testing/mozbase/mozlog/mozlog/structured/formatters/machformatter.py
@@ +18,5 @@
>      return '%2d:%05.2f' % (minutes, seconds)
>  
> +class NullTerminal(object):
> +    def __getattr__(self, name):
> +        return self._id

This seems a little tricky, but I see the appeal. Could shorten to 'return lambda x: x'.

Comment 3

[James Graham [:jgraham]]

> > +class NullTerminal(object):
> > +    def __getattr__(self, name):
> > +        return self._id
> 
> This seems a little tricky, but I see the appeal. Could shorten to 'return
> lambda x: x'.

That wouldn't work as written because it wouldn't be a bound method. It's possible to do that way ofc, but I think this is easier.

I'll write some tests.

Comment 4

[James Graham [:jgraham]]

Attachment: Provide a run summary in the structuredlog mach formatter.

Comment 5

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8467872 [details] [diff] [review]
Provide a run summary in the structuredlog mach formatter.

Review of attachment 8467872 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozbase/mozlog/tests/test_structured.py
@@ +469,5 @@
> +        lines = self.output_file.read().split("\n")
> +
> +        self.assertIn("OK", lines)
> +        self.assertIn("Expected results: 5", lines)
> +        self.assertIn("Unexpected results: 0", lines)

It would be good to have something to express that test/subtest names end up in the summary.

Comment 6

[James Graham [:jgraham]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ac00c8a8a7a6

Comment 7

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/ac00c8a8a7a6