Mach formatter has forgotten how to count

RESOLVED FIXED in mozilla34

Status

RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: jgraham, Assigned: jgraham)

Tracking

unspecified
mozilla34
x86_64
Linux
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

4 years ago
It thinks that 0 subtests passed, always.
(Assignee)

Comment 1

4 years ago
Created attachment 8472341 [details] [diff] [review]
Count number of passing subtests in mach formatter
Attachment #8472341 - Flags: review?(cmanchester)
Comment on attachment 8472341 [details] [diff] [review]
Count number of passing subtests in mach formatter

Review of attachment 8472341 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozbase/mozlog/mozlog/structured/formatters/machformatter.py
@@ +221,5 @@
>                                                             data["expected"],
>                                                             data.get("message", "")))
> +        if data["status"] == "PASS":
> +            self.status_buffer[test]["pass"] += 1
> +

It's a little confusing that pass + len(unexpected) == count except if we have expected failures/timeouts/etc. or unexpected passes. I can imagine looking at the summary line in test end and wondering why they don't add up.
Attachment #8472341 - Flags: review?(cmanchester) → review+
(Assignee)

Comment 3

4 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/a87da9366285

Agreed that the summary can be confusing if you don't realise it's trying to tell you two different things: how you did on the tests in an absolute sense (the number that passed) and how you did relative to expectations (the number of unexpecteds). I'm not sure if we can present those two things in a clearer way or if one of them isn't that useful.

Comment 4

4 years ago
https://hg.mozilla.org/mozilla-central/rev/a87da9366285
Assignee: nobody → james
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla34
You need to log in before you can comment on or make changes to this bug.