1294652 - [mozlog] HTMLFormmater does not handle for subtests(test_status) log

Status: NEW

(Testing :: Mozbase, defect, P3)

Description

[JunHo Seo]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36

Steps to reproduce:

Run Web Platform Test(wpt), wptrunner with Output Logging --log-html option
wpt: https://github.com/w3c/web-platform-tests
wptrunner: https://github.com/w3c/wptrunner


Actual results:

Assume that there are 10 tests, and 2 tests have 5 subtests, all test passed.
A html output reports
1) in summary, there are 10 tests, no shows how many parents and subtests.
2) in results table, total rows in table only 10.


Expected results:

A html output should report
  1) in summary, there are 20 tests(10 parents, 10 subtests). 
  2) in results table, total rows in table should be 15.

One real example in wpt, pointerlock test result is that there are 2 tests, just 2 rows in table. But, it should show that there are 28 tests(2 parents, 26 subtets) in summary, also 28 rows in table.

The cause is HTMLformatter does not handle test_status log. test_status log has each subtest result. We should make HTMLformatter handle test_status log, also apply each status count.

Comment 1

[JunHo Seo]

Attachment: Fix_HTMLFormatter.patch

I submit for fix this issue. Please look at this.

Comment 2

[JunHo Seo]

"2) in results table, total rows in table should be 15."

Mistake. 15 -> 20.

Comment 3

[JunHo Seo]

PTAL :)

Comment 4

[James Graham [:jgraham]]

Comment on attachment 8780430 [details] [diff] [review]
Fix_HTMLFormatter.patch

Review of attachment 8780430 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozbase/mozlog/mozlog/formatters/html/html.py
@@ +105,5 @@
> +            tc_time = (data["time"] - self.start_times.pop(data["test"])) / 1000.
> +            self.last_times.pop(data["test"])
> +            test_name = data["test"]
> +        else:   # 'test_status'
> +            tc_time = (data["time"] - self.last_times[data["test"]]) / 1000.

I think this is kind of misleading because tests can run many subtests concurrently. I guess not outputting a time for these tests would be better.