Open Bug 1294652 Opened 3 years ago Updated 10 months ago

[mozlog] HTMLFormmater does not handle for subtests(test_status) log

Categories

(Testing :: Mozbase, defect, P3)

51 Branch
defect

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: sejunho, Assigned: sejunho)

Details

Attachments

(1 file)

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36

Steps to reproduce:

Run Web Platform Test(wpt), wptrunner with Output Logging --log-html option
wpt: https://github.com/w3c/web-platform-tests
wptrunner: https://github.com/w3c/wptrunner


Actual results:

Assume that there are 10 tests, and 2 tests have 5 subtests, all test passed.
A html output reports
1) in summary, there are 10 tests, no shows how many parents and subtests.
2) in results table, total rows in table only 10.


Expected results:

A html output should report
  1) in summary, there are 20 tests(10 parents, 10 subtests). 
  2) in results table, total rows in table should be 15.

One real example in wpt, pointerlock test result is that there are 2 tests, just 2 rows in table. But, it should show that there are 28 tests(2 parents, 26 subtets) in summary, also 28 rows in table.

The cause is HTMLformatter does not handle test_status log. test_status log has each subtest result. We should make HTMLformatter handle test_status log, also apply each status count.
I submit for fix this issue. Please look at this.
Attachment #8780430 - Flags: review?(james)
"2) in results table, total rows in table should be 15."

Mistake. 15 -> 20.
Assignee: nobody → sejunho
PTAL :)
Comment on attachment 8780430 [details] [diff] [review]
Fix_HTMLFormatter.patch

Review of attachment 8780430 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozbase/mozlog/mozlog/formatters/html/html.py
@@ +105,5 @@
> +            tc_time = (data["time"] - self.start_times.pop(data["test"])) / 1000.
> +            self.last_times.pop(data["test"])
> +            test_name = data["test"]
> +        else:   # 'test_status'
> +            tc_time = (data["time"] - self.last_times[data["test"]]) / 1000.

I think this is kind of misleading because tests can run many subtests concurrently. I guess not outputting a time for these tests would be better.
Attachment #8780430 - Flags: review?(james) → review+
Component: Untriaged → DOM
Product: Firefox → Core
Component: DOM → Mozbase
Product: Core → Testing
Priority: -- → P3
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
You need to log in before you can comment on or make changes to this bug.