379030 - Allow same reporting options for testdays and test runs

Status: RESOLVED WONTFIX

(Webtools Graveyard :: Litmus, enhancement, P3)

Description

[Chris Cooper [:coop] (he/him)]

Now that test runs have landed, testdays have become a bit redundant (which makes sense because they were always a stop-gap measure for test runs).

I think the concept of a testday is still valid, but we can now create more targeted testing efforts with test runs, so we should consider rolling the existing testday reporting (which I think is quite useful) up into test runs and doing away with testdays in Litmus. We could still refer to testdays in the wikis and blog, but our testing links would point to specific test runs instead.

For comparison, here are the various things you can specify for a testday vs. a test run:

Testday:
Start/Finish timestamp
Product
Branch
Testgroup (single)
Build ID (single)
Locale

Test Run:
Start/Finish timestamp
Product
Branch
Testgroup (multiple)
Build ID (multiple)
Platform (multiple)
Opsys (multiple)

The only thing that test runs are missing is a specific locale flag, i.e. locale is not specified at the creation of the test run, but users do input their locale as they are actually running tests.

Does anyone have any feelings about whether testdays can serve a purpose distinct from test runs, or whether it is OK to merge the two?

Comment 1

[Marcia Knous [:marcia]]

if the test runs will allow us to have a rolled up summary of test day activities similar to what we have now, that is fine with me. Having a snapshot of what was tested in a single page is useful.

Comment 2

[Carsten Book [:Tomcat]]

I like the result pages like http://litmus.mozilla.org/test_run_report.cgi?test_run_id=3 - maybe a summary like this would also a good idea for testdays.

Comment 3

[Chris Cooper [:coop] (he/him)]

I'm morphing this bug a bit. I feel both testdays and test runs are useful as distinct events, but we do want to be able to see the same statistics for both.

Comment 4

[Chris Cooper [:coop] (he/him)]

Oops.

Comment 5

[Chris Cooper [:coop] (he/him)]

Litmus is being decommissioned in bug 802674. 

Visit https://moztrap.mozilla.org for your testing needs.