625213 - Add a results pane for test runs

Status: VERIFIED DUPLICATE of bug 650969

(Mozilla QA Graveyard :: MozTrap, enhancement, P1)

Description

[Aakash Desai [:aakashd]]

The TCM will need a results pane for test run results collected from logged
in users. In this pane, the following will be needed:

* A table of results entries with values shown for name, failed, invalid and bugs
* Default list of 20 results shown
* A combobox to set the number of results per page going from 10,20,50 and 100.
When selected, the page should AJAX-ily update the listings area rather than
require a new page load.
* A checkbox per row entry when a logged in user is an admin
* Pagination that allows users to go to a certain page and to the beginning and
end of the set of results AJAX-ily.

Comment 1

[Aakash Desai [:aakashd]]

Attachment: mockup

Comment 2

[Carl Meyer (:carljm)]

We'll need a new API call for getting summary passed/failed/invalid stats on a test run.

Comment 3

[Carl Meyer (:carljm)]

From Matt's comments on #634966, it seems like it will be important to add drill-down from a test run to its component suites/test-cases, and from there to individual results. That drill-down isn't currently reflected in these bugs.

Comment 4

[Carl Meyer (:carljm)]

From comments on https://bugzilla.mozilla.org/show_bug.cgi?id=634966 it sounds like it will be important to report test-run results by test suite.

This will require API changes, because the platform doesn't currently store the suite grouping in the test run, it just dumps the contents of the suite into the run as individual test cases. So currently the only way to report test run results by suite, AFAIK, would be to iterate over every individual result in the test run and check its testcase for membership in every suite in the system. This is not feasible performance-wise.

Comment 5

[Matt Evans [:mevans]]

If we make test runs to only allow inclusion of test suites, then I think this would necessitate that the included test suite name(s) would be a data member list of the test run. Upon reporting, it would be a matter of bucketing the merged test case results per test suite name.

Comment 6

[Carl Meyer (:carljm)]

I'll make a note to Vadim of this requirement.

Comment 7

[Cameron Dawson [:camd]]

APIs:  As always: Carl, please advise

/testruns/ GET POST
/testruns/assignments/{assignmentId}/results/"
/testruns/results/"
/testruns/results/{resultId}/"
/testruns/results/{resultId}/approve/"
/testruns/results/{resultId}/environments/"
/testruns/results/{resultId}/retest/"
/testruns/{id}/"
/testruns/{id}/approveallresults/"
/testruns/{id}/components/"
/testruns/{id}/includedtestcases/"
/testruns/{id}/includedtestcases/testplan/{testPlanId}/"
/testruns/{id}/includedtestcases/testsuite/{testSuiteId}/"
/testruns/{id}/retest/"
/testruns/{id}/team/members/"
/testruns/{id}/testsuites/"

Comment 8

[Carl Meyer (:carljm)]

These look comprehensive, probably overly so. We probably won't be accessing individual results or includedtestcases or assignments on this screen, but as I mentioned above, we'll need a new API (that currently doesn't exist) for getting per-test-run summary results (thus this is blocked on #636566).

Comment 9

[Cameron Dawson [:camd]]

Finally finished implementing tests for most of the endpoints listed above.  Some we will not use at this time:

/testruns/{id}/components/"
/testruns/{id}/includedtestcases/testplan/{testPlanId}/"

Comment 11

[Matt Brandt [:mbrandt]]

Bumping to verified duplicate