355585 - feature request: ability for other users to continue with a test suite run

Status: RESOLVED WONTFIX

(Webtools Graveyard :: Litmus, enhancement, P5)

Description

[juan becerra [:juanb]]

Recently we needed to run some smoketests, and a community member finished about half of the tests before we even started. But then our tester had to go and we wanted to continue the test run from where he left off. Litmus allows you to see coverage percentages, but there's no easy way to find out which tests remain to be done.

It would be useful to have the option to continue test runs that were not completed or even a way to get a list of the tests that remain to be done for a specific run.

Comment 1

[Chris Cooper [:coop] (he/him)]

How about I add another icon to indicate community coverage? I can also lookup who exactly has submitted the results for a given testcase, and display the list as a dropdown (or similar) when you mouseover/click the icon.

Comment 2

[Chris Cooper [:coop] (he/him)]

Planning to fix this concurrently with bug 353377.

Comment 3

[Wayne Mery (:wsmwk)]

Coop, Marcia, might this be possible to have within the next month?  (I see priority was reduced a few days ago)  Or is there an alternative?

This will really help with Thunderbird QA testing management and user environment because testing is distributed amongst volunteers - both statically/pre-assigned and dynamically (changes over time).  (contrast this where it looks like FF milestone QA testing is preassigned)

Examples:
- next person fills in the holes where previous person didn't test
- joe does some testing Friday, and wants to resume and do the rest on Sunday
- QA lead wants to see which tests of BFT suite were completed, and not completed, for Mac PPC  

Will this bug and bug 353377 also address the last point?

Comment 4

[Wayne Mery (:wsmwk)]

Do you want the following as a seperate bug?  

On the page shown after committing test results for a test group...


    Show me the report for this Test Run.
is misleading - clicking shows results for *all* testruns, not just this user's test run.


    Continue testing this Test Run under the same configuration.
could be offset/spaced from the rest of the choices so it stands out better for the new and casual tester, since it should be the nominal choice for anyone doing a full test run.

Comment 5

[Marcia Knous [:marcia]]

Wayne: Regarding Comment 3, the Litmus maintenance work will be transitioning over to the webdev team. I am not sure if they will have the resources to work on P5 requests. Right now we are just trying to target the highest priority Litmus requests, which are mainly ones blocking work. coop and I triaged all the litmus bugs while we were at the Summit, and we evaluated what we thought was reasonable given the resources we have for Litmus at this time.

I would suggest making Comment 4 a separate bug.

Comment 6

[Wayne Mery (:wsmwk)]

(In reply to comment #4)
> On the page shown after committing test results for a test group...

filed Bug 475777

Comment 7

[Chris Cooper [:coop] (he/him)]

Litmus is being decommissioned in bug 802674. 

Visit https://moztrap.mozilla.org for your testing needs.