701110 - a user should be able to see "what's left to test?" on a test run

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: MozTrap, enhancement)

Description

[Cameron Dawson [:camd]]

For test runs, we need to be able to answer this question.  

I think this page should show a list of test cases, sortable by suites, tags and environment elements, showing what tests per environment are left to be tested.  it should be aggregated so that, if two testers are pending for the same test case/env combo, it only shows 1.

Test run details should have a link to it.  Possibly also the % complete chicklet.

Issues:
=======
the main challenge may be depicting this with a large env matrix...

Comment 1

[Cameron Dawson [:camd]]

A Pivotal Tracker story has been created for this Bug: http://www.pivotaltracker.com/story/show/20804951

Comment 2

[Cameron Dawson [:camd]]

Cameron Dawson added a comment in Pivotal Tracker:   
   
OK, I have a great idea I just stole from Mint.com:  This would be great as a dynamic pie chart that could show coverage.  See the attached picture.

What we have here is the environment categories listed on the left.  You can re-order them, as well as uncheck them to not be counted in the aggregation.  The first pie you will be shown will have the breakdown of each element in the first category.  It will show a split of run vs. not-run.  if you click a pie piece, say "Win" in this case, the pie changes to show the slices cut up by the next category (browser in this case) filtered by the first category element "win"  You can drill down as deep as you'd like to go.

Future: some day it would be amazing to click a pie-piece of not-run at any level, and have that create a link to that test run filtered to that environment and those not-run test cases.  But we don't really have this ability in the product, yet.

Eric: what do you think?

Comment 3

[Cameron Dawson [:camd]]

Eric Meyer added a comment in Pivotal Tracker:   
   
off-hand I think we could do something effectively the same, and more intuitive if less visually stunning, using the 'finder' UI we already have built. Each 'pie' is simply a column, and we add a tool for rearranging the columns. If we want the graph feel, it is a small matter turning each item into a bar graph showing completion.

Comment 4

[Cameron Dawson [:camd]]

Cameron Dawson added a comment in Pivotal Tracker:   
   
Eric: hmm... Interesting.  For each item, the bar would be something like this:  Windows [       82%      | 18% ]  That might work.  Yeah this might be a much easier way to get this feature.  But I think the pie would be so pretty.  I mean... pie, man.

Comment 5

[Cameron Dawson [:camd]]

Cameron Dawson added a comment in Pivotal Tracker:   
   
The rearranging tool could be very much like the multi-select rearranging.

Comment 6

[Cameron Dawson [:camd]]

Cameron Dawson added a comment in Pivotal Tracker:   
   
Eric:  yeah, let's go with your idea with the finder.  I wonder if, later, we could incorporate a non-interactive pie graph to update as you make finder choices...

Comment 7

[Cameron Dawson [:camd]]

Cameron Dawson added a comment in Pivotal Tracker:   
   
finder should allow you to easily nav to these pages
    need better icon for "show this list" in the finder.  maybe an "eye" for view or "reveal" spyglass
    each column is currently hard-coded in a path from product->version->run->etc..
        we should make it so that you can take different paths:
            product->version->run->cases
            product->version->run->suites->cases
            product->version->run->envs->cases
            product->version->run->envs->suites->cases
            product->version->run->suites->envs->cases
            etc...

Comment 8

[Stephen Donner [:stephend] Not actively reading bugmail]

Mass-closing remaining MozTrap bugs as WONTFIX, due to 1) the Mozilla-hosted instance being decommissioned (see https://wiki.mozilla.org/TestEngineering/Testrail), and, for now, 2) the still-up code archived at its GitHub page: https://github.com/mozilla/moztrap (we'll decide what's next for that, in the near future).

See also the history and more-detailed discussion which led us here, at https://groups.google.com/forum/#!topic/mozilla.dev.quality/Sa75hV8Ywvk

(If you'd like, you should be able to filter these notification emails using at least the unique string of "Sa75hV8Ywvk" in the message body.

Thanks!