618136 - Users should be able to choose test cycles to run based on selected configurations

Status: VERIFIED FIXED

(Mozilla QA Graveyard :: MozTrap, enhancement, P1)

Description

[Aakash Desai [:aakashd]]

Attachment: mock up

The test cycle chooser page offers users to select one or more test cycles to run in a given session. This page is loaded after configurations are selected (or after login if they've set a permanent configuration). There they can search within the list of cycles offered (via title, ID and creation date) as well as paginate on depending on the sort options available at the bottom of the page (same as the ones in the panels in "manage"). 

Non-Obvious Actions
Start - The list of testcases (in groups of 10's) are populated for a user to run through based off of the test cycles selected.

Comment 1

[Carl Meyer (:carljm)]

I think it's really "test runs" not "test cycles" that are being selected here, from the backend perspective. Not that it makes much difference to the user.

Is there a good reason why it needs to be possible to select multiple test runs at once? That complicates the UI significantly; unless it's a critical feature, I'd recommend we start with one test run at a time. Seems like you can just do a test run, when you're done get dumped back to this screen and choose the next one; adds very little time/click overhead, in exchange for a much simpler and more intuitive UI model.

Also, uTest raised a question about assignment of individual test cases. Since individual test cases within a test run can be assigned to individual testers, or can be self-assignable, they thought there needs to be an additional step once you select a test run, where you view a list of all the test cases in that test run that are either assigned to you or are self-assignable, and choose to self-assign whatever ones you want to do (rather than assuming the tester wants to automatically self-assign everything in the entire test run that is self-assignable).

Comment 2

[Aakash Desai [:aakashd]]

After a conversation on IRC, the data model doesn't support the wireframes thus far. So, lets try the following:

1. users pick the product 
2. users pick the cycles to run 
3. users picks the env vars for each run in the cycle
4. users run the tests

Comment 3

[Carl Meyer (:carljm)]

Aakash - can you respond to my other questions above, re whether it's really critical to be able to select multiple test-cycles at once (particularly as a core feature in early version), and how all of this interacts with assignment of test runs and testcases (where some may be self-assignable and some pre-assigned, even within the same test run)?

Comment 4

[Aakash Desai [:aakashd]]

Ah, yes sorry about that. It's not critical to select multiple test-cycles at once. Like you've mentioned before, the safer bet to iterate correctly is take out complexity. So, let's only allow a user to select one test cycle.


> how all of this interacts with assignment
> of test runs and testcases (where some may be self-assignable and some
> pre-assigned, even within the same test run)?

For .1, let's make everything self-assignable for now and add the ability to assign test cycles to specific people in .4.

Comment 5

[Carl Meyer (:carljm)]

> For .1, let's make everything self-assignable for now and add the ability to
> assign test cycles to specific people in .4.

So if someone selects a test cycle, you want to assume that they are self-assigning every test case within every test run of that cycle? No fine-grained selection of test runs or cases?

Comment 6

[Aakash Desai [:aakashd]]

> So if someone selects a test cycle, you want to assume that they are
> self-assigning every test case within every test run of that cycle? No
> fine-grained selection of test runs or cases?

Yep

Comment 7

[Carl Meyer (:carljm)]

There isn't a backend call in the uTest API for getting a percentage-progress bar as shown on the wireframe, and building that manually will probably require too many individual calls (to get all the test cases, check which ones have been executed, etc). So we may need to leave the progress bar out for now, unless it's a high priority.

Comment 8

[Aakash Desai [:aakashd]]

> So we may need to leave the progress bar out for now,
unless it's a high priority.

That's fine, just file a new bug to implement this in a later version.

Comment 9

[Carl Meyer (:carljm)]

Filed https://bugzilla.mozilla.org/show_bug.cgi?id=624856

Comment 10

[Carl Meyer (:carljm)]

HTML wireframe at http://tcm.oddsites.net/test/product/1/cycles/

Comment 12

[Matt Brandt [:mbrandt]]

Bumping to verified, this refers to an old version of the site that has since shipped.