302688 - Update interface to improve QA workflow

Status: RESOLVED FIXED

(Webtools Graveyard :: Litmus, defect)

Description

[Zach Lipton [:zach]]

As should be painfully obvious, Litmus is in need of some serious UI work. This
bug is a place for us to work, share ideas, and collaborate. 

Let's start with the test display page. A static HTML copy with a couple of
testcases and the Litmus stylesheet is attached to this bug. Download the file
and start hacking! When you've got something you're happy with, post it on your
website and post a comment here with the URL or just attach the mockup to this
bug. We can all look at our various ideas and mockups, take the best elements
from each one, and create a new template file for Litmus to use.

I've posted a couple of ideas below to get everyone started.

Comment 1

[Zach Lipton [:zach]]

Attachment: The existing test display page

Here's our starting point: the existing test display page.

Comment 2

[Zach Lipton [:zach]]

Attachment: Tracy Walker's rough multitest mockup

One of the things we want to look at here is how best to display more than one
testcase. A simple list of tests just isn't going to cut it because it's too
cluttered and confusing. One idea is to only show one test at a time, and use
JavaScript to have next/previous buttons that move between the tests (each test
would be a div). 

Tracy Walker created this mockup (attached), which while very rough,
illustrates one implementation of this idea. It would be great if we took some
of the ideas from this under consideration when creating and discussing new
designs.

Comment 3

[Zach Lipton [:zach]]

A couple of "guidelines" before we get too far into this: 

(I use the word guideline pretty loosely here. A better term might be "things to
think about.")

* Litmus's future users are already accustomed to Bugzilla. Thus, Litmus's UI
should be consistant with that of Bugzilla, in that they should have a similar
overall style. Obviously, Bugzilla is far from pefect in terms of UI, and this
is not to say that we should be making the same mistakes as Bugzilla, but
Bugzilla's look-and-feel is a good thing to keep in mind. 

* While the initial users of Litmus will all be using recent versions of
Firefox, Litmus is designed to be used by others. As such, pages should work
equally well in Internet Explorer 6. 

* The existing HTML is quite horrid. Any redesign should clean this up as much
as possible and use CSS for style. Note that I am certainly not opposed to
tables when we are actually representing tabular data; the table tag is by no
means a Bad Thing. That being said it would be good to use CSS for style and
positioning where it makes sense. 

* Lastly, thank you all for your help! I'm looking forward to seeing what we can
come up with. 

Comment 4

[Leroy]

http://www.phatfool.com/mozilla/litmus_draft1/

First 'mockup' and a few notes.

I've split/moved the javascript and css to their own files, see inc/script.js
and inc/style.css respectively. JavaScript has not been modifed or been added
to. All css has been rewritten except for the mozilla banner css, see note in file. 

Ive kept it simple, no images just css and html, but this does require
javascript to be enabled to open/view the tests. Tested in Firefox1.0.6, IE6 and
IE7b1.

Comments appreciated, good or bad.

I've got a few more ideas but I'll wait and see what the feedback is with this
draft before I work them.

Comment 5

[Roman Rudenko]

Here is my attempt to provide a better prototype of group and subgroup selection 
(NOT actual test display). Should be more comfortable for more lengthy lists of 
categories, and for 3-4 levels of hierarchy. Inspired by http://alistapart.com/
articles/complexdynamiclists/ (and uses modified JS from there).

http://24.80.239.218:5190/litmus/run_tests.html

Feel free to tell me that I am a talentless hack :)

Comment 6

[Tracy Walker [:tracy]]

A comment on my mockup: After using testrunner for hundreds of smoketests and
BFT's I found it cumbersome to have to scroll through a long list of full
testcase instructions. Also Mike B mentioned in an email that long test case
lists would be imposing and undesirable in our testcase presentation. But I also
found it helpful to know what was coming up during in a test run, especially in
closely related tescases.  Two birds with one stone theory often works to speed
up testing. I envisioned the small "Preview" blocks on the left side of the page
as a scrolling mechanism to browse through the test case list. Click on the
testcase in the center (or dynamically update the main view) to make it the
current testcase. Perhaps the "Preview Bar" could be toggled on and off?

Keeping the Preview Bar in mind during rewriting of the BFT's for FX, I
attempted to make the brief descriptions more concise.  Many were previously
unclear as to what the test case really was about.  I still have some work to do
in that area.  Anyway, I ramble. 

I've seen a dynamic scrolling preview mechanism is some web interface, but I
can't recall for the life of me what the application was.  But I know it can be
done by people with the right knowledge and creativity. 

Comment 7

[Zach Lipton [:zach]]

Leroy - Cool! I like the idea of doing collapsible testcase boxes and you've
managed to make it look good to boot. I think this is a good starting point for
all of us to look at, and it's something that we could even deploy immediately
while we work out a more fancy dynamic preview mechanism. I think it might be
best though to get away from the linear model of having a long listing of
testcases, and look at something that's more along the lines of what Tracy
describes at https://bugzilla.mozilla.org/show_bug.cgi?id=302688#c6. 

Roman - You are by no means a talentless hack; that's really awesome. I think it
would be best though if we did one page at a time, at least at first, just so I
don't completely loose my sanity. If things progress nicely from here, we can
even split off the work into a couple of different bugs and and double-up. Keep
that mockup around though; it looks great. 

Thanks to both of you for your hard work. Now I'm really getting excited :-)

Comment 8

[Zach Lipton [:zach]]

Also, as a general note for everyone, keep in mind that these pages are
dynamically generated by the cgi scripts. This means that if it would help you
in your work to have some additional content generated (say, a javascript array
structure containing all the testcase data), then that is absolutely doable. For
now, just mockup whatever that content would look like using sample data for a
few testcases, and we can turn it into a template later once we've refined the
final design. 

Comment 9

[Leroy]

Zach, thanks for the comments. 

After reading Tracy's comments (and hopefully I've understood you correctly
Tracy) I've just finished a second draft for you to review. I don't think it's
exactly what was wanted as it doesn't contain a list but I'll work on that part
in draft3.

Here's draft 2 .. http://www.phatfool.com/mozilla/litmus_draft2/

CHANGES

- rewritten the way testcases are displayed. Shows the first testcase and using
a simple menu at the top right of the testcase you can move to the next or
previous testcase.

- ability to show all testcases, click 'show all' in the test result menu (top
right)

- ability to show only one test result after clicking 'show all', this will show
only the test result of the one you have clicked

- removed 'Product', 'Group' and 'Subgroup' from each testcase due to
duplication from test summary (from what I noticed)

- submit results are per testcase

KNOWN ISSUES

- scrollbar increases when you navigate using 'next', adding white space to the
bottom of the page, only happens in Firefox, IE6/7 seems to be fine (rendering
bug?).

- 'Test ID:' cell doesnt abide by the 200px width setting in IE7b1 only.

CODE / DRAFT NOTES

- code needs to be cleaned up a bit .. as in js moved to its own file with a
function written, styles moved to the stylesheet and general js/html optimizing.
Also <form> code needs to be re-written due to 'submit results' button moved per
testcase instead of global.

- Currently the content is static but the code can quite easily be used with the
database, everything is laid out ready.


again, comments appreciated good or bad.

Comment 10

[Roman Rudenko]

Zach: In that case, could you change title of this bug to "Style Litmus test 
display page"?

Leroy: 
1. "Group: Smoketests; Subgroup:Install and Startup" part needs rework. This is 
part of "where am I?", so it should be brought together with testcase name 
(another "where am I?"). Suggested layout:
Kill group/subgroup
Amend testcase line to say "Smoketests » Install and Startup » Testcase: #1 - 
Start"
2. There is little reason for having separate "Pass" and "Pass with Bug". Can a 
testcase "Fail with Bug"? Sounds reasonable to me. I would suggest treating list 
of bug numbers as another optional data field, and leave "Pass/Fail/Not run/Test 
broken or unclear" independent from it.
3. If "Not Run" means "I skipped this test", it should be selected by default, 
and possibly moved to top or bottom.
4. What does "Submit Results" button do? Submit a single testcase? Submit them 
all, and leave page? If single, can't it autosubmit of next/previous? If all, 
why is it within testcase box?

Comment 11

[hhh]

Hi, my name is Hasan (call me Hass, like the avocado). I hope I can be helpful!

Zach, thanks for taking such initiative. Fantastic work, everyone! Here's my
first contribution. Just some very initial html recoding of Zach's attachment so
that it validates. Zach, what is the attribute/property wrap="soft"? It wouldn't
validate so I removed it for now. It's my opinion that most of this html if not
all can be recoded w/out tables and that stylesheets should go on hold until the
html is settled on. Opinions?

Hass 

Comment 12

[hhh]

Attachment: 1st recode of primary attachment

Comment 13

[hhh]

My apologies,  Tracy, Leroy and Roman's excellent mockups ALL validate. I still
would love to see this project incorporate as much standards-aware markup as
possible and so my point about the tables (and all that styling in the html)
still stands.

Comment 14

[Asa Dotzler [:asa]]

I want to say (and I'm not actually looking at the code here) that there is very
little value in removing tables when you're actually dealing with tabular data
(like we are in our testcase lists, I contend) and I think it's actually
problematic to go overboard on the de-tabling of content only for the sake of
de-tabling. 

If there are cases where we can more easily transform the layout of the page
_in_necessary_ways_ by moving to css based layout, then let's do that but please
do not pull simple tabular data out of tables without some actual necessity or
serious value in the results. Most browsers understand tables and do a fairly
decent job with them and I'm loathe to add a bunch of hacks just to make our
layout functional in more than Firefox. 

Comment 15

[hhh]

So valid html transitional and let the styling resume?

Comment 16

[Leroy]

Following from the second draft I've now created a third with a list of the
testcases while keeping with the same format for viewing each test case in draft 2.

Draft 3 can be found here .. http://www.phatfool.com/mozilla/litmus_draft3/

The code (css & js) needs some serious optimizing but as it's getting late here
(just gone midnight) I'll work on that tomorrow.

CHANGES

- Introduced a list/menu of test cases on the left, click [show>>] to change the
test case, you can also move through the testcases using the menu/navigation
(located to the top right of the testcase)

- Menu/navigation updated, move to the next or previous test case. 'Show All'
will show every test case as before, then clicking 'Show One' will hide all test
cases except the test case that was displayed before the 'Show All', hope that
makes sense.

- Changed submit buttons text on feedback given, also added a "Submit All
results" to the bottom of the page.

KNOWN ISSUES

- none, scrollbar issue seems have been fixed, ie7b1 still messing up the 200px
width, ie7 specific.

CODE / DRAFT NOTES

- As mentioned above the code needs some serious optimization work as its
currently bloated, I'll work on that tomorrow at some point.


Roman, I'll let Zach or someone else reply to your suggestions as I'm not really
in a position to change the way data is taken and added to the db, although I've
made some changes to the UI from you suggestions.

Comment 17

[Richard Herrera]

(In reply to comment #14)

Absolutely in agreement with Asa here. Standards are not about table-less, but
about accessible code. I haven't checked all the code, but the tables used here
fit their purpose.

However, it's also not about validation either. A sample from Leroy's excellent
mock-up:

 <table border="0" width="100%">
 <tr>
  <td class="v1" style="width:211px;"> 
   <div class="t_desc">Product:</div><div class="t_value">Firefox</div>
  </td>
 </tr>
 </table>

While technically valid, is not ideal. A better format would be:

 <table>
 <tr>
  <th>Product:</th>
  <td>Firefox</td>
 </tr>
 </table>

Where table headers and table data tags are used properly instead of excessive
div-itis. Strip out the presentational markup and stick that in a CSS file as
well. Table summary and/or caption tags also would bump the accessibility
levels. Validation is only an error-check, not a solution.

Comment 18

[hhh]

Richard,

Absolutely correct, my intent was standards-aware (seperation of styles wherever
possible) and not standards fanatical;) and that the html should maybe be
decided on before styles are added. I'd like to see an eye kept as to what is
tabular data and what isn't.

Excellent point about table headers, I'll add that an h1 and an h2 might be nice
as well:)

Comment 19

[Richard Herrera]

Re: a test case listing that was mentioned earlier, I was envisioning something
along the lines of Apple's widget browser seen here:
http://www.apple.com/downloads/dashboard/

Thoughts?

Comment 20

[hhh]

Richard,

That Apple interface is easy to use, I like it.

Comment 21

[Roman Rudenko]

On standards:
The idea is that one shall use the elements that fit the purpose. Using tools 
for intended purpose makes source much easier to understand and maintain. 
Obviously, one could use tables or divs for everything, but that's like coding 
in Brainfuck.
Want a list of similar things? Use ul. Want a list of names and values? Use dl 
(definition list). Want a grid in which there is a relationship of some sort 
among the rows and columns? Use table.
Leroy's mockup uses divs for "Product, Platform, OS, branch", while dl would be 
much more appropriate here. Using divs means class on everything, using dl 
requires class or ID on dl itself, and allows us to omit classes for its 
children.
Compare the code:
<td class="v4"> 
   <div class="t_desc">Product:</div><div class="t_value">Firefox</div>
   <div class="t_desc">Platform:</div><div class="t_value">Mac</div>
   <div class="t_desc">Opsys:</div><div class="t_value">Mac OS 10.4</div>
   <div class="t_desc">Branch:</div><div class="t_value">Trunk</div>
  </td>
And
<dl class="v4">
   <dt>Product:</dt><dd>Firefox</dd>
   <dt>Platform:</dt><dd>Mac</dd>
   <dt>Operating System:</dt><dd>Mac OS 10.4</dd>
   <dt>Branch:</dt><dd>Trunk</dd>
</dl>

(Of course, both are much better than sticking every name/value pair in a 
separate table. Urgh.)

Comment 22

[hhh]

Attachment: 2nd recode of Zach's original attachment.

Here is one possible version. Some points:

1)nested tables removed
2)simple lists moved from table to list form
3)headline structure added
4)classes/id's to be added still (just a few hooks to start)
5)a few styles added for uniformity/ clarity

Tell me what you think Tracy, Zach, Leroy, Roman, Asa, and of course Richard!

Comment 23

[Roman Rudenko]

Hass:
<dt><strong>Platform:</strong></dt> -> dt { font-weight: bold; }
Obvious. If you are putting the same code everywhere, check if it is redundant.

body { font: 100%/1.2em Helvetica, Arial, sans-serif; } -> font: 100%/1.2 
Helvetica, Arial, sans-serif;
Very important difference here. 1.2em will be a computed height value. So, if 
someone comes along and uses 200% font size on some element, the line height 
used will be 1.2 of *base* font size, and not 1.2 of current font size. For 
correct behaviour, use 1.2 without a unit. This will force line height to be 
recomputed for every font size.

Remember to zero margins and paddings on all list elements (ul, li, etc). 
Different browsers have different defaults, and zeroing them all before setting 
something is a good idea if you like consistency.

Finally, the "Steps to Perform:	Expected Results:	Actual Results:	Notes/
Comments (optional):" area is *definitely* not a table :). YOu will be better 
off with a div wrapper for every major area, and it's title being a header 
inside it.

Comment 24

[hhh]

Comment on attachment 191085 [details]
2nd recode of Zach's original attachment.

><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
><HTML><HEAD><TITLE>Litmus - Run Tests</TITLE>
>
>
> 
><META content="text/html;charset=iso-8859-1" http-equiv="Content-Type"/>
><SCRIPT type="text/JavaScript" language="JavaScript">
>// don't ask why this is here
>function MM_findObj(n) { 
>  var x = document.getElementById(n);
>  return x;
>}
>
>var editedtests = new Array();
>var fields = ["summary", "steps", "results", "testgroup", "subgroup", "communityenabled_row"];
>
>function showEdit(testid) { 
>    for (var i=0; i&lt;fields.length; i++) {
>        show(getField(fields[i]+"_edit",testid));
>        hide(getField(fields[i]+"_text",testid));
>    }
>    
>    hide(getField("editlink", testid));
>    show(getField("canceleditlink", testid));
>    
>    MM_findObj("Submit").value = "Submit Results and Edits";
>    editedtests.push(testid);
>}
>
>function findEdited() {
>    MM_findObj("editingTestcases").value = editedtests.toString();
>}
>
>function cancelEdit(testid) {
>    for (var i=0; i&lt;fields.length; i++) {
>        hide(getField(fields[i]+"_edit",testid));
>        show(getField(fields[i]+"_text",testid));
>    }
>    
>    show(getField("editlink", testid));
>    hide(getField("canceleditlink", testid));
>    
>    // remove testid from the editedtests array:
>    var newarray = new Array();
>    for (var i=0; i&lt;editedtests.length; i++) {
>        if (editedtests[i] != testid) {
>            newarray.push(testid);    
>        }
>    }
>    editedtests=newarray;
>    
>    if (! editedtests[0]) {
>        MM_findObj("Submit").value = "Submit Results";
>    }
>}
>
>// fields are in the format fieldname_testid
>function getField(fieldname, testid) {
>    return MM_findObj(fieldname+"_"+testid);    
>}
>
>function show(obj) {
>    if (obj) {
>        obj.style.display = "";
>    }
>}
>
>function hide(obj) {
>    if (obj) {
>        obj.style.display = "none";
>    }
>}
></SCRIPT> 
>
>    <STYLE type="text/css">
>
>body {
>font: 100%/1.2em Helvetica, Arial, sans-serif;
>margin: 0;
>padding: 0 10px;
>}
>
>h4 {
>background: #d7d7d7;
>font-size: 1em;
>margin: 0 0 .5em;
>}
>
>div {
>font-size: .85em;
>margin: 1em 0;
>padding-bottom: 10px;
>border-bottom: solid #c3c3c3 1px;
>}
>
>td {
>vertical-align: top;
>width: 25%;
>}
>
>ol {
>margin: 0;
>padding-left: 23px;
>}
>
>ul {
>margin: 0;
>padding-left: 23px;
>list-style: none;
>}
>
>li {
>padding-bottom: .5em;
>}
>
>input {
>margin: 0;
>}
>
>#summary {
>}
>
>.odd {
>background: #f5f5f5;
>}
>
>.even {
>background: #f6eff7;
>}
>
>.description {
>color: #008000;
>font-weight: bold;
>}
>
>#submit_button {
>text-align: center;
>}
>
>     </STYLE></HEAD><BODY>
>
><DIV><IMG alt="" src="http://testrunner.mozilla.org/litmus/skins/mozilla-banner.gif"/></DIV>
>
><H1> <A href="index.cgi">Litmus</A> - 0.0.1</H1>
>
><H2>Run Tests</H2>
>
>    <DIV id="summary">
><H3>Testing Summary:</H3>
>
><DL>
> <DT><STRONG>Product:</STRONG></DT>
>   <DD>Firefox</DD>
> <DT><STRONG>Platform:</STRONG></DT>
>   <DD>Mac</DD>
> <DT><STRONG>Opsys:</STRONG></DT>
>   <DD>Mac OS 10.4</DD>
> <DT><STRONG>Branch:</STRONG>
>   </DT><DD>Trunk</DD>
> <DT><STRONG>Build ID:</STRONG>
>   </DT><DD>20050727</DD>
></DL>
>
><DL>
> <DT><STRONG>Test Group:</STRONG></DT>
>   <DD>Smoketests</DD>
> <DT><STRONG>Subgroup:</STRONG>
>   </DT><DD>Install and Startup</DD>
></DL>
>
><P><A href="run_tests.cgi">change configuration</A></P>
>    </DIV>
>
><FORM id="form" name="form" method="post" action="process_test.cgi">
><INPUT type="hidden" value="true" name="isTestResult"/>
><INPUT type="hidden" value="" id="editingTestcases" name="editingTestcases"/>
><INPUT type="hidden" value="1" name="product"/>
><INPUT type="hidden" value="1" name="testgroup"/>
>
>
><!-- Test 1-->
>
>    <DIV class="odd">
><H3>Test ID:1</H3>
>
><DL>
>  <DT><STRONG>Summary:</STRONG></DT>
>     <DD class="description">Start Application</DD>
>  <!-- dt><strong>Product:</strong></dt>
>
>     <dd>Firefox</dd>
>
>  <dt><strong>Group:</strong></dt>
>
>     <dd>Smoketests</dd>
>
>  <dt><strong>Subgroup:</strong></dt>
>
>     <dd>Install and Startup</dd -->
></DL>
>
><P><A href="show_test.cgi?id=1">show test</A></P>
>
><TABLE border="1">
> <TBODY><TR>
>  <TH>Steps to Perform:</TH>
>  <TH>Expected Results:</TH>
>  <TH>Actual Results:</TH>
>  <TH>Notes/Comments (optional):</TH>
> </TR>
> <TR>
>  <TD><OL><LI>Start the Mozilla Firefox browser.</LI>
>        </OL>
>  </TD>
>   <TD><OL><LI>The browser should start.</LI>
>         </OL>
>  </TD>
>  <TD><UL><LI><INPUT type="radio" value="1" name="testresult_1"/> Pass</LI>
>              <LI><INPUT type="radio" value="2" name="testresult_1"/> Fail</LI>
>              <LI><INPUT type="radio" value="3" name="testresult_1"/> Test unclear/broken</LI>
>         </UL>
>  </TD>
>  <TD><TEXTAREA rows="8" cols="30" name="testnote_1"/>
>  </TD>
> </TR>
></TBODY></TABLE>
>    </DIV>
>
>
>
><!-- Test 2-->
>
>    <DIV class="even">
><H3>Test ID: 2</H3>
>
><DL>
>  <DT><STRONG>Summary:</STRONG></DT>
>     <DD class="description">Application Installation</DD>
>  <!-- dt><strong>Product:</strong></dt>
>
>     <dd>Firefox</dd>
>
>  <dt><strong>Group:</strong></dt>
>
>     <dd>Smoketests</dd>
>
>  <dt><strong>Subgroup:</strong></dt>
>
>     <dd>Install and Startup</dd -->
></DL>
>
><P><A href="show_test.cgi?id=1">show test</A></P>
>
><TABLE border="1">
> <TBODY><TR>
>  <TH>Steps to Perform:</TH>
>  <TH>Expected Results:</TH>
>  <TH>Actual Results:</TH>
>  <TH>Notes/Comments (optional):</TH>
> </TR>
> <TR>
>  <TD><OL><LI>On Windows and Linux, Launch the Install wizard.</LI>
>              <LI>Navigate through the wizard with the Next buttons.</LI>
>              <LI>Accept the licensing agreement and perform a "Standard" installation.</LI>
>              <LI>When the installation has completed, ensure that the checkbox to launch the application is checked and click the "Finish" button.</LI>
>              <LI>On Mac, drag the Firefox package to your Application folder.</LI>
>        </OL>
>  </TD>
> <TD><OL><LI>The installer should be complete, all of its content should fit properly in the wizard panels, and the application should launch at completion of the installation.</LI>
>       </OL>
>  </TD>
>  <TD><UL><LI><INPUT type="radio" value="1" name="testresult_1"/> Pass</LI>
>             <LI><INPUT type="radio" value="2" name="testresult_1"/> Fail</LI>
>             <LI><INPUT type="radio" value="3" name="testresult_1"/> Test unclear/broken</LI>
>         </UL>
>  </TD>
>  <TD><TEXTAREA rows="8" cols="30" name="testnote_1"/>
>  </TD>
> </TR>
></TBODY></TABLE>
>    </DIV>
>
>
>
>
><!-- Test 3-->
>
>    <DIV class="odd">
><H3>Test ID: 3</H3>
>
><DL>
>  <DT><STRONG>Summary:</STRONG></DT>
>     <DD class="description"> Launch Migration Dialog</DD>
>  <!-- dt><strong>Product:</strong></dt>
>
>     <dd>Firefox</dd>
>
>  <dt><strong>Group:</strong></dt>
>
>     <dd>Smoketests</dd>
>
>  <dt><strong>Subgroup:</strong></dt>
>
>     <dd>Install and Startup</dd -->
></DL>
>
><P><A href="show_test.cgi?id=1">show test</A></P>
>
><TABLE border="1">
> <TBODY><TR>
>  <TH>Steps to Perform:</TH>
>  <TH>Expected Results:</TH>
>  <TH>Actual Results:</TH>
>  <TH>Notes/Comments (optional):</TH>
> </TR>
> <TR>
>  <TD><OL><LI>Before or after installation, delete the Firefox folder in the Application Data directory (Application Support on Mac, .mozilla on Linux).</LI>
>              <LI>Launch Firefox.</LI>
>              <LI>At the Migration Dialog, select "Cancel" or "Do not import anything".</LI>
>        </OL>
>  </TD>
>   <TD><OL><LI>Upon initial launch, the Import (Migration) Wizard should appear.</LI>
>               <LI>The dialog should dismiss without any migration taking place.</LI>
>         </OL>
><P><STRONG>Note:</STRONG> Smoketests require a clean profile. Migration itself will be tested after the rest of smoketesting has been completed.</P>
>  </TD>
>  <TD><UL><LI><INPUT type="radio" value="1" name="testresult_1"/> Pass</LI>
>              <LI><INPUT type="radio" value="2" name="testresult_1"/> Fail</LI>
>              <LI><INPUT type="radio" value="3" name="testresult_1"/> Test unclear/broken</LI>
>         </UL>
>  </TD>
>  <TD><TEXTAREA rows="8" cols="30" name="testnote_1"/>
>  </TD>
> </TR>
></TBODY></TABLE>
>    </DIV>
>
>
>
>
><!-- Test 4-->
>
><DIV class="even">
><H3>Test ID: 4</H3>
><DL>
>  <DT><STRONG>Summary:</STRONG></DT>
>     <DD class="description">Talkback</DD>
>  <!-- dt><strong>Product:</strong></dt>
>
>     <dd>Firefox</dd>
>
>  <dt><strong>Group:</strong></dt>
>
>     <dd>Smoketests</dd>
>
>  <dt><strong>Subgroup:</strong></dt>
>
>     <dd>Install and Startup</dd -->
></DL>
><P><A href="show_test.cgi?id=1">show test</A></P>
>
><TABLE border="1">
> <TBODY><TR>
>  <TH>Steps to Perform:</TH>
>  <TH>Expected Results:</TH>
>  <TH>Actual Results:</TH>
>  <TH>Notes/Comments (optional):</TH>
> </TR>
> <TR>
>  <TD>
><H4>Versions 1.0.x</H4>
><!-- filtered (?) -->
><OL><LI>On Windows you'll need to do a custom install to select talkback.</LI>
>      <LI>Make sure that Talkback is installed. Look under components/talkback/ for the appropriate files.</LI>
>      <LI>If time permits, ensure that talkback comes up when crashing Firefox.</LI>
></OL>
><H4>Versions 1.1.x</H4>
><OL><LI>On Windows you'll need to do a custom install to select talkback.</LI>
>      <LI>Ensure Talkback is present in Extensions Manager. Also check that pertinent talkback files are present in *application*\ extensions\ [e-mail]\ components.</LI>
></OL>
>  </TD>
>  <TD><OL><LI>Talkback and its associated files should be present.</LI>
>        </OL>
>  </TD>
>  <TD><UL><LI><INPUT type="radio" value="1" name="testresult_1"/> Pass</LI>
>              <LI><INPUT type="radio" value="2" name="testresult_1"/> Fail</LI>
>              <LI><INPUT type="radio" value="3" name="testresult_1"/> Test unclear/broken</LI>
>         </UL>
>  </TD>
>  <TD><TEXTAREA rows="8" cols="30" name="testnote_1"/>
>  </TD>
> </TR>
></TBODY></TABLE>
></DIV>
>
>
>
><DIV id="submit_button"><INPUT type="submit" onclick="findEdited()" value="Submit Results" name="Submit" id="Submit"/></DIV>
>
></FORM>
>
></BODY></HTML>

Comment 25

[hhh]

Attachment: recode

Sorry for all that code, that was a wrong edit button pushed :(
Roman, thanks for the quick and excellent feedback. Good eye, I had meant to
strip all those strong tags. You are of course right about the line-height (I
never set it in the body element, that'll larn me). Margins on lists are at
zero, I added that padding or the ol's will lose there numbers, checked in IE
and Opera on PC. About you're last point, here comes the debate! I'm not using
the th tag right, those cells describing what's in the head column or row? If
not, give me an outline of your div wrap w/ header suggestion.

Comment 26

[Roman Rudenko]

On Apple's UI: It is different from mine by wrapping stuff in scrollbars... I'll 
consider this idea.
Right now, MY challenge with test group selection is "how to integrate number of 
tests and quality of coverage information in a more graceful manner", and I am 
considering the following: Installation (7), with green-and-red background whose 
split matches the percentage rate. Reason why it is better: In multicolumn 
display, supplementary data eating its own column might bring user to a column 
overload. Number of entries in brackets is a frequently used primitive. Red and 
green background will provide easy visual cue on progress. See following 
attachment for mockup.

Hass:
I was talking about zeroing margins on li. Good habits come first :). I've seen 
a lot of "why does this gap appear here" problems caused by not zeroing default 
values.
Using table is bad there, since you do not want all that stuff side by side - 
too little space for every individual element. Consider something like
<div id="steps">
<h3>Steps to perform</h3>
<ol>
  <li>Feed the cat</li>
  <li>Wash the dishes</li>
</ol>
</div>
<div id="expected">
<p>Candy</p>
</div>
and so on...
And then position this stuff with CSS to your heart's content.

Comment 27

[Roman Rudenko]

Attachment: Mockup of proposed presentation of a single testcase group

Comment 28

[Patrick Quinn-Graham [:pqg]]

Attachment: Individual test recode (no tables)

I'm not convinced the 4 elements of a test represent tabular data, so here is
my thoughts on a way of representing it.

Comment 29

[hhh]

Attachment: horizontal layout

Comment 30

[hhh]

Attachment: Adapting Patrick's html

Comment 31

[hhh]

Attachment: Lists, yay

Comment 32

[Roman Rudenko]

Hass:
First of all, I would like to note that your code is a hair away from validating 
as HTML 4 Strict. All you'll have to change to make it work is wrapping inputs 
from <form action="process_test.cgi" method="post" name="form" id="form"> in 
something like <p> or <div> (as inputs are inline elements, and form expects 
only blocky children). Giving your page HTML 4 Strict doctype is a good idea, 
since that's what your code is - a semantic document with no legacy table layout 
stuff.

Also, please look up <label> in HTML specification. That's what you should use 
next to radio inputs - as that way, clicking on label will switch to input that 
label links to. Please also look in into using JS to zebra-colour your tests. 
That's what JS is for - assorted beautification of page.
http://www.alistapart.com/articles/zebratables/ should provide a nudge in right 
direction.

All:
Next is the topic of MSIE support. Microsoft is currently moving ahead with IE7. 
Does it make sense to uglify the code to support IE6? I think no - we should 
work on making it good in browsers that are good to us. If IE7 ends up the same 
problematic beast AND someone wants to support IE, then they should not have 
much trouble with hacking good clean code to work around IE bugs. If IE7 ends up 
sane, we saved effort from working on dead end browser. So, for now, I say 
"screw IE". Anyone else with me?

If we drop IE, we can use all the kinds of nice features that were "forgotten" 
by IE team. We can rely on DOM implementation being sane. We can use advanced 
CSS selectors and pseudoclasses/pseudoelements, such as >, +, [attr="value"], .
class1.class2, :first-child, :hover, :before, :after. We could no longer worry 
about all the strange bugs it has (and believe me, it has more than plenty). 
Finally, we could have a great "shining beacon" app that shows the people what 
exactly MSIE prevented us from doing.

Comment 33

[Zach Lipton [:zach]]

Hi folks. My appologies for being a little inactive over the past few days;
being on the road is not my favorite activity either. I'm really amazed and
impressed by  what's gotten done with this, and now want to see what we can do
to take it to the next level and implement it on the site. Thanks again to
everyone for your work on this. 

A few general comments before I launch into specifics: 

Roman--Sorry, but it's got to work in IE. I know there are a lot of great things
that we can do if we drop IE support, but it's important that this work with IE
6. If your design doesn't, we can of course work together to make whatever
changes are nessecery. 

More generally, I'd like to focus more on design at this point rather than
coding details. Margins on <li> tags and other nits are important and there's a
time and a place to worry about them, but I think it's best for us to narrow
down an overall design that we can all improve, rather than trying to polish
multiple designs at once. 

Lastly, don't worry about stuff like how many results there should be (pass,
fail, broken, broken with bug, etc...). Those are all dynamically generated from
the database. With all of these designs, keep in mind that new fields may be
added in the future, so there should be some flexibility. 

Roman - your "progress bar" for a testcase group
(https://bugzilla.mozilla.org/attachment.cgi?id=191097) is really really
awesome! If anyone has any sugguestions on how to generate something like that
dynamically (from perl), that would be great. I can dig into that when I have
the time, but it's going to be a long while. 

Ok with that out of the way... 

I think there are two things that need to be done here. The first is to put
something together that is good and workable, and post that up on Litmus in the
next few days. The existing page is obviously hideous, and I'd like to have
something for the medium term. I think this can be something along the lines of
https://bugzilla.mozilla.org/attachment.cgi?id=191099. I will work on this in
the coming days, once I finish redoing the JavaScript select boxes. 

What I'd like to do here, going forward, is try to come up with something that's
a little more dynamic and innovative. I think
http://www.phatfool.com/mozilla/litmus_draft3/ is a step in the right direction,
but we should look at adopting some more of the features of Tracy's mockup. 

It would be awesome to either start with Leroy's page and make changes from
there, or build something entirely new. Since building something like this
obviously requires a fair amount of time, both in design and HTML, and in
JavaScript work, it's totally fine (one might even say prefered) to do
non-working mockups. Pencil and paper mockups are cool too. In any case, you can
just assume that any required backend work to support your design will be done;
there will likely be some sort of XML-RPC or SOAP interface to get testcase data. 

Thanks again for all your work on this. I'm really excited to see what we can
build. 

Comment 34

[Leroy]

Hi Zach, good to hear from you. 

I'm keen to get this moving but I think I知 missing something here. You
mentioned .. 

"What I'd like to do here, going forward, is try to come up with something
that's a little more dynamic and innovative. I think
http://www.phatfool.com/mozilla/litmus_draft3/ is a step in the right direction,
but we should look at adopting some more of the features of Tracy's mockup."

I take it you're referring to 'Tracy Walker's rough multitest mockup'
(https://bugzilla.mozilla.org/attachment.cgi?id=190986), and I知 interested to
know exactly what functionality/features are missing from 'draft3'? 

I think it would be helpful, especially for me, to have a list of the dynamic
functionality or features you would like to see, for example ..

Ability to:

 - view one test case at a time, and navigate test cases without reloading the page
 - view all test cases at one time, without reloading the page
 - see a list of test cases in view of an existing test case
 - submit one or all test cases

.. so from there we can build a design that will take this functionality into
consideration.

anyway, just my POV :)

Comment 35

[Tracy Walker [:tracy]]

Leroy, all,

I agree with zach on Phatfools draft3. It's close to what we're after in general. 

First some comments about the mockup:
 I think the real estate under "Testing Summary" should be reduced. The
information there is nice to have but not necessary during test case running.
That doesn't mean I'd like to see it nuked.  Spread it out horizontally and even
make that information altogether hideable with a Show/Hide toggle button. 

 I like the testcase presentation area.  I like the Previous | Next | Show All
location and functionality. I think the space for Summary, Steps and Expected
results is structured and used well. One nit is that the Bugs fields should also
be available for Failed test cases.

What we're looking for in dynamic display.
On testcase result submission the following should happen:
-The Next test case in the list is displayed.
-The Summary List (small blocks on the left) automatically scrolls to "focus"
that same test case in a "current" bracket/frame of some sort with a wider border.
-A colored background denoting the state of the result of the testcase just
submitted would appear in the summary for that testcase. They certainly
shouldn't be primary colors. Faded red, green, yellow, orange; whatever we
decide to represent each state of the test case would be good.  What I mean by
state is not run, passed, failed...etc.

The list of summaries should be scrollable (probably not truly scrollable, but
rather moveable with the Next | Previous links and/or Up | Down buttons). All of
the tescase summaries should have the color coded state.  

  

Comment 36

[Tracy Walker [:tracy]]

A bit more on the "summary sidebar":

Let's say we create a little more verticle space from reducing Testing Summary.
That gives us room for maybe 5 summaries in the sidebar.  Make the 3rd, center,
summary the "current frame".  Clicking on Show in any testcase summary would
adjust the list so that that testcase is in the "current frame" and the full
testcase associated with it is displayedin the main area.  Under the five
summary sidebar scenario, it would be nice to have "Next 5" and "Previous 5"
buttons/links in addition to Next and Previous.  This way a tester could scan
through the testcase list without setting focus to a testcase until they
selected "Show".  Hmmmm, it just occured a Goto Current button/link would be
good too. That is, say we've scrolled 25 test cases from the current one with
"Next 5", but then we want the list to go back to show that current testcase in
the summary sidebar.  Without a Goto Current button/link we have to use Previous
5 five times.  But I can also see a Goto Current button as being confusing.

just some thoughts that occcured while staring at the draft3 mockup.

Comment 37

[Roman Rudenko]

All (esp. Zach):
First of all, let me demonstrate the new version of my group/subgroup switcher. 
I implemented colour-coded progress bars in items, and changed list mechanics 
from "on click" to "on hover" (feel free to flame me on this one).
http://24.80.239.218:5190/litmus01/run_tests.html
Width of red bar can be easily controlled from HTML. Just look at those spans in 
source, their width is desired width of red bar.
(Yes, I know that we are concentrating on tests page, but Great Master Zach 
expressed interest in coloured progress bars, so here I go.)

Tracy, Leroy:
I do not understand the reason for trying to avoid page reloads, forcing user to 
employ YOUR controls instead of their browser controls. The following usually 
happens:
* User clicks on your JS pseudolink.
* User wants to go back to where they were. So, instead of using your special 
navigation, they do what they do on all other sites: press Back in browser, or 
do a Back mouse gesture.
* User swears profoundly, since they are taken to a page they did not want to go 
to, and JS-maintained state of original page is now lost.
This sucks. Don't reinvent the back/forward buttons, make the browser ones 
useful instead.

Same goes for "reinventing" scrolling. A developer says "let us kill the ugly 
scrollbar, and instead make user rely on our JS-driven previous/next buttons. So 
they do, and end up with **** implementation that does not allow user to 
do many useful things, such as scrolling with mouse wheel or searching through 
entire list with search-as-you-type. End result: a lynched developer.

Want to show a lot of test descriptions? Use a box with scrollbar. Want to make 
item-by-item navigation comfortable for user? Make your navigation links just 
plain links. Let them keep their browser controls.

Give me back button or give me death! :)

Comment 38

[Leroy]

This isn’t exactly complete as I haven't had much time to work on this tonight
but I've added some extra features in response to the feedback and comments.

http://www.phatfool.com/mozilla/litmus_draft4/

as mentioned it's not complete as I wanted to add the list feature, showing 5
summaries with the ability of moving to the previous or next 5, but I ran out of
time tonight. If I get some time tomorrow I’ll work it and add another comment here.

Comment 39

[Roman Rudenko]

Comment on attachment 191097 [details]
Mockup of proposed presentation of a single testcase group

Obsoleted mockup, as working version was created.

Comment 40

[hhh]

Attachment: Summary Squash (IE-text-resizable)

Squashing the summary. I think this is solid, but can you all check whether or
not the resizable text is legible in your browser(s)and whether the floats or
negative margin mess with anything?

Comment 41

[Chris Cooper [:coop] (he/him)]

> Squashing the summary. I think this is solid, but can you all check whether or
> not the resizable text is legible in your browser(s)and whether the floats or
> negative margin mess with anything?

OK, now we're getting somewhere. I like the idea of being able to hide what is
essentially useless information (Testing summary info), once you've verified
it's correct at the start, of course.

(In reply to comment #38)
> This isn’t exactly complete as I haven't had much time to work on this tonight
> but I've added some extra features in response to the feedback and comments.
> 
> http://www.phatfool.com/mozilla/litmus_draft4/

Looking good, although I also enjoy zebra lists (thanks for the link, Roman) for
navigating this type of data. I think we might be able to go with lists of some
kind if we were able to get rid of some of the clutter at the top of the page.
Do we really need the mozilla header _and_ a big, fat "Run Tests" title? Could
we not combine them somehow?

(In reply to comment #37)
> Tracy, Leroy:
> I do not understand the reason for trying to avoid page reloads, forcing user to 
> employ YOUR controls instead of their browser controls. The following usually 
> happens:
[deletia]

I agree that we shouldn't be getting too fancy here. If we want this to work
from within a browser, then let's use the browser's existing functionality. We
may as well write our own XUL app for Litmus if we're going to start disabling
and re-implementing features.

Hass' Test Summary squash will already buy us back some screen real estate, and
if we can compress the banner/title, we would have to worry less about scrolling
and such anyway. I presonally don't want to have to scroll if I can avoid it.

Comment 42

[Chris Cooper [:coop] (he/him)]

(In reply to comment #37)
> All (esp. Zach):
> First of all, let me demonstrate the new version of my group/subgroup switcher. 
> I implemented colour-coded progress bars in items, and changed list mechanics 
> from "on click" to "on hover" (feel free to flame me on this one).
> http://24.80.239.218:5190/litmus01/run_tests.html

Does anyone else have trouble using these lists? I think they need a background
color set or something, because the lists are expanding out over existing text
and I can't tell/read what any of the options are.

UA: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6

Comment 43

[Tracy Walker [:tracy]]

Trying to keep an eye on on this from the big picture.

I had been considering this display from a "Wow, I've got 250+ testcase to get
through here!" point of view.  I do not like scrolling.  And just thinking about
using the mousewheel on a bug of this length gives me carpul tunnel.  Testrunner
BFT's helped me discover "Autoscroll".  :)

I've been thinking about comments from Mike Beltzner (MoFo usability expert),
Roman and coop and agree that lists will be okay.  The "scrolling" I suggested
would be clunky and probably only partially useful to a few people in long test
runs. But part of what Mike has suggested in an email thread is that long test
lists are daunting to users.  Even 26 test cases under Smoketest | Functionality
is far too large.  With that in mind, I suggest that the grouping of testcases
is going to have to be further subgrouped into closely related tests of say no
more than 5-7 testcases. I believe this would work well from various angles of
approach in the overall design of Litmus.

First, by squashing(or eliminating) the information at the top of the page we'll
have enough space to display 5-7 summaries on the left and have plenty of room
for long testcase instructions.  Leave it to the user to decide how they want to
view the testrun; one testcase at a time and/or "view all" (subgroup) testcases
with zebra striped diplay.

Asa and I have envisioned expanding the BFT by an estimated 5 fold (1200+
testcases). We'll do that by introducing tests in areas with little or no
current coverage.  And by going a step deeper on many areas that already are
covered.  This would mean we'd have many tests of very closely related
functionality. These could be identified, sorted and made into nice little
handfuls of tests that shouldn't discourage most users willing to help out
beyond a single testcase. 

We might have to revist our testcase heirarchy. Also design an associated test
group selection method that won't be too daunting.

  

Comment 44

[Tracy Walker [:tracy]]

(In reply to comment #42)
> (In reply to comment #37)
> > All (esp. Zach):
> > First of all, let me demonstrate the new version of my group/subgroup switcher. 
> > I implemented colour-coded progress bars in items, and changed list mechanics 
> > from "on click" to "on hover" (feel free to flame me on this one).
> > http://24.80.239.218:5190/litmus01/run_tests.html
> 
> Does anyone else have trouble using these lists? I think they need a background
> color set or something, because the lists are expanding out over existing text
> and I can't tell/read what any of the options are.
> 
> UA: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050716
Firefox/1.0.6

It looks okay on Deer Park. If expanding out over the Run Tests and other page
text is okay.  The expansions have a progress driven background.  However, on
Mozilla 1.7.11 the isn't that background on the expanded items.

Comment 45

[Roman Rudenko]

Yes, my menu does not work right in Firefox 1.0.x. I would be grateful for any 
pointers to workarounds or relevant bugzilla bugs. Setting explicit width on ul 
(e.g. 15em) appears to fix it mostly, but this is a fairly ugly solution. I will 
work on getting it to function right in IE and FF 1.0.x, while you folks can 
give me suggestions based on how it looks and works in Deer Park or Opera.

I agree that test hierarchy should be thought over, and large piles of testcases 
recategorized. I don't think we should opt for dynamically collapsible header - 
this is confusing, and can avoided if we streamline data display and remove that 
header banner.

Comment 46

[Chris Cooper [:coop] (he/him)]

Thanks for all the suggestions and submissions so far. For those who haven't seen it yet, many of the suggestions have already been incorporated into the current version of Litmus:

http://litmus.mozilla.org/

At this point, I'm widening the scope of this bug to include all of Litmus instead of just the test run interface. I invite people to poke around and use the complete tool. We encourage you to submit feedback about areas of the UI that are confusing. Better still, if you have time to mock-up a potential solution, we are always looking for a better way to do things.

Idea for new interface:
* statistics: what types of test run statistics would users want to see, and how should they be displayed? (see: http://litmus.mozilla.org/stats.cgi)

Comment 47

[Chris Cooper [:coop] (he/him)]

We'll file targeted for specific UI issues from now on.

Comment 48

[Chris Cooper [:coop] (he/him)]

We'll file targeted bugs for specific UI issues from now on.