Closed Bug 736273 Opened 14 years ago Closed 14 years ago

running a test and marking result is confusing

Categories

(Mozilla QA Graveyard :: MozTrap, enhancement)

Product:
Mozilla QA Graveyard
Component:
MozTrap
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: camd, Unassigned)

Details

Feedback from OTS with Tony C and Kevin B: The pass button mixed with the status of the case isn't clear that you didn't already mark it passed, and too easy to re-start the test, when you just mean to expand to see the details. The mark fail button looks like a delete button and it's not clear how to mark a test failed. The invalid / unclear button isn't really clear it's a button and that it's a result option Our take-aways: Status indicator needs to be separate from any of the buttons to change the status. Restart test button should be near the status, once status is given.
A Pivotal Tracker story has been created for this Bug: http://www.pivotaltracker.com/story/show/26529975
Eric Meyer changed story state to started in Pivotal Tracker
Eric Meyer added a comment in Pivotal Tracker: ok, so I made an initial proposal for this on the runtests-cleanup branch (which should maybe get merged with filter-runtests, since they will overlap quite a bit). For now I'm still supporting the 'start' feature, but I like the idea of dropping it. I also kept pass/invalid buttons available in the item summaries. There's a trade off here between clarity and speed. Keeping them in the summary you can get to them quickly, but they are split off from the fail options. If we moved them into item details it might be more clear for first-time users, but it would slow down advanced users.
Cameron Dawson added a comment in Pivotal Tracker: This looks like a great step in the right direction. It feels to me like the status should be on the right, and the buttons should be on the left. And I think it might be clearer if the text on the "Pass" and "Invalid" buttons actually said "Mark Passed" "Mark Invalid" to be clear. I realize that's bulky text, however. It also seems like the status of "assigned" or "started" and the buttons, while a different color, look too similar. A better distinction between button and state would be great.
Cameron Dawson added a comment in Pivotal Tracker: I actually think I prefer to have the pass and invalid buttons in the details, not summary. This might be sufficient distinction between status and action button.
Cameron Dawson added a comment in Pivotal Tracker: Here's many user comments on the screen: * User assumed "pass" button was indicating already-passed state. * User could not find fail buttons. * User found the checkmark on the "invalidated" state odd. * User was initially confused by "Start" button, expected to see Pass/Fail options right away. * User was confused once they clicked Start, because the "Pass" action button looks too much like the case is already in Passed state. Also, no obvious Fail option (fail button is comparatively quite subtle, user thought it was a "delete step" button because they are a manager), which increased user's sense that somehow they'd already marked the test passed. * "invalid or unclear" link needs better tooltip, current tooltip just repeats the text of the link. * User generally had lots of trouble with the runtests screen, and almost all of it was due to having the "current state" and the "take action" combined into a single UI element. * User also kept clicking the start/pass/restart button accidentally when they just wanted to expand the test; need bigger target for expand and/or further from start/pass/restart button. * User didn't understand "restart" action from subtle circular arrow on icon; text "restart" needed to be visible somewhere. * Combination of "current state" and "next action" into one button is especially problematic given that hover state indicates "what will happen if you click", meaning the immediate feedback after click-and-reload is not the state you just put it into, but the next state that would happen if you click again. * wrong icon on invalidated status in runtests
Cameron Dawson added a comment in Pivotal Tracker: * User assumed "pass" button was indicating already-passed state. * User could not find fail buttons. * User found the checkmark on the "invalidated" state odd. * User was initially confused by "Start" button, expected to see Pass/Fail options right away. * User was confused once they clicked Start, because the "Pass" action button looks too much like the case is already in Passed state. Also, no obvious Fail option (fail button is comparatively quite subtle, user thought it was a "delete step" button because they are a manager), which increased user's sense that somehow they'd already marked the test passed. * "invalid or unclear" link needs better tooltip, current tooltip just repeats the text of the link. * User generally had lots of trouble with the runtests screen, and almost all of it was due to having the "current state" and the "take action" combined into a single UI element. * User also kept clicking the start/pass/restart button accidentally when they just wanted to expand the test; need bigger target for expand and/or further from start/pass/restart button. * User didn't understand "restart" action from subtle circular arrow on icon; text "restart" needed to be visible somewhere. * Combination of "current state" and "next action" into one button is especially problematic given that hover state indicates "what will happen if you click", meaning the immediate feedback after click-and-reload is not the state you just put it into, but the next state that would happen if you click again. * wrong icon on invalidated status in runtests
Cameron Dawson added a comment in Pivotal Tracker: more feedback: * user expected clicking a bug link to open a new tab, not in same window
Eric Meyer added a comment in Pivotal Tracker: Cam: Forcing a new window (there is no way to force a new tab, unless users have a plugin that opens new windows as tabs) is generally considered poor form, and the HTML (target="_blank") has been deprecated for that reason. I see how this might be one case in which it is helpful to some users, but I want to think through it more before we break best-practice on it.
Cameron Dawson added a comment in Pivotal Tracker: eric: oh ok. Yeah, let's not do anything that's bad form. Just surprising to me. When I read google news, and click on a news item, it always opens that item in a new tab. Seems like lots of sites do that. Perhaps Firefox is doing that for me. Yeah, Chrome does the same thing. But Safari opens a new window. Ok, well, do what you feel is right here. :)
Eric Meyer added a comment in Pivotal Tracker: I'm not writing it off entirely. For being against best-practice, everyone sure seems to do it. It's possible that FF and Chrome now default to tabs rather than windows. For now, I moved all the actions into the item details. I'd still favor getting rid of assigned/started and the "start" action. I would also consider moving "restart" back into the summary (far-right), but I don't have strong feelings on that. Cam: over to you for comment.
Cameron Dawson added a comment in Pivotal Tracker: Eric: this is looking lots better. I feel like if the user clicks anywhere in the summary bar, it should open or close the test details. And, yeah, getting rid of the start button is fine by me. Could just make the initial status of a test "Pending." I agree that the restart button should be on the summary bar. I think it would be great either on the far right, or even right next to the state of the test, in a non-prominent color. The button of the "mark invalid" button is truncated too small. I really like adding the comment text to the status line. That's an awesome idea.
Eric Meyer added a comment in Pivotal Tracker: Cam: The invalid text wasn't meant to be truncated, it was meant to be gone (with a tooltip). Fixed in my latest push. I've added tasks for Carl and Jonny to make a few adjustments.
Cameron Dawson added a comment in Pivotal Tracker: eric: I'm wondering about the text "Pass all" on the pass button. People might think it means "Pass all the test cases on this screen." I wonder if "Pass Test" might be better?
Eric Meyer added a comment in Pivotal Tracker: Cam: good call. fixed.
Carl Meyer added a comment in Pivotal Tracker: Ok, we've already got conflicts between this branch and filter-runtests. I think what we should do is merge filter-runtests into this branch, resolve the conflicts, then complete the remaining tasks on both stories on the runtests-cleanup branch. Eric, I tried to merge and resolve conflicts, but there's a conflict in a Sass file that I have no idea what to do with. The other conflicts are in templates, and I could resolve those but I think you can just as well, and there's no way to commit a partial resolution. So you'll have to do this merge (from the commandline, as GHfM doesn't do merges with conflicts right). Just "git co runtests-cleanup" then "git merge filter-runtests", then one by one edit the files it tells you that have conflicts and resolve the conflict, doing "git add <filename>" on each file after its resolved. When they're all resolved ("git status" shows no red files), commit the merge and push it.
Eric Meyer added a comment in Pivotal Tracker: ok, after more research I decided that bug-urls and the help link fit the acceptable use policy for target="_blank" (supporting/reference documents). It also looks like our target browsers support that now as a tab-opening rather than window-opening by default. So I added that in master.
Eric Meyer changed story state to finished in Pivotal Tracker
Eric Meyer changed story state to delivered in Pivotal Tracker
Eric Meyer changed story state to started in Pivotal Tracker
Eric Meyer changed story state to finished in Pivotal Tracker
Jonny Gerig Meyer changed story state to delivered in Pivotal Tracker
Cameron Dawson changed story state to accepted in Pivotal Tracker
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.