697308 - User experience notes for bughunter UI

Status: RESOLVED DUPLICATE of bug 661941

(Testing Graveyard :: Sisyphus, defect)

Description

[cmtalbert]

So, I tried to run through this like a dumb user that doesn't know anything.  Then I started applying my domain knowledge of what's happening and started poking deeper.  I wrote this pretty much as it happened, so you'll notice that I answer my own questions, but from reading the thread you can see how the flow I went through worked out and how it wasn't exactly optimal for a new user of the app.

First thing I noticed, it's pretty.  Very nicely put together, well polished.  I was impressed.

Issue 1:
First thing I did on the UI is click on one of the URL links.  And nothing happened, leading me to immediately think "the app is broken".  Then I noticed the little expando mark next to the url.  I clicked on it, and now I see three notes, none of which would I understand as a normal user.  It's not clear what view the click sent signals to, what those signals were.  The query names in the resulting drop down are pretty good but they could be better.  "Open in link in Fatal Message" -- what is the fatal message?  Is that the last thing on the stack when we crashed?  

Issue 2: Actually the second thing I did when clicking on a link didn't work was click on a signature.  And yeah, you guessed it, it didn't work.  Then I noticed the drop downs and started exploring those.

For these issues, I think we need to either override the default behavior of clicking on these links make them open the info box directly without needing the extra dropdown widget. I'm still not clear on what signals are sent to other views.  I click on the links and nothing happens in the other bughunter tabs.  What will I see when this is working?  Why is that the default action I want versus the "extra" drop down?

Issue 3:
Because I knew to go hunting for it, I found the little "extra connections" button way off to the right.  And I can see how the default click action really starts working to show me correlations between the crashes and the data.  Can we open a blank "view" shell on first run so that it when a user goes and clicks on something, they can see the extra view loading and then the app doesn't appear to be broken from the outset.

Issue 4: the help button is hilarious.

Issue 5: the refresh button on derived view connections returns with "no data available".  It was just displaying data so I don't believe there is *no* data available.  Perhaps it is a bug?  Or should it read, no *new* data available.

Issue 6: On the soccorro record view, there should be a link to the socorro record itself on crash-stats.mozilla.org, right?  Or do we have that data in bugzilla?  I think we can create it because all the signatures on crash-stats have a defined URL based on their signature, but I could be wrong about this. Bc can correct me here.

Issue 7: Given that filing a bug is the number 1 thing that we want the site to help people do, it seems odd that the button for that is hung off the view connections menu.  I'm not even sure why it's there, perhaps I missed something?  I'd expect that to be a prominent top level control.  But I'm not sure how I'd indicate to the system what I'm filing a bug *on*, so that it can then pre-populate the bug with interesting bits of data gleaned from here and set things (like the security checkbox, for instance) properly for me.

Issue 8: Now that I've had the site open a while, I've been going between my two views a bit, and I clicked on the sorting widget to sort by os, my top signature is a mac os x signature, with a particular dylib.  I click the signature link and the connected view below doesn't refresh.  It still shows a top level instance of a linux crash that seems unrelated.  I thought it was a bug until I realized that I was looking at *socorro records* in my lower view, and of course clicking on a signature generated from a reproduced crash wouldn't refresh my "socorro records" stupid user.  Although it took me three minutes to figure out that wasn't a bug, and I bet most users won't figure that out at all.  We need to indicate by some way when we have a connected view that doesn't update based on user selection.  A "disconnected" view :)

Issue 9:
I don't see any way to cut down the list of results or show them based on metadata like the os I'm interested in or the branch I'm working with.  That would seem like things I'd want to be able to do.

Issue 10:
The visualization drop down isn't working, but I just don't think that is implemented yet.  So no worries there.

Overall, this UI has great potential.  I can see it being very useful, but it (by necessity) is a complex UI and there are places that the user can paint themselves into corners.  I think we need to do a good job showing the user how not to do that, alerting them when they have, and highlighting the things that the app wants to help them with (like showcasing connections and the file a bug functionality) so that those are primary UI components.

We've made a great start.  Hopefully some of the ideas above can help move us in good directions for the next set of design decisions.

Comment 1

[Jonathan Eads ( :jeads )]

Issue 1: Totally agree, the default behavior for links is not finished.  I think the default page that loads should have two panels open by default and they should be configured to what bc thinks is the 80% use case.  So for example "Site Test Crash" with "Crash Details" or something like that.  This would prevent the user thinking the application is broken on first click.
   I don't really know what the correct default behavior is so to get started with the application I adapted SQL queries from a use case document bc created.  That's why the view names, column names, and default behavior is not intuitive at the moment.  It was intended to give us enough of a starting spot to make decisions on what data should be there.  The help ? icon should have detailed explanation for all columns in a view to help guide the user but there's no point in writing that until we agree on what the first set of views will be and I don't think we are there yet, I'm really depending on bc's feedback to make those decisions.

Issue 2:  Essentially Issue 1 but for the signature link.  In a previous patch I had conditional logic that detected when only one view was open and a user clicks on a link that's a signal the app would open the connections window.  But I think this is also unintuitive, I don't think the connections window will be easy to understand until the user see's two views working together.  I really think the user basically needs a self-induced demo and the only way to do that is configure the landing page to have two connected views so if any signal link is selected the application immediately responds.  Any signal sent to a view is loaded as a value in one of the input fields on the control panel.  This is not intuitive because the control panel is closed by default, we may need some place to just display signals sent.  This will be a bit tricky because the crash signatures and url's can be pretty large.

Issue 3: I think this is addressed in Issue 1-2

Issue 4: Thanks!  I have some big plans for this modal.  It will have a static portion that is the same across all views.  The static content will explain all of the dropdown menus and buttons.  It will also have a dynamic portion that will address the content displayed in the view so a user can figure out what things like fatal message mean.  Honestly, this would probably help me too, I have a set of cryptic notes that I made when I started to help understand the terminology used in the schema and I'm still a bit confused so I will need a lot of help from bc here.  Again, we should not address this until we agree on an initial set of views otherwise there will be a lot of wasted work.

Issue 5: This is a bug, it's caused by a character escaping issue when sending data through the control panel.  It's already on my list.  The character escape/utf8 encoding/decoding issues have been numerous and challenging.  I had a really tough time accurately transferring utf8 encoded urls from/to the database that had non english characters in them.  Django/jQuery/xhr all want to encode the data twice which is a bit of a show stopper.  I've got lots of gory details on that bug.

Issue 6: We have the data in bugzilla but there's a gotcha here.  The crash signature coming from socorro is not the same type of crash signature coming from other tables in bughunter.  So we will need to message the user about that somehow, definitely need some help from bc on that one.  This would be a possible place to link up to socorro, I'm thinking it should be one of the connection options in the "Open New Connection Window" though.  I need to see if socorro has any webservices we could plug into, even if we have to screen scrape something, I think it would be worth getting some socorro only views that would help tie the UI together for those users.

Issue 7: That button is a place holder.  It's identified as that in the functional spec.  It's hard to envision exactly where it should go without fully understanding an 80% use case for this application.  I agree it should be in a more prominent area but I think we need to nail down what data is displayed before making the decision.

Issue 8: I somewhat addressed this in issue 6.  I think one way to make this a bit more intuitive is to find an area in the view to display a list of signals the view sends/receives.  I'm not really sure where to put it maybe in the status message at the bottom of the view, I would like it to be static and always visible.  Maybe this would help the user make the association between link/signal and a view sending/receiving.  If i displayed them in the status bar I could have them flash or highlight when the corresponding signal is sent or received.  This definitely needs to be communicated directly.  Any ideas?

Issue 9: So we have a couple options here.  You can cut down the results by adding values to the input fields in the control panel.  Currently the values available are url, signature, fatal_message and date depending on the query structure.  You can also use the Search box which does filtering of the data set in the client only and that will take any meta data.  Keep in mind the data displayed is generated by some heavyweight queries with 4 way joins + nested selects so opening it up to a "WHERE constraint" for any column is going to be a bit tricky... but possible. Before going down that road I would need some consensus on that being a big win for the app.  I added the "In memory Search" to try to alleviate this problem.  Over the last two days I've been analyzing the queries and database, looking for performance issues and have found some low hanging fruit that might enable what I think you're asking for but I would like to get some more feedback from bc before going in this direction.

Issue 10: I really want to implement the visualization dropdown.  I think it would let the users do some interesting stuff with aggregate data but sadly... I have not managed to finish it yet and I think there are much bigger fish to fry with the app right now.

At this point, I'm doing a performance pass so that I can set up a paging system.  This is a required piece but I'm not really sure I can get it right without having a clear definition of what the first version views are.  I think the most critical next step is coming up with that list: what are the views and associated columns that really need to be there.  I don't think a lot of the issues you've raise can be effectively dealt with before that is done.  At the point at which I have the first version views defined: the default functionality can be cleaned up, the help can be written, user messaging can be improved, the copy displayed in the views can be made user friendly, and I can figure out where to put the file a bug button and maybe even make it work.  In its current state, the app is really just spitting out the results of SQL statements with way too much low level database laundry being aired but I think this was a necessary step to get the UI rolling so that I could get feedback from bc.  So any contributions regarding what views are useful and which ones are not? are we missing data that should be displayed?  Are the signals (url, signature, fatal_message) the right ones?  Should they be expanded to include all metadata?  Or can we just filter result sets in memory by meta data?  Is the workflow of cross view signaling really useful for identifying bugs?  or should we be doing something different? If I can get answers to these types of questions we can get ready for a polishing pass on the UI.

Comment 2

[Jonathan Eads ( :jeads )]

In Issue 6 I meant bughunter not bugzilla.  Whoops, guess I have bugzilla on the brain.

Comment 3

[Bob Clary [:bc] (inactive)]

I think we've handled most if not all of this in the sisyphus-webapp bug.