151686 - sort buglist on the client rather than resubmit the query

Status: RESOLVED FIXED

(Bugzilla :: Query/Bug List, enhancement)

Description

[Asa Dotzler [:asa]]

A problem (as I see it):
Right now (I think) when I sort a buglist what I'm actually doing is submitting
a new query with an additional parameter for sort. There are two problems with
this. The first is that it isn't really sorting my list. Something that was on
my list might no longer be on the new list or something new might be added. This
can lead to serious confusion. The second problem is that it takes a long time
for many queries to return results. Sorting shouldn't take as long as querying. 

My proposal:  
Move sorting to the client, or provide an option for sorting on the client. This
could be done with some wiz-bang JavaScript or something.

Potential problems:
Clients which don't support JavaScript (or whatever scripting is used) wouldn't
be able to take advantage of this.

Comment 1

[Bradley Baetz (:bbaetz)]

We could xslt, I guess, and have fallbacks resubmit the stuff.

Note that you can order on columns you aren't displaying - not sure how we'd
want that to work.

Comment 2

[GavinS]

A possible temporary workaround for your first problem is to view any bug on the
list, and then use the 'show list' link. Doesn't this turn the buglist into a
list of bugids, so that subsequent sorting would not lose any bugs or add any
new bugs to the buglist?

Comment 3

[Bradley Baetz (:bbaetz)]

Yeah, we could do sorting that way, using the list of bugs which we have anyway.

Hmm....

Comment 4

[Jesse Ruderman]

You can almost do this with the sort table bookmarklet on
http://www.squarefree.com/bookmarklets/pagedata.html.  That javascript code
works in both IE and Mozilla.  The bookmarklet needs to be smarter about sorting
numerical data like the votes column, buglist needs use one table instead of one
table for each 100 bugs (bug 103523), and buglist needs to use <thead> around
the header row(s) that contain <th>s.

Another possibility is to use xul outliner for buglist when the browser supports
it, like at http://bugzilla.mozilla.org/duplicates.xul.  Outliner gives you
sorting, column resizing, column picking, and in the future will give you title
tips for cropped data.  I don't know if outliner lets you have links or style
certain links as :visited, though.

Comment 5

[Matthew Tuck [:CodeMachine]]

This would be difficult if not impossible to do properly in the presence of
sortkeys.

Comment 6

[Bradley Baetz (:bbaetz)]

Not if the sortkeys were part of the data, somewhere.

Comment 7

[Matthew Tuck [:CodeMachine]]

Yes, well obviously, but that's a lot of data to be sent.  But come to think of
it, if we can get external JS cached, we might be able to share the sortkey data
with the query page and it wouldn't be so bad.

Comment 8

[Antonis Christofides]

Attachment: Simple demonstration of re-ordering the bug list with JavaScript

I played around and experimented a bit. The attachment demonstrates how
we can delete all rows from the bug tables and put them in a JavaScript
variable (called rowList), and how we can put them back into the tables
in a different order. I tested it in Mozilla and Konqueror 3.1.1.

So what remains is to sort the rowList. It's far from trivial, but it
mustn't be too hard either. The most difficult problem appears to be
that we don't always sort lexically, for example if priorities are High,
Medium, Low, sorting on priorities should put High first, then Medium,
then Low. It can be solved by somehow sending the sort order to the
client.

How to deal with browser compatibility: Initially the buglist sorting
links (i.e. table headings) will have the value they have currently,
that is, resubmit the query. After the page finishes loading, the onload
handler will check that the browser's JavaScript has all necessary
features; if it does, the sorting links will be changed to run
JavaScript functions instead.

How to sort on columns you aren't displaying: Unless I'm missing
something, this is not a problem. The client won't have to sort the list
from scratch, but only re-sort it. You can't re-sort on columns you
aren't displaying, can you? So if, for example, we sort on state, all
bugs that have the same state will appear in their initial order,
eventually retaining their original sort key as second sort key.

Comment 9

[Antonis Christofides]

Attachment: Improved demonstration: sort a buglist by summary

OK, here are some new results. First, I tested the previous demonstration
(attachment 120720 [details]) in IE 5.5, and it worked, but much slower than Mozilla
(which was considerably slower than Konqueror). The problem with IE is that it
didn't give any visual indication that it was processing; you'd think it wasn't
responding.

The new demonstration takes 5 seconds on Mozilla on a 866 MHz Celeron for a
list of 238 bugs. It doesn't run properly on Konqueror, but it takes about 5
seconds to do whatever it can (i.e. empty the buglist and not fill it in again
:-) It doesn't give any processing indication either.

There are thus two important problems:
  a) Too slow
  b) Lack of processing indication in some browsers

Now please don't tell anyone, but I'm a JavaScript newbie. I don't know if this
can be made faster.

I need some feedback at this point. I'll probably stop working on this until I
get some.

Comment 10

[Jesse Ruderman]

I wrote a Greasemonkey user script that does this:
http://www.squarefree.com/2005/04/18/bug-sort-user-script/.  myk filed bug
291397 for integrating it with Bugzilla. 

Comment 11

[Vlad Dascalu]

This is not a security bug - clearing the flags.

Comment 12

[Douglas Calvert]

Maybe this should block bug 325501 

Comment 14

[Max Kanat-Alexander]

See also the patch in bug 291397.

Comment 15

[Max Kanat-Alexander]

Whoever implements this needs to take into account the fact that our databases currently sort international text correctly, and so any client-side code will have to do the same.

Comment 16

[Michael Morgan [:morgamic]]

Didn't know about this thread, but I messed around with the tablesorter plugin for jquery, and it does a halfway decent job.  Messed around with it and in a few minutes I made this:
http://people.mozilla.org/~morgamic/bugzilla.html

Minus some adjustments, what do you guys think about something like this?

Comment 17

[Frédéric Buclin]

mkanat, I think as soon as the cookie thing is fixed (bug 425619), your code you wrote for bmo's buglist would be mature enough to be checked in upstream.

Comment 18

[Albert Ting]

I suggest taking advantage of YUI's datatable.

Datatable allows you to not only sort columns, but supports custom sorting capabilities, such as sorting by time, as well as display dates/times based on user-defined timezone.  In addition, you can do multi-tier column sorting, reorder columns via drag&drop, change column widths, pagination, etc.  All of this would be done on the client side.

Comment 19

[David Lawrence [:dkl]]

(In reply to Albert Ting from comment #18)
> I suggest taking advantage of YUI's datatable.
> 
> Datatable allows you to not only sort columns, but supports custom sorting
> capabilities, such as sorting by time, as well as display dates/times based
> on user-defined timezone.  In addition, you can do multi-tier column
> sorting, reorder columns via drag&drop, change column widths, pagination,
> etc.  All of this would be done on the client side.

Agreed that YUI would make an excellent choice for managing the buglist table and provide the features you listed. Once thing when I have done this in the past was that you need to take special care when you change the sorting, pagination, etc. you update history and URL so that that it is bookmarkable. 

YUI2 provides the DataSource.TYPE_HTMLTABLE which allows us to still render the current table for clients with javascript turned off, otherwise replacing the bug list table with the YUI datatable.

YUI3 has a way to do the same except using DataSchema.XML to parse the html table and then rerender as a datatable.

Another caveat is that you have to write custom column sorters for different columns such as severity, priority, status, etc. The default sort is alphanumeric but with those columns you want them sort as they display in the drop down list of a bug report based on their sortkey.

dkl

Comment 20

[Albert Ting]

Agree on the URL/history and custom field sort order issues.  But I don't think it matters too much, those can be added as needed.  Been using YUI Datatable (JSON-style) and never got such a request.  Main custom sort are the timestamps, especially change time, since it gets displayed in minutes vs. days.  Plus adjusting it for user's preferred timezone.

Comment 21

[David Lawrence [:dkl]]

(In reply to Albert Ting from comment #20)
> Agree on the URL/history and custom field sort order issues.  But I don't
> think it matters too much, those can be added as needed.  Been using YUI
> Datatable (JSON-style) and never got such a request.  Main custom sort are
> the timestamps, especially change time, since it gets displayed in minutes
> vs. days.  Plus adjusting it for user's preferred timezone.

I also have some experience in the past doing this so would love to work on a patch for this. But not sure how soon I would even be able to start on it. If you have the time, feel free to work up a patch and I can review it.

dkl

Comment 22

[Albert Ting]

Hi David, the only solution I have is JSON-based.  A bit off topic but I'm unclear why we can't enforce javascript.  There's a lot more that could be done, such as a menu bar.

Comment 23

[Frédéric Buclin]

(In reply to Albert Ting from comment #20)
> Agree on the URL/history and custom field sort order issues.  But I don't
> think it matters too much, those can be added as needed.  Been using YUI
> Datatable (JSON-style) and never got such a request.

It's important that 1) changes are persistent (bug 425619), and 2) the navigation links still work as expected when bugs are sorted differently. The current implementation on bmo keeps the original sort order in mind and all links to navigate across bugs are wrong, i.e. they don't match the new sort order. That's a real regression, making the navigation links useless once you sort bugs differently than the original order. This problem must be fixed before being accepted upstream.

Comment 25

[Kohei Yoshino [:kohei]]

This has already been solved in the current Bugzilla trunk = BMO so I’d mark this closed.