Closed Bug 15809 Opened 25 years ago Closed 16 years ago

buglist.cgi URL character length optimization

Categories

(Bugzilla :: Query/Bug List, enhancement, P2)

Product:
Bugzilla
Component:
Query/Bug List
Version:
2.13
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
Bugzilla 3.4

People

(Reporter: CodeMachine, Assigned: dottedmag)

References

(Blocks 1 open bug)

Details

Attachments

(3 files, 2 obsolete files)

javascript v1
5.26 KB, patch
Details | Diff | Splinter Review
combine multiple params, clean up the url and paste a short link in the results
4.76 KB, patch
Details | Diff | Splinter Review
Cleanup the search URL by redirect. v1
1.42 KB, patch
mkanat
: review+
Details | Diff | Splinter Review 
At the moment buglist.cgi takes huge URLs.  Quite by accident I've noticed that
you can remove part of this and they'll take defaults.

It would be really nice is the default buglist.cgi URLs that query.cgi generates
were minimised.  This would make it easier to determine the nature of the query
from the URL, and also to quote the URL in things like bug reports and emails.
Status: NEW → RESOLVED
Closed: 25 years ago
Resolution: --- → INVALID
I would love to do this.  I don't think it's possible, given the way HTML forms
work.  Let me know if I'm wrong!  But meanwhile, I am marking this bug INVALID.
You could use a minimisation intermediate URL with redirect I imagine, although
would slow things down.

Maybe you could include at the bottom -

The shortform of this URL is
<A...>http://bugzilla.mozilla.org/buglist.cgi?...</A>.  This would at least
allow obtaining the short form for quoting purposes if not easy bookmarking.

Or I could say I suspect this could be done with JS (?), but then you might have
to kill me.
Maybe allow a user to have a preference saying whether they want this feature
done with a redirect, slightly slowing things down?  I would turn it on.
I'd use this feature, too. Especially handy when you're trying to paste Bugzilla 
query URLs in e-mail/Usenet messages.

Reopening for reconsideration by Tara.  There are a number of possibilities that
might work:

(a) Javascript. (?)
(b) A redirect, or at least a preference to allow this for those who want it.
(c) A link on the bug list that does the minimisation .  At the very least it
could display the URL on the page, if not in the URL bar.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
Reassigning ...
Assignee: terry → tara
Status: REOPENED → NEW
QA Contact: matty
*** Bug 38489 has been marked as a duplicate of this bug. ***
Bug 38489 duplicates possibility #3.
See also bug 65391, "Give brief summary of bug list query".
Blocks: 65391
Target Milestone: --- → Bugzilla 3.2
One problem with this is that AIUI if you "reduce" a URL and share it, it could
give different results if the other person has a different default query.
IIRC, it only loads the default query if no other parameters are passed to 
query.cgi, but I could be wrong so that's certainly worth looking into.
Moving to new Bugzilla product ....
Assignee: tara → endico
Component: Bugzilla → Query/Bug List
Product: Webtools → Bugzilla
Version: other → 2.13
I'll attach a patch in a moment which provides a link containing a shortened
version of the URL for bookmarking/copying.


(And also, as an aside, query.cgi only seems to use the default query if there
are no parameters specified in the URL, as Jake suggested in an earlier comment.)
Keywords: patch, review
Target Milestone: Bugzilla 3.2 → Bugzilla 2.16
Review comments:

This function can be about 1/3 the length by:

a) using a prefix relationship on the params to be deleted.

@list = ('short_desc', 'long_desc', ...)

foreach @list
  if !defined($useful_params{$_})
    remove all members of $useful_params beginning with $_

Then, you can delete multiple entries from $useful_params using a list.

+        $url .= "?";

You can add that unconditionally because it'll get chopped if it's all there is.

+        my $aref = $M{$key};
+        if (defined($aref)) {
+            if (scalar(@$aref) > 0) {
+                foreach (@$aref) {
+                    $url .= "$key=" . url_quote($_) . "&";
+                }
+            }
+        }
+    }

I'm sure there's a better way to do this using map and join.

+    chop($url);
+
+    return $url;

Combine these.

Gerv




We are currently trying to wrap up Bugzilla 2.16.  We are now close enough to
release time that anything that wasn't already ranked at P1 isn't going to make
the cut.  Thus this is being retargetted at 2.18.  If you strongly disagree with
this retargetting, please comment, however, be aware that we only have about 2
weeks left to review and test anything at this point, and we intend to devote
this time to the remaining bugs that were designated as release blockers.
Target Milestone: Bugzilla 2.16 → Bugzilla 2.18
*** Bug 148570 has been marked as a duplicate of this bug. ***
Wow. I didn't realize there was such an old bug for this already (till myk cc'd
me on an email refering to this bug) Anyhow... my idea for fixing this was that
we could at least add some minimal javascript at the top of the query page that
looked at all the fields marked "dirty" (from catching onchange of all fields)
and just added those as the query fields in a constructed URL. It could "on
Submit", do a document URL replace for that constructed URL (which hopefully
would be a heck of a lot smaller than the default) Makes sense? 

FWIW an year back (or maybe more) I had written some test to see if there was a
difference in bugzilla's response time for long or unedited version versus
condensed versions. Not surprisingly the smaller queries did shave off some
seconds. So this bug may very well help improve bugzilla performance as well. 
Gagan: Personally, I'd like the full form data to be shown by default so you can
manually edit it. It would be nice to have a link or button at the top of the
page that says "Create linkable URL" or "Strip unused header fields".

I would be ok, I guess, with :-/ the reverse too, if need be, where we strip by
default and add the extra headers if they ask for it.

> Not surprisingly the smaller queries did shave off some
> seconds. So this bug may very well help improve bugzilla performance as well.

I'm not sure, but that seems to be a performance issue that can (Can't test that
because I don't have a bugzilla up atm) be solved without fixing this bug, and
IMHO should be if that is the case. A field such as:
&order=& should intuitively not slow down the CGI because it should do the same
thing as if it didn't exist.







> I'd like the full form data to be shown by default so you can
> manually edit it. 

Well... that's what the Bugzilla query page is for. You are saying you prefer
editing the URL by hand more than going to the page and editing the fields? How
many other people like to do that? 

> A field such as: &order=& should intuitively not slow down the CGI because it 
> should do the same thing as if it didn't exist.

But it does. I'm no expert at server stuff but my thinking is that it still has
to be parsed to be considered empty/no-op and that work over several fields and
over several queries probably starts to add up.
Attachment #51856 - Flags: review?
Comment on attachment 51856 [details] [diff] [review]
patch against 2.14 to provide a shortened URL to link/bookmark

Not ready for review -- needs updating to cvs tip
Attachment #51856 - Flags: review? → review-
Comment on attachment 51856 [details] [diff] [review]
patch against 2.14 to provide a shortened URL to link/bookmark

Not ready for review -- needs updating to cvs tip
*** Bug 196356 has been marked as a duplicate of this bug. ***
sirLurxalot wrote a bookmarklet that does this:
http://www.squarefree.com/bookmarklets/mozilla.html#shorten_bug_query
Assignee: endico → nobody
These bugs appear to be abandoned.  Retargeting to 2.20
Target Milestone: Bugzilla 2.18 → Bugzilla 2.20
Bugzilla 2.20 feature set is now frozen as of 15 Sept 2004.  Anything flagged
enhancement that hasn't already landed is being pushed out.  If this bug is
otherwise ready to land, we'll handle it on a case-by-case basis, please set the
blocking2.20 flag to '?' if you think it qualifies.
Target Milestone: Bugzilla 2.20 → Bugzilla 2.22
Blocks: 290313
Blocks: 290977
Attached patch javascript v1 — — Splinter Review 
here's a first cut of a javascript solution to this problem.

it's not ready for review as there's a lot of simliar tests that i'd like to
change to a lookup from an array/hash.

there's also a problem where if your query is optimised to nothing, editing the
query resets it back to the default.
Assignee: nobody → bugzilla
Attachment #51856 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Note that bug_status=NEW&bug_status=ASSIGNED can be shortened to
bug_status=NEW,ASSIGNED.

It would be nice to get this bug fixed, as it is a pain in the but to share query URLS over IRC or any 
other chat.
The trunk is now frozen to prepare Bugzilla 2.22. Enhancement bugs are retargetted to 2.24.
Target Milestone: Bugzilla 2.22 → Bugzilla 2.24
Blocks: 317243
I don't understand why you would use javascript or redirects. The problem (so as i understand) is that the url string is huge because you use the get methode and all parameters are in the url. I would suggest to use the post methode instead of the get methode or why isn't this an allowed solution for this bug
Simply using POST would break bookmarking, reloading, and sharing URLs of buglists.
QA Contact: mattyt-bugzilla → default-qa
another approach, which is probably better, is to use POST, but provide a "bookmarkable link" on the search results page that contains the optimised form.
Assignee: bugzilla → query-and-buglist
Status: ASSIGNED → NEW
We are freezing the code for 3.0 in two weeks and we don't expect this bug to be fixed on time.
Target Milestone: Bugzilla 3.0 → ---
As a first step how about at least adding a cleaned-up link somewhere for
the resulting returned page.
(In reply to comment #28)
> Note that bug_status=NEW&bug_status=ASSIGNED can be shortened to
> bug_status=NEW,ASSIGNED.

I guess this shortening would already solve the problem in most cases. This trick was the only way I found that allowed me to save a query with a lot of &target_milestone= segments (to find all open bugs for past milestones).
It would be nice if we kept records of queries run.  Possibly storing the query in a table via POST data and getting a short URL that references it is the best possible solution.
Summary: Query.cgi/Buglist.cgi URL optimisation → Query.cgi/Buglist.cgi URL optimization
Taking.
Assignee: query-and-buglist → eseyman
Status: NEW → ASSIGNED
Attachment #322174 - Flags: review?(mkanat)
Comment on attachment 322174 [details] [diff] [review]
trimming the query part of a buglist

Nooooo, don't make this spaghetti code inside of buglist.cgi.

Did you notice the already-existing Bugzilla::CGI::clean_search_url?

Also, this doesn't accomplish the intention of this bug, which is to trim the URL that is submitted to the *browser* by query.cgi.
Attachment #322174 - Flags: review?(mkanat) → review-
(In reply to comment #40) :
>
> Also, this doesn't accomplish the intention of this bug, which is to trim the
> URL that is submitted to the *browser* by query.cgi.

I'm seeing two different requests in this bug (and comments from other bugs describing this one).

1) Trim the URL upon submit
- Uncheck options that apply to other options that haven't been set (e.g. email*1 for email1)
- Uncheck selections that don't change the result (e.g. deselect the products if all of them have been selected).

2) Trim the URL further (with stuff like bug_id=1,2 instead of bug_id=1&bug_id=2) server-side (because it can't be done client-side, to the best of my knowledge but I'ld love to be proven wrong).

Is this one, the other or both ?


Either of those approaches would help in the overall goal of making Bugzilla search result URLs easier to share. But approach #1 -- stripping entire unused parameters -- probably would produce the greatest average URL shortening, since it's more common not to use a parameter than to enter multiple values for a parameter.
(In reply to comment #42)
> striping entire unused parameters -- probably would produce the greatest
> average URL shortening

I don't know that we can strip parameters.
We can definitely unset their value(s), though.

> it's more common not to use a parameter than to enter multiple values for a
> parameter.

I ran into this bug when a user wanted to search across the whole database,
selected all 200 products and hit an apache error (the URL was too long).
I'ld say selecting all products or all components must be a somewhat common thing to do.

I'm a bit stuck on this one.
I don't see any parameters whose value I would feel comfortable removing (the only one that comes close is "order").

This patch deselects menus if all the values inside have been selected and,
on the server side, generates a minimised url for which a link is given on
the results page.

I'ld appreciate comments.
Attachment #322174 - Attachment is obsolete: true
Priority: P3 → P2
Summary: Query.cgi/Buglist.cgi URL optimization → buglist.cgi URL character length optimization
it probably doesn't help this issue that some of the links, such as 'change columns', include both the &<fieldname>=value, as well as &field-1-0-0=<fieldname>&value-1-0-0=value.  Those links should ignore either the -1 boolean chart, or the declared values.
Last patch in this bug had no review flags, so here's implementation from (duplicate) bug 453707.

There is already cleanup_search_url method, so let's use it.
Attachment #336923 - Flags: review?(mkanat)
Comment on attachment 336923 [details] [diff] [review]
Cleanup the search URL by redirect. v1

Yeah, actually, this looks find to me. I have to think a tiny bit about two things, though:

1) Is POST really a good idea here?

2) Do we really want to do a temp redirect there? I don't want the POST page to appear in the browser's History--will it?
(In reply to comment #49)

> 1) Is POST really a good idea here?

The other idea is the hidden field "needscleanup".

> 2) Do we really want to do a temp redirect there? I don't want the POST page to
> appear in the browser's History--will it?

Tested in FF3 and Opera - it doesn't. However with GET and hidden field it may.
Comment on attachment 336923 [details] [diff] [review]
Cleanup the search URL by redirect. v1

Okay, this is great. :-) I tested it, and it really works nicely.

I also have another patch that makes clean_search_url even better, and I'll attach it to a separate bug.
Attachment #336923 - Flags: review?(mkanat) → review+
Assignee: eseyman → dottedmag
Flags: approval+
Target Milestone: --- → Bugzilla 3.4
Thanks, Mikhail! :-) It's funny, about a month before you posted this patch, I had that exact idea on how to fix this, though I hadn't thought of the POST. I think I said something about it in IRC, maybe, but I'm not sure I ever wrote it down anywhere.

Checking in buglist.cgi;
/cvsroot/mozilla/webtools/bugzilla/buglist.cgi,v  <--  buglist.cgi
new revision: 1.384; previous revision: 1.383
done
Checking in template/en/default/search/search-advanced.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/search/search-advanced.html.tmpl,v  <--  search-advanced.html.tmpl
new revision: 1.33; previous revision: 1.32
done
Status: ASSIGNED → RESOLVED
Closed: 25 years ago → 16 years ago
Resolution: --- → FIXED
Max, please reference that related bug number here for an improved clean_search_url.  I, like others, may want to backport these fixes to solve our existing problems.  :-)
This isn't exactly complete though.  The links provided on the pages (Change Columns, etc) still contain the negative-numbered boolean searches, which can double or more than double the URL length.  Is there any way to also improve these links?

http://server/colchange.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&chfieldto=Now&emailassigned_to1=1&emailassigned_to2=1&emailcc2=1&emailreporter2=1&field-1-0-0=product&field-1-1-0=bug_status&product=bugzilla&query_format=advanced&remaction=&type-1-0-0=anyexact&type-1-1-0=anyexact&value-1-0-0=bugzilla&value-1-1-0=NEW%2CASSIGNED%2CREOPENED&query_based_on=
Since I'm getting myself so involved...

Adding the following line to the canonicalise_query in Bugzilla/CGI.pm will remove the negative-numbered boolean charts from the links to solve the other URL length problem:

--- CGI.pm.old  2008-09-12 09:56:02.388007334 -0400
+++ CGI.pm.new  2008-09-12 09:55:47.710174516 -0400
@@ -120,6 +120,7 @@
     foreach my $key (sort($self->param())) {
         # Leave this key out if it's in the exclude list
         next if lsearch(\@exclude, $key) != -1;
+        next if ($key =~ /-\d-\d-\d/);
 
         my $esc_key = url_quote($key);
(In reply to comment #54)
> The links provided on the pages (Change Columns, etc) still contain 
> the negative-numbered boolean searches

  Those aren't negative-numbered, those are valid boolean charts. You're talking about some other bug entirely--please report it as a separate bug.
OH, and the other bug is bug 454892.
Added to the release notes for Bugzilla 3.4 in bug 494037.
Blocks: 524395
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: