61561 - [RFE] Integrate simple search into bugzilla front page

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: General, enhancement, P3)

Description

[Andreas Franke (gone)]

On the main bugzilla page at http://bugzilla.mozilla.org , there is already a
text field where you can enter a bug number. This text field can be overloaded
to access a simple search functionality.

The original simple search page (by C. Begle) is currently located at
http://www.mozilla.org/quality/help/simplesearch.html (using the JS at
http://www.mozilla.org/quality/help/bugreport.js ). 
In its default configuration it searches for bugs where the summary field
contains all given words as substrings (roughly).

Another "Fast search" tool has been created by J. Ruderman (see
http://www.cs.hmc.edu/~jruderma/s/bugz.html ). 
Depending on the user input (nothing, a bug number, several comma-separated
bug-numbers, or other stuff), it performs different actions.

A combination of these tools can be used to offer additional search power on the
main bugzilla page while preserving the old show-bug functionality.

I'm going to attach the diffs against the current bugzilla page served to NN4.x.
The result can be viewed at
http://www.ags.uni-sb.de/~afranke/mozilla/bugzilla.html (it imports the script
http://www.ags.uni-sb.de/~afranke/mozilla/quicksearch.js ).

Comment 1

[Andreas Franke (gone)]

Attachment: diff -u against http://bugzilla.mozilla.org/index.html

Comment 2

[Jesse Ruderman]

A few things..

1. Copy the js file onto bugzilla itself so we don't have to worry about your 
server being down, or about security problems from loading scripts from other 
domains.  Also make the js source url relative so other bugzilla installations 
don't have to waste mozilla.org's bandwidth.

2. The bookmarklet and the form don't work the same way.  Also, the bookmarklet 
doesn't work on IE.  (I should be able to fix most of this stuff.)

3. Put the instructions for using the javascript link in the [Help] alert, or 
maybe in an onclick alert for the link (can you do that for javascript 
links?).  Right now they take up vertical space and push "log out of bugzilla" 
off a 800x600 screen.

4. Document that the search does each word as a substring of the original 
summary by giving an example.  It's a very useful feature, and I wouldn't have 
guessed that it worked that way by looking at cbegle's or your page.

5. We should try to get a "all words as substrings" option on the normal query 
page eventually.  The bookmarklet / tiny form hack is hard to combine with 
other query options such as "only critical bugs", and takes up lots of space on 
the front page (creates an extra get request!).

Comment 3

[Andreas Franke (gone)]

Attachment: current version of quicksearch.js

Comment 4

[Andreas Franke (gone)]

Attachment: bugzilla page, without instructions about "Mozilla bug" link

Comment 5

[Andreas Franke (gone)]

1. I have attached the current version of the js file. If I make the js source
url relative, then I can't test it anymore because there is a <base> element at
the beginning of the document that points to bugzilla.mozilla.org. I agree that
the final version should have a relative URL, or the script could even be
directly included. This can be changed at checkin time, I think.

2. The bookmarklet is different, that's true. I haven't touched it at all, and I
don't intend to change it in any way. I'll leave that to people who have the
necessary knowledge. The only reason why it showed up on my diffs is that I made
some changes to the text around it.

3. I have commented out the instructions for the "Mozilla bug" link in the
bugzilla front page attached, so we can see how it looks if this space is saved.
I haven't put the instructions elsewhere yet. I'm not sure what the best
solution would be. Anyway, the JS help function needs more work. Maybe the alert
could be replaced it with a web page that opens in a new window.

4. The search functionality needs to be documented, that's true. I have extended
it in several ways, so the documentation part will be non-trivial. However, I
want to get the functionality itself right first. Some current examples:
    
        table image overlap       --> "Images sometimes overlap in table

New features:

        -foo		summary must _not_ contain substr.
	+foo|bar|baz	summary must contain any words (does not match "fool")
        :foo,bar        product or component must contain one of the substrings
        @foo,bar        owner's email must contain one of the substrings
	[foo            summary or status whiteboard must contain "[foo"
        !foo,bar        _all_ keywords must be present (should it be _any_??)

The default case is foo,bar,baz . A bug qualifies if it qualifies for at least
one of them. A bug qualifies for "foo" in the following cases:
	- if foo is a platform (all,dec,hp,mac{intosh},pc,sgi,sun),
          a bug qualifies only if its rep_platform is the specified one.
          None of the other criteria will be applied. Not sure if this is the 
          right thing to do.
        - if foo is a priority range (e.g. p2-4) or something like p1,p2,p3 
          or p1,2,3 , the bug qualifies only if its priority is ok.
          None of the other criteria will be applied. I believe this is
ok.         
        - if foo is a severity (blo,cri,maj,nor,min,tri,enh), then a bug 
          qualifies only if the severity is ok.
	- if foo is not such a special case, then a bug qualifies if one of the
          following conditions hold:
		* foo is at least 3 chars long (oh, just found a bug), and the
                  bug's product or component contains foo as a substring
                * foo is a keyword, and the bug has it
                * summary or status whiteboard contain foo as a substring
                  (at least if the length of foo is greater than 3)
                * foo is a substring of the bug's URL value, and foo contains
                  "http" or the the input consists of only one word.

Submitting this now to avoid dataloss in case my browser crashes... :)

Comment 6

[Andreas Franke (gone)]

> 5. We should try to get a "all words as substrings" option on the normal 
> query page eventually.  

That's a good plan. But it does not affect this bug. :)

> The bookmarklet / tiny form hack is hard to combine with other query options 
> such as "only critical bugs", 

Most frequently used query combinations should be possible now. We could use
other "escape characters" besides "-+:@[!,|", e.g. "`~#$%^&*()_=;'\/?>.<" This
should be sufficient to express all we want, we just have to agree on the
semantics of each of these keys.

> and takes up lots of space on the front page (creates an extra get request!).

If you really want it, you can drop the Mozilla bug bookmarklet to the [Help]
page, then everything fits on one line. I can't see that the additional search
functionality takes too much space. Nearly every portal site finds a place for
one or more search facilities, why should this be impossible for bugzilla.

The extra HTTP request can be avoided by inlining the script in the page, if
that's a real problem.

Comment 7

[Jesse Ruderman]

- give a way for users to search for "maj" in the summary field if they want 
to.  google uses + for that kind of thing 
- don't check for "http" in the url field.. some users forget to put it, and i 
don't see why https sites shouldn't be found
- doesn't always searching the status whiteboard, etc. slow the search down?  i 
like the summary-or-keyword thing, though, because some crash and meta bugs 
don't have the keyword set.

Comment 8

[Jesse Ruderman]

Btw, this suddenly seems like a lot of javascript for the front page.  It's 
also hard to remember the syntax for those query options, and which ones 
are "or" (vs "and").

The reason I said it was hard to combine the "words as summary substrings" 
search with other options was that the query page doesn't let you do that kind 
of summary search, not that the small form doesn't let you do other things.  
That doesn't mean it's a bad idea to try to make a search form that can be used 
from one textbox, but I think it's important to keep the simple search 
intuitive.

Comment 9

[Andreas Franke (gone)]

Attachment: diff -u for quicksearch.js to make + and - symmetric

Comment 10

[Andreas Franke (gone)]

- I have made + and - symmetric (see above patch). Their meaning is now:
  	+foo	summary must contain foo as substring
	-foo	summary must not contain foo as substring
  Searching for the _word_ foo is now possible with #foo .

- By looking for "http" the current version finds https:// sites, too. You can 
  verify that by typing "security https://www.verisign.com". It probably won't 
  find ftp:// URLs, though, but that could be changed if I had an exhaustive 
  list of schemes.
  The assumption that led to the current system was that it shouldn't be 
  necessary to combine the search for bugs where the URL value matches a given
  fragment with other searches, so the URL field is only queried if you enter a 
  single word. The number of bugs returned should be sufficiently small.
  I would expect the search to contain too much noise if the URL field is always 
  matched, but that can be changed if you or others disagree. I admit that the 
  current system is somehow inconsistent. 
  Probably there should be an "escape" key to indicate that _only_ the URL
field   should be searched for a given string foo. What about the dot?
.apple.com?
  Other suggestions?

- I am not sure about the effects on performance. However, I have not    
  experienced any slowdown so far, to my surprise. No matter which queries I 
  try, they are always completed within a few seconds.

- "seems like a lot of javascript for the front page". Well, it's a lot of    
  javascript, yes. But what's the difference if this is on the front page or 
  somewhere else? The lookup-by-bugID functionality works without javascript,
  and the only practical difference I see between the front page and any other 
  page is that Gerv is not allowed to make any changes, so updates would be a 
  little harder. Dawn, what do you think about this?

- "... hard to remember the syntax for `and' and `or'": IMO, we have no choice
  but to use whitespace for conjunctions. If a user types three words and we
  show her all bugs where the summary contains _any_ of the words, this would
  be much too confusing. Users just expect typing more words to narrow down the 
  queries, we can't change that.
  The other question is if we should offer an "or" operator. I think this is
  valuable functionality. How is this done in other search engines? 
  One approach would be to use "," for this purpose. It is easy to type, easy to 
  read, and the meaning of queries like "blo,cri,maj p1,p2,p3 lock icon" is 
  obvious. 
  An alternative would be to use "|" as the "or" operator. This may be more
  intuitive, but it's also a little harder to type. 
  One could also implement "a | b" as "or" (with spaces around "|"), and I like
  the idea of supporting "a OR b", but what should be the semantics of
  "a | b c" or "a OR b c d" then? If we only allow "," or "|" _within a word_,
  then we don't have any problems.

- I agree that the we need to make the default behaviour as intuitive as 
  possible. I doubt that it is counter-intuitive to assign a special meaning to
  the "-" character when used at the beginning of a word. Similarly, I don't 
  think that using other special characters as escapes is counter-intuitive.
  Probably the "foo bar baz" notation should be supported to mean search for
  exactly this substring, for people who want to search for special character  

Comment 11

[Andreas Franke (gone)]

Dawn, please review both the general ideas and this concrete suggestion to make
Bugzilla more accessible. Thanks.

General ideas:
0. We should allow for having multiple Bugzilla query forms. No query form can
be the best one for all possible users in all circumstances.
1. We should create a page that links to all known, working Bugzilla query tools
for b.m.o (there are several already, inofficial/unsupported ones could be
mentioned in a separate section)
2. This page should be linked to from both http://bugzilla.mozilla.org/ and
http://bugzilla.mozilla.org/query.cgi

This concrete suggestion:
3. Integrate a SimpleSearch tool for bugzilla into the Bugzilla front page.

This could be http://www.mozilla.org/quality/help/simplesearch.html or something
similar, like the one proposed here.

The current versions are newer than the ones attached, but the general principle
should be clear from the attachments that are already present. The newest should
be always online, and they seem to be stable:
http://www.ags.uni-sb.de/~afranke/mozilla/quicksearch.js
http://www.ags.uni-sb.de/~afranke/mozilla/quicksearch.html

CCing mpt based on the rumour that he designed the bugzilla front page and
because of his well-known usability reasoning abilities.

Comment 12

[Matthew T (active 1999-2002)]

Yes, I admit it! The rumors are true. *sob*

A bunch of separate query pages has the potential to cause more confusion than it 
would solve. Ideally, the default query page should be simple, and should be 
easily expandable by clicking `More' buttons to add more criteria to the form. 
The complexity of the page could be recorded using a cookie, or (even better) in 
Bugzilla's preferences for a logged-in user.

Dawn, how hard would it be to rearrange the items in query.cgi? It could be made 
considerably simpler, and we could remove the need to scroll to get to the 
`Submit query' button all the time.

Someone should file the `all words as substrings' and `any words as substrings' 
bug.

Comment 13

[Andreas Franke (gone)]

I have filed the "all [any] words as substrings" RFE in bug 65190.

> A bunch of separate query pages has the potential to cause more confusion 
> than it would solve. 

This is probably true for the ordinary Bugzilla user. Thus I take back my
"general idea no. 3". Instead, I propose to create such a "collection of
unofficial bugzilla query resources" page for use by "bugzilla wizards" only,
and link to it only from http://quality/help/ (idea no. 3a).

> Ideally, the default query page should be simple, and should be 
> easily expandable by clicking `More' buttons to add more criteria to the form. 
> The complexity of the page could be recorded using a cookie, or (even better)
> in Bugzilla's preferences for a logged-in user.

Completely agreed. But remember:
- this is not implemented yet, and given the number of other Bugzilla RFEs and
the current progress speed, it won't be implemented soon
- bug 16775, and bug 41654 may be better places to discuss this (in the former,
some short-term changes to query.cgi are currently under review)

Maybe it was wrong to put this bug in the context of "multiple query forms".
Okay, so forget that context if you don't like it. What I'm really asking for
here is offering *two* query forms for everyday use: A simple, text-oriented one
where you can type some words, press enter, and it just works, and the regular
graphical one (query.cgi) that we all know very well. You can improve the
graphical one (based on select boxes, checkboxes, radio buttons, etc.) a lot,
but that will never make it an optimal solution for keyboard freaks who prefer
typing in things like on a command line.
Are you still saying that having *two* alternative query forms (a simple one and
an advanced one) will cause confusion, even if the simple one is integrated into
the front page, and the advanced one link deeper? The only response on the
newsgroups about this was a comment from Steve Wix:
> Sounds like you are asking a usability question about simple search and
> advanced search.  I recently looked at a bunch of web sites and found that
> most had a simple search and advanced search page.
E.g. google has both a simple search and an advanced search page, much the same
way I'm proposing here. Is google causing confusion? I don't believe that.

This bug could be seen as a first step to solve bug 33067:
> I want something like Altavista. By this I mean
> I want a search that covers all fields at once,
> and that lets me include or exclude as many terms
> as needed.
My search covers keywords, product, component, and status whiteboard by default.
The ability to search comments could be re-enabled by simply re-adding the
radiobutton from the original SimpleSearch. (I removed it because Gerv said it
was causing performance problems.) Inclusion just means adding more terms,
separated by space. Exclusion is done by prepending the term with a - (minus).
(Though I admit that exclusion currently only works for summaries.)

The basic reasoning is:
- some people want a SimpleSearch
- currently it's "hidden" on the QA help page
- it's most useful for beginners
- beginners don't look at the QA help page
==> make it more visible, at least as visible as query.cgi
==> integrate it into the front page

Another supporting sign (in my view) is that the bookmarklet (which has a very
similar goal) is already is located on the front page.

Please excuse my emotions here... :)

Comment 14

[Andreas Franke (gone)]

Oops, I meant "I take back my general idea no. 2", and propose linking to it
solely from http://www.mozilla.org/query/help would be 2a.

Of course, I still support no. 3. After all, it's what this bug is about.

Comment 15

[Dawn Endico]

>Dawn, how hard would it be to rearrange the items in query.cgi? It could be
>made considerably simpler, and we could remove the need to scroll to get to
>the 'Submit query' button all the time.

in general changing the ui in bugzilla is pretty painful. However the
query page is also painful so if you came up with a good redesign it
probably would be worth switching. Even if it took a long time to get
around to the implementing the change, the prototype could be checked
into the web site and used. whenever new compents, operating systems etc,
get added the page would have to be updated by hand. check out redhat's
bugzilla. their ui is pretty nice. http://bugzilla.redhat.com/bugzilla/

Comment 16

[Karl Ove Hufthammer]

Is there a separate bug for a quicksearch sidebar search plugin?

Comment 17

[Andreas Franke (gone)]

Dawn, mpt, please take the discussion about improving query.cgi to bug 16775 and
friends. As long as query.cgi consists of more than a single text field, this
bug is about something different. Please give me a reason for not integrating a
simplesearch tool in the bugzilla front page. If you think it's causing
confusion, then what about google.com? I would be a thousand times happier if I
could point people to b.m.o in my posts to the newsgroups (instead of my
quicksearch.html).

> Is there a separate bug for a quicksearch sidebar search plugin?
A search for "bugzilla sidebar" gives me bug 37546 and bug 37339, but I'm not
sure if that's what you are looking for. A sidebar search plugin for _this_
query tool doesn't exist yet, I'm sure. Feel free to file a bug (and implement
it :), and post the bug number here.
Another thing would be to update the existing bookmarklet on b.m.o to provide
the functionality of this quicksearch tool. But this would really be a lot of
javascript for a bookmarklet. If this could be implemented as a server-side CGI
script that takes the input of the text field and does-the-right-thing, this
would make it easier, I guess. It would then fully work even with JS turned off
in the browser, at the expense of flexibility (you would have to change the
server, or setup your own, everytime you want to test a little change).

Comment 18

[Matthew T (active 1999-2002)]

Yes, I have no problem with separate simple search and advanced search pages. It 
was just the prospect of having a smorgasbord of options for simple search, fast 
search, quick search, advanced search, super-duper ultra-advanced search, etc etc 
on the front page which disturbed me.

Comment 19

[Andreas Franke (gone)]

Great! :) So Dawn, what do you say? Do you agree that this makes sense? If so,
I'd like to have a list of requirements that would make a simplesearch tool
acceptable for the bugzilla front page. In other words, I'd like to have a
review, please...

Comment 20

[Jesse Ruderman]

Several things I'd still like to see added to the simple search 
<http://www.ags.uni-sb.de/~afranke/mozilla/quicksearch.html>:

1. Ability to search for a phrase ("foo bar").
2. Ability to search any field (summary:"foo bar" reporter:[e-mail])
3. Ability to search for unresolved, unresolved/dup, and all resolutions 
quickly.

Making the bulk of the code server-side would probably make searching slightly 
faster (summary+keywords+component could be cached as a single string), and 
would finally give us readable query strings (bug 15809), in addition to making 
the simple search work for people who have turned off JS.

Comment 21

[Andreas Franke (gone)]

> 3. Ability to search for unresolved, unresolved/dup, and all resolutions 

This is fixed. You must use ALL CAPS, and it must be the first thing in your
query. Here are some groups of (hopefully) equivalent examples:

1. Search for all open (i.e. unresolved) bugs. (This is the default.)
"foo"
"UNCONFIRMED,NEW,ASSIGNED,REOPENED foo"
"NEW,ASSI,REOP,UNCO foo"

2. Search for all bugs, regardless of status or resolution.
"ALL foo"
"UNCO,NEW,ASSI,REOP,RESO,VERI,CLOS foo"
"+RESO,VERI,CLOS foo"
"+FIXED,INVALID,WONTFIX,LATER,REMIND,DUPLICATE,WORKSFORME,MOVED foo"

3. Search for open bugs and duplicates (unresolved/dup):
"+D foo"
"+DUPLICATE foo"
"U,N,A,REOP,DUPL foo"

The default and "ALL" are special cases; the general case is a comma-separated
list of prefixes (that may start with a + to indicate additions to the default).
You can combine open status values and resolution values, but as soon as you
mention a resolution value, all resolved statuses are automatically included:
E.g. "VERIFIED,FIXED" is the same as "FIXED", and in this implementation there
is no way to say "I want all bugs that are VERIFIED or FIXED".
To avoid accidental matches, I recommend using three- or four-letter prefixes
like "+DUP,FIX" or "+WORK".

Comment 22

[Andreas Franke (gone)]

Phrases like "lock icon" should work now, too. The implementation consists of
two simple steps:
1. In the input string, everything between "..." is escaped.
2. When outputting a chart value, this is reversed, and the quotes are dropped.
This means that foo"bar"baz is the same as foobarbaz. Especially quoting does
not have any influence on the comparison relationship, and the search is still
case-insensitive. If someone want's case-sensitive search, this has to be
indicated separately, or the implementation has to be changed.

Comment 23

[Andreas Franke (gone)]

Helpwanted. I won't be able to do the perl port to turn this into a server-side
CGI script. Any improvements to the existing client-side JS implementation are
welcome, too. (E.g. no. 2 "search any field" is still missing.)

Dawn, you haven't said anything about this idea yet. If you want me to make any
changes, now would be a good time to say so. If you want to resolve this as
WONTFIX, this woulbe be a good time, too. My priorities will be shifting away
from mozilla by the end of today :)

cc'ing some more Bugzilla people for independent opinions. Hope you don't mind.

Comment 24

[Andreas Franke (gone)]

Attachment: latest version of quicksearch.js (may have problems with IE)

Comment 25

[Gervase Markham [:gerv]]

One data point: Updating component changes by hand for the Bugzilla Helper is a 
right pain, and if you forget, horribly scary error messages about database 
corruption appear.

It would be good if we could avoid exacerbating that problem by having a second 
page which also needed updating.

Gerv

Comment 26

[Andreas Franke (gone)]

This search tool doesn't have any hard coded products or components. (The only
product or component specific things are the exception lists that avoid matching
all "Browser" bugs if you are searching for "row" for example. Forgetting to
update these exception lists wouldn't have serious consequences in most cases.)
However, keywords _are_ hardcoded here. For a client-side tool, there is no way
around this because you have to _know_ if something is a valid keyword _before_
submitting the query. Otherwise buglist.cgi spits out errors like "foo is not a
valid keyword. Please use your Browser's back button bla bla bla". To avoid
hardcoded keywords, this error message would have to be taken out of the cgi
scripts at least for the case where you do a substring search on the keywords
field. The rest of the hardcoded values are not so critical: status and
resolution values don't change too often, the same goes for platform and
priority. (I think platform support could be dropped since it is not too
useful.)

Though I wouldn't say that it's a critical issue in this case, I think avoiding
another page that needs updating is a good idea. There are two possible
solutions:
- Create a shared JS file that defines all hard-coded values. Then you only have
to change one file. You could even write a script that generates this file every
day from the database.
- Make all the JS tools server-side CGI scripts that pull the values from the
database every time. This would be an advantage for the users because
  * the tools would even work with JS turned off, and
  * the pages would load faster with slow connections
at the expense of flexibility for developers because
  * you have to set up a web server every time you want to test a little change.

I have neither the required perl knowledge nor the time to turn this tool into a
server-side cgi script, but if someone else did this I'd be willing to help. 
As far as the client-side JS tool is concerned: Any suggestions on how to avoid
hard-coded values are welcome. 

Comment 27

[Andreas Franke (gone)]

Attachment: another quicksearch.js (better for IE, supports field:value)

Comment 28

[Andreas Franke (gone)]

Can someone develop a specification what the functionality should be for b.m.o?
Currently, the algorithm is (roughly) as follows:

1. escape everything between "..."
2. split into words at spaces
3. use first word for status/resolution stuff if appropriate
4. for all remaining words:
   a) case split according to first char:
      "-": summary must not contain the word (substring)
      "+": summary must contain the word (substring)
      "#": summary must contain "anywords"
      ":": split at "," one of the parts must be 
           a substring of product or component
      "@": split at "," one of the parts must be
           a substring of the owner
      "[": word (including "[") must occur in summary 
           or status whiteboard (maybe not very useful)
      "!": keywords must contain "anywords"
   b) the default case (if none of the above applies):
      split at "," into parts, for each part apply
      the first applicable cases:
      - generic field:value notation
      - platform ("pc","hp","mac",...)
      - priority ("p1-2","p1,p2","p1,2")
        (not very useful anymore because of "--")
      - severity ("blo","cri","maj","nor","min","tri","enh")
      If none of these cases is applicable, 
      then add charts (ORed) for all of these:
      - product and component 
        (if length>2 and not blacklisted)
      - keywords (if it's a legal keyword)
      - summary and statuswhiteboard
        (if length>2 or it's not a legal keyword)
      - url (if exactly 1 word was entered)
   The sets of charts for each space-separated
   word are ANDed.

Things like "reporter:foo", "summary:bar" etc. should work now.

Comment 29

[Gervase Markham [:gerv]]

"Make all the JS tools server-side CGI scripts that pull the values from the
database every time"

That's the correct solution :-) Having the time to implement it, on the other
hand... I started with Bugzilla Helper, but ran out of time and effort.

On the other hand, I think we should still use this, even if it's not a CGI. If
it doesn't have hard-coded component values, that makes things better - although
the keywords thing is a bit of a pain.

We have two choices. We can try and persuade Dawn to integrate this into
bugzilla.mozilla.org, or we can do the same thing as the Bugzilla Helper, and
give it its own page on www.mozilla.org... I'd rather it was part of Bugzilla,
though.

Gerv

Comment 30

[Andreas Franke (gone)]

I would like to have both :) Seriously, if it's on http://bugzilla.mozilla.org/
it still needs a "[Help]" or "[?]" link for those who want to know what they're
doing. And since the search form is only one line in this case, the "[Help]"
page can easily host a second instance of it, besides some documentation. In
short, the help page could look a bit like the current quicksearch page
<http://www.ags.uni-sb.de/~afranke/mozilla/quicksearch.html>. It should probably
mention only query.cgi as an alternative query form to avoid confusion. And the
documentation can be much improved. The "Mozilla bug" bookmarklet could be moved
from the front page to this help page if there are space problems on the front
page.

This help page would be "its own page on www.mozilla.org", so we had both. The
JS file could be shared, and we would only have two versions of the form itself,
which is very few code which will hardly ever have to be updated (see the
proposal at http://bugzilla.mozilla.org/showattachment.cgi?attach_id=20035 
or the diff at http://bugzilla.mozilla.org/showattachment.cgi?attach_id=19927).

IMO having a textfield search tool on the front page is very desirable because
you could refer everyone directly to b.m.o, regardless of whether you are
pointing to bugzilla or to the search tool.

Comment 31

[Andreas Franke (gone)]

BTW, I can't see why having lookup-by-bug-number as the only method for
accessing bug reports on the front page is a good thing. Other bug tracking
systems offer more:
http://www.debian.org/Bugs/ (bug_id|reporter|owner|component; open/archived)
http://bugs.gnome.org/      (bug_id|component)
http://bugs.kde.org/        (bug_id|component|full_text<-[this is great!])
http://www.fvwm.org/cgi-bin/fvwm-bug (JitterBug; bug_id|bug_status|regexp)
http://cvs.xemacs.org/XEmacs-bugs    (JitterBug; same...)
http://www.delorie.com/djgpp/bugs/   (bug_id only, but they have only 6 open
bugs, accessible through an "all bugs" link)

Only then comes bugzilla in a google search for "bug tracking system"...

Comment 32

[Andreas Franke (gone)]

Attachment: quicksearchhack.html version 1 - QuickSearch for Hackers

Comment 33

[Andreas Franke (gone)]

Attachment: quicksearch.js - another version

Comment 34

[Andreas Franke (gone)]

Attachment: quicksearchhack.html version 1 with working link to JS src

Comment 35

[Andreas Franke (gone)]

Attachment: quicksearch.html - version 1 of an introductory page

Comment 36

[Andreas Franke (gone)]

So this is yet another version of QuickSearch. Changes in functionality:
- It is now possible to load the queries into the standard query.cgi form.
- The "generic" format now supports comma-separated alternatives on both sides:
      summary,longdesc:foo,bar    means that the summary OR the     
  long-description/comments must contain one of "foo" or "bar".
- votes>xx, votes=>xx, votes>=xx and votes:xx (same as =>) should work now,
  although not well-tested, and undocumented
- If an error is detected in the input, the form is not submitted any more.

The largest change however is the documentation:

- quicksearchhack.html is for people already familiar with query.cgi:
  * It assumes familiarity with the bug fields.
  * It is designed to be short and precise.
  * It describes the tool as a (case-insensitive) ``all words as substrings'' 
    search where (space-separated) words have the following form:
    "field_1,field_2,...,field_n:value_1,value_2,...,value_m"
    with a certain default prefix "summary,keywords,...:".

- quicksearch.html is for people who haven't seen any Bugzilla bugs yet.
  * It's more verbose, contains lots of examples.
  * It skips all advanced features, in particular:
    - the generic fields:values notation 
    - shortcuts/abbreviations like @owner and !keyword
  * It divides the information into three sections:
    1.) a very short readme-first section ("Getting Started")
    2.) a medium-sized how-to section ("Features") for those who want to know 
        what they are doing
    3.) an optional "Tips" section (probably the worst of the three)

I hope that this is enough controversial stuff to make some of you comment in
this bug. BTW, the patch&review keywords still apply.

Comment 37

[Andreas Franke (gone)]

I'd like to respond to some criticism I received on IRC some days ago:

1. The issue: "There shouldn't be a simplesearch tool on the bugzilla front page
   because people would use it to generate bug lists and then click the "New"
   link in the footer for filing their bugs, thus using enter.cgi instead of
   the Bugzilla helper."
   The response: This is obviously not a problem with the bugzilla front page. 
   The problem already exists if newbies are pointed to any non-default Bugzilla
   query page. IMO it is not a reasonable strategy to keep people away from 
   searching Bugzilla until they file their first bug. If this was the case, you
   could not even allow them to view a single bug, since the footer is the same.
   So I think this is a bug with the "New"->enter_bug.cgi link in the footer 
   which should be fixed independently of this one. 
   As soon as there is a bug number about it, feel free to mark this one 
   dependent on that one, then I'll vote for it and see how many years it takes 
   till that one gets fixed. Maybe I'll try to contribute a patch, maybe not.

2. The issue: "I don't think a simplesearch tool belongs on the front page."
   The response: 
   a) Other bug tracking systems have better query facilities than Bugzilla
on         their front page, as I noted in a previous comment. This is not 
      necessarily a strong point, because it could be that Bugzilla has the
      best-designed front page of all these systems.
   b) Why does it have the bookmarklet then? AFAICT the bookmarklet _is_ a 
      simplesearch tool, or at least it would be one if it wasn't broken.
      So what's the difference?
   c) One of Bugzilla's main functions, at least for me, is its search engine.
      If you see Bugzilla mainly as a search engine for bugs, then it seems
      natural to have a one-line simple search form on the front page, and an
      advanced query page one link deeper, like google, altavista, and probably
      many other search engines have. I do not think that google has any 
      usability problems. So the main question is: is it legitimate to say that
      Bugzilla _is_ a search engine, to an important extent? At least for
      some volunteer QA people, like me, this may well be the case. Especially
      for duplicate hunting the search becomes the dominating task.
      So the remaining question is: what would be the downside of doing this?
      Why would anybody object to this? Is it just "I think a long text field
      on the front page looks ugly"? Would it take too much space (one line)?
      Personally, I find it much more irritating that bugzilla uses an ant image 
      instead of a real bug.

Comment 38

[Andreas Franke (gone)]

And here's another one:

3. The issue: "The query language is completely new, and not a simple one."
   The response: I have tried to make clear in the introductory documentation
   that it's possible to explain the main points in two sentences and four short 
   paragraphs. And I'm sure there's still a lot of room for improvement.
   If you have a better, simpler query language, let me know.
   If this is the reason for rejecting this idea, then simply drop all the
   features and make it a simple "all words as substrings in summary" search,
   pretty much like C.Begle's simplesearch tool is. For the front page this
   would be sufficient, although I still don't understand why it would be
   any better than leaving the features in and just not advertising them.
   On the front page, there is no space to advertise them anyway, and if you
   compare it with all the explanatory documents related to query.cgi, I don't
   see anything too complicated here.

So, this is enough for today, I'm tired, and I apologize for any effects of this
on the tone of my comments. I hope this was the last time for me fighting
ghosts... :)

Andreas

Comment 39

[Asa Dotzler [:asa]]

I like this.  I haven't looked too deeply into it but I noticed that I got
unexpected results on my first query.  If you type "12345 test" it fails with
test is not a keyword.  Is this the intended result?  What caused it to think
test was a keyword or that I wanted to look at keywords?

Comment 40

[Andreas Franke (gone)]

> If you type "12345 test" it fails with test is not a keyword.  

Murphy's law. Somehow the "case" from the "testcase" keyword got lost when I
manually created the (hardcoded) list of keywords. I have fixed it in my local
version(s) at http://www.ags.uni-sb.de/~afranke/mozilla/quicksearch.html .

As mentioned in previous comments, the hardcoded keyword list is necessary for
an "intelligent" keyword handling (which finds bugs with the crash keyword _and_
bugs with "crash" in the summary when you search for "crash") because
buglist.cgi errors out (as observed) if it detects an non-existing keyword, even
if you're doing a substring search.

Comment 41

[Asa Dotzler [:asa]]

I like it :)  Without even learning the simple query semantics it seems to get
the bugs I'm after most of the time.  For exaple, I typed critical 0.8 and it
nailed the list of bugs which I've flagged (some in status whiteboard some in
comments).  
I'm wondering if it would be a good idea to search dupes by defualt and just
sort them to the bottom of the list.  

Comment 42

[Andreas Franke (gone)]

> search dupes by defualt and just sort them to the bottom of the list.

I think that wouldn't be hard, and it doesn't seem to affect performance much.
I have added the DUPLICATE resolution to the default configuration, and they
should be sorted to the bottom by default, so you can try this out. You can turn
this off by typing OPEN as first word.
But I'm not sure if this is what we want: There are often a lot of duplicates,
compared to the open ones (e.g. table border: 36 open bugs, 31 duplicates), and
only some of the duplicates are relevant: 
- some are duplicates of bugs that were resolved long ago(--> highly irrelevant)
- some are duplicates of open bugs that are already shown(--> no added value)
- some are duplicates of other open bugs                 (--> highly relevant)
- some are duplicates of recently fixed bugs             (--> relevant)
and it's tedious to distinguish between relevant and irrelevant ones.
Another disadvantage is that sorting the result list by clicking on one of the
column headers will mix open bugs and duplicates together.

I think something like this would be better:
1.) open bugs
2.) recently fixed bugs (resolution=FIXED and resolution changed after ...)
3.) duplicates of (duplicates of) both 1.) and 2.) 
4.) maybe also all recently filed bugs (bug creation after ...), regardless of 
    status or resolution
A possible criteria for "recent" would be the (branch or release) date of the
last milestone.

Also a duplicate should appear differently (e.g. greyed out) if it's a duplicate
of (a duplicate of) a bug that is already shown. (This should probably be filed
as a separate RFE).

As I understand it, Bug 24957 "include duplicates in search" suggests to only
show the "real" bug, not the duplicate, in the case where a duplicate matches.

The problem with this is obvious: Currently it seems to be impossible to search
for duplicates of (duplicates of) a subset of bugs (please correct me if I'm
wrong). And even if this would be possible, I'm sure it would be dog slow
(without doing massive changes to the database).

Is this analysis correct? If so, what's the plan? Sit'n wait, or attack it?

Comment 43

[Andreas Franke (gone)]

Attachment: localconfig.js for bugzilla.mozilla.org

Comment 44

[Andreas Franke (gone)]

For the latest (and final) version of quicksearch.js, please see
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=26161 or
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=26162
from bug 69793. The difference is that the former includes DUPLICATE bugs by
default while the latter only searches for open bugs.

Comment 45

[Andreas Franke (gone)]

Recent changes are:
- made negation work properly together with the generic field:value notation
  (now -summary,desc:foo,bar is the logical negation of summary:desc:foo,bar)
- allowed comma-separated alternatives for "+" 
  (now +foo,bar means summary:foo OR summary:bar instead of summary:"foo,bar")
- added automatic substitutions " AND "-->space, " OR "-->comma, " NOT "-->dash
  (now the syntax it pretty much compatible with Google, see
   http://www.google.com/help/basics.html#and )
- the auto-inclusion mechanism for the URL field in default searches has been
  improved: now it triggers for any word that contains an ip address, starts
  with something that looks like a host.name, or contains "://" , "localhost" or
  "mailto"
- a new feature has been added to make understanding and debugging easier:
  ShowURL() opens a new browser window with a pretty-printed description of the
  generated query url. A "preview URL" button has been added to
  http://www.ags.uni-sb.de/~afranke/mozilla/quicksearchhack.html

The documentation pages ( quicksearch{,hack}.html ) will have to be updated.
The simplest possible documentation is now the following:

  This tool is similar to the Google <http://www.google.com/> search engine
  (see here <http://www.google.com/help/basics.html> for an overview), with
  one important exception: while Google uses exact-word matching, QuickSearch
  uses substring matching for each word: the query `clip crash' will find bugs
  like "copying a hyperlink to the clipboard and pasting it into the URL field
  causes mozilla to crash", "Xlib-based Mozilla crashes at shutdown in 
  nsClipboard::Callback" and even "[ABS POS] Scrollbars are appearing when not 
  needed when clipped content is present" which does not contain "crash" in the
  summary, but has the "crash" keyword set. So to get best results, you should
  always use stems instead of full words in your query.
  
  Google's other important basic features are the same in QuickSearch:

  - Automatic "and" Queries:
    By default, QuickSearch only returns those bugs that include all of your 
    search terms. There is no need to include "and" between terms.
    For example, to search for bugs related to dragging a bookmark, simply type
    `drag bookmark'. To restrict a search further, just include more terms.

  - "OR" Searches 
    QuickSearch supports the logical "OR" operator. To find bugs with either 
    word A or word B in their summary, use an uppercase OR between terms.
    For example, to search for bugs related to dragging or dropping a bookmark,
    just type `drag OR drop bookmark'.

  - Capitalization doesn't matter 
    QuickSearch queries are not case sensitive. All letters, regardless of how
    you type them, will be understood as lower case. For example, searches for
    "table border", "Table Border", and "tAbLe bOrDeR" will all return the same
     results. 
  
  Note that for performance reasons the long description and additional 
  comments are not included in the search by default.

Comment 46

[Jesse Ruderman]

afranke: regarding your 2001-02-10 comment, I think searching open bugs and all 
duplicates is fine.  Duplicates of other bugs that are shown do add value by 
makign it easier to pick out the most frequently reported bugs (which are more 
likely to be the ones you're searching for).  Not including duplicates of fixed 
bugs might be useful, but I don't think it would be worth spending a lot of 
time on.

Comment 47

[timeless]

will you support ipv6 addresses? and mangled ipv4 addresses (127.1)

Comment 48

[Andreas Franke (gone)]

timeless: the code to recognize ip addresses is one line in quicksearch.js:
                        // URL field (for IP addrs, host.names, scheme://urls)
----------------------> if (w.match(/[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+/)
                           || ...
followed by several other heuristics. If ipv6 addresses still contain such a
pattern, they should trigger the mechanism. If you want any other patterns to be
supported, simply post the regexp you want to be added...

Comment 49

[timeless]

that won't even match 127.1 [an ipv4 addy] :-(, full ipv6 addresses look like 
HH:HH:HH:HH:HH:HH where H is a hex digit. there are contractions like :: just 
like ipv4. i don't know of a good regexp for it :-( no big deal, just a heads 
up

Comment 50

[Andreas Franke (gone)]

I have split off bug 70907 for making QuickSearch a server-side perl script.

Now that QuickSearch is part of the bugzilla CVS (see bug 69793), this is not
really a bugzilla issue any more. Moving to mozilla.org product.

Dawn, maybe you can do this when you upgrade to Bugzilla 2.12?

To fix this bug, the following things have to be done:

1. modify the localconfig.js file to suit mozilla.org's needs, e.g. use
   http://bugzilla.mozilla.org/showattachment.cgi?attach_id=26160

2. change index.html so that it integrates the tool. The original change is:

-<FORM METHOD=GET ACTION="show_bug.cgi">
-<INPUT TYPE=SUBMIT VALUE="Find"> bug # <INPUT NAME=id SIZE=6></FORM>
-<SCRIPT LANGUAGE="JavaScript">
-document.forms[0].id.focus()
-</SCRIPT>
+<p>
+<script language="JavaScript" src="localconfig.js"></script>
+<script language="JavaScript" src="quicksearch.js"></script>
+
+<form name="f" action="show_bug.cgi" method="get"
+      onsubmit="QuickSearch(); return false;"> 
+  Enter a bug # or some search terms:<br>
+  <input type="text" name="id">
+  <input type="submit" value="Show">
+  <a href="quicksearch.html">[Help]</a>
+</form>
+
+<script>
+<!--
+document.forms['f'].id.focus();
+//-->
+</script>

3. (optional) modify quicksearch.js include DUPLICATEs by default

319c319
<         // add("DUPLICATE",resolution);
---
>         add("DUPLICATE",resolution);

Comment 51

[Andreas Franke (gone)]

Attachment: patch for localconfig.js (upgrade mozilla.org keywords)

Comment 52

[Andreas Franke (gone)]

Attachment: same as above, but as diff -u

Comment 53

[Andreas Franke (gone)]

You can either update the localconfig.js file on bugzilla.mozilla.org and keep
the keyword list up to date, or disable the "auto-recognize keywords" feature 
entirely by putting an empty keywords array in localconfig.js (as it is the case
in the CVS version). If you do the latter, the documentation needs to be
updated.

Comment 54

[Gervase Markham [:gerv]]

afranke: this got done, right?

Gerv

Comment 55

[Andreas Franke (gone)]

Gerv: No.

Comment 56

[Gervase Markham [:gerv]]

Well, part 2 is certainly done on the front page of b.m.o...

Gerv

Comment 57

[Andreas Franke (gone)]

Attachment: updated patch to bugzilla.mozilla.org's localconfig.js

Comment 58

[Andreas Franke (gone)]

Attachment: same as above, but as diff -u

Comment 59

[Andreas Franke (gone)]

Well, yes, the main part of this bug has been fixed.
Since query-by-keyword works fine for popular keywords like "crash" and "meta",
feel free to resolve this bug. If you want to upgrade to the current keyword
list, you can apply the latest patch to b.m.o's localconfig.js .
Once bug 72837 ("Bugzilla should produce JS version of Components/Products
etc.") is fixed, updating the static list of keywords can be done automatically.

Comment 60

[Andreas Franke (gone)]

-> mozilla.org's Bugzilla component.

Comment 61

[Andreas Franke (gone)]

Marking fixed. The latest keywords-update patch to localconfig.js is attached to
bug 103885.