145588 - Search query formation requires expert a priori knowledge of reported bugs - should support fulltext indexes

Status: RESOLVED FIXED

(Bugzilla :: Query/Bug List, enhancement, P3)

Description

[Trevor Jenkins]

When searching the database for existing bug reports a user has to formulate
their searches such that all possible variants of their search terms are
included. If searching for errors with table presentation under Show All Tags or
for searching for "aminated gif" the user has to enter prefixes rather than
natural language terms. For example with "animated" the user is forced to
remember that this must be searched for as "animat" in case someone enter
"animation" instead of "animated". These are simple cases and have lead to
duplicate entries being made.

Bugzilla requires a thesaurus or controlled vocabulary feature provided by the
better text retrieval systems. So when the user enters "animated" a search is
made for "animated or animation or animate" depending upon what Use For/Related
 terms are entered for animated/animation.

Comment 1

[Matthew Tuck [:CodeMachine]]

Hmm, was just thinking of this tonight with regard to customised/customized, and
then there's all sorts of synonyms.

We obviously can't do a perfect job here (words with multiple meanings anyone?),
but we can possibly things a bit better.

There's no problem with converting words into the canonical form for their word
group, but we need what we search to be in canonical form too, so we either need
special MySQL support (and I doubt there is any in MySQL let alone all databases
we intend to support eventually), or maintain a copy of every comment and status
in canonicalised form, which is a large waste of space.

Then there's the fact this could theoretically bring up bugs where the reason
they matched is not obvious, and this will result in bug reports if it happens.
 Possible ways of dealing with this is through smaller groups (which could
result in a reduction of usefulness), requiring this feature to be turned on, or
highlighting the found words in the text like some search engines do.

If you have some implementation knowledge from other systems by all means I'd
like to hear it.

Comment 2

[Trevor Jenkins]

The specific problem is with the use of search terms. I don't believe that any
"canonical form" is either necessary or desirable. Because the user base is
drawn from a very wide range of experience and languages some form of
normalisation would be ueful. The convention way of doing this is by creating a
thesaurus. (See ANSI/NISO Z39.19 Monolingual Thesaurus Creation Standard at 
http://www.niso.org/standards/resources/Z39-19.html for specifics.) 

Bug reporters would continue to enter their reports in whatever fashion suits
them best. However, when someone (else) searches for these reports searches are
not made for terms directly but are routed via a thesaurus where synonyms (and
possible broader terms) can be added to the search criteria. Some users might
like to switch this feature off because it will be slower.

Rather than using mySQL (or any other system) a faster way would be to use an
inverted file system. Careful selection of techniques can provide an almost O(1)
search of the inverted list.

As to whether I have implementation knowledge ... well after 25 years in the
text retrieval market I know a thing or two.

Comment 3

[Matthew Tuck [:CodeMachine]]

What you say about adding search terms is true, but I didn't mention because I
was thinking it would only work for "any words" searches, but after thinking
about how MySQL does this it would just be ORs nested within ANDs for "all
words" searches.

I suppose we could put a copy of every comment on a file system or something but
that's a hefty amount of disk space and a radical departure from the way
Bugzilla currently works.

And I have no idea what an inverted file system is.  If you mean a full text
index, MySQL has this feature I think, although I don't think Bugzilla uses it,
and our comment searching is pretty slow because of it.

Unfortunately MySQL doesn't handle ORs very well in its optimiser (as evidenced
by past slow queries) so this sort of thing is probably potentially really slow
even if we do use full text indexes which will likely get prompted ignored in
the presence of ORs.

This really is the sort of thing you'd want to be on by default but I can't see
that happening yet.  Quicksearch on the front page already puts a strain on the
server apparently.  For this to be useful the word groups probably have to be
reasonable size and it feels like the search terms list would be about 4 times
bigger.

Comment 4

[Andreas Franke (gone)]

We should not rely on a full text index provided by MySQL if we have any plans
to support other databases. Full text indexing using porter stemmers or
something like this is not too hard, I have done that as an exercise for our
information retrieval lecture, though not in perl. A full text index only
increases the database space used by a factor of two, at maximum.

> Quicksearch on the front page already puts a strain on the server apparently. 

I would like to see some facts backing up this claim.

Comment 5

[Matthew Tuck [:CodeMachine]]

> I would like to see some facts backing up this claim.

Well I don't know, I thought I heard that, adding Myk.  Given there are no
indexes I can see, I'd be suprised if it wasn't true.

Comment 6

[Myk Melez [:myk] [@mykmelez]]

QuickSearch certainly contributes to the load, although its most harmful feature
has been removed.  I don't see why we shouldn't take advantage of MySQL's
full-text search feature.  Implementing multiple-database support shouldn't make
Bugzilla worse, after all.  We just need a way to disable the feature for
installations using databases that don't support it.

Comment 7

[Andreas Franke (gone)]

Fair enough. But do you agree that a cross-database solution is as good as a
mysql-specific one? Does anyone know how the mysql feature works and what it
offers? Trevor, do you know of any existing perl code that could help us here?
Or some free software in any other language that is similar enough to perl that
porting should be easy?

Comment 8

[Myk Melez [:myk] [@mykmelez]]

A cross-database solution is better than a MySQL-specific one, but that doesn't
exclude the possibility of implementing a MySQL-specific solution until a
cross-database one is available.

We shouldn't dumb down Bugzilla to the intersection of those features available
on all databases we (plan to) support, and that goes for other databases just as
much as for MySQL.

Comment 9

[Matthew Tuck [:CodeMachine]]

Actually I'm pretty sure FT indexes only search for whole words, whereas we use
INSTRs, so you might need a whole new syntax (and UI) to use them.

Comment 10

[Myk Melez [:myk] [@mykmelez]]

I think full-text searches probably would require new UI, although it's possible
they could replace the full-word searches we currently have except that if we
implement this using the MySQL 3.x feature, then we need to let people know the
results are sorted by relevance in that case and may not contain all the words
they searched on.  Searching on all words and ordering by something other than
relevance only exists in MySQL 4.0.

Comment 11

[Trevor Jenkins]

Free-text/full-text search system can provide truncation. Many provide
right-hand truncation so that one can search for prefixes. The really good ones
provide arbitrary pattern matching so you can do left hand truncation as well as
right and mid-term patterns. Having patterns accommodate the bad spelling in
America ;-) so us English can spell colour correctly and yet search for colo?r
and get all the mis-spellings. :-))

Of course (right) truncation can be simulated with a stemmer such as Porter,
available at snowball.sourceforge.net but seemingly not in Perl. However, this
doesn't cope with languages that have a highly irregular verb system or where
consideral morphological analysis has to be performed so that the root term can
be discovered. (One of the most difficult I've had to deal with in that respect
has been Finnish with its huge case system.)

There are a number of open source retriecval systems around. They don't all
offer the same facilities, they're not all available in Perl. Some of these open
sourced systems don't do truncation, some don't do boolean searches, some don't
do proximity searches. Some still insist on using stop-lists.

Comment 12

[Myk Melez [:myk] [@mykmelez]]

Trevor, can you recommend one that's open source, works with Perl, and is the
most attractive in terms of features, usability, etc.?

Comment 13

[Daniel Wang]

Bugzilla Thesaurus (work-in-progress):
http://devsupport.mozdev.org/QA/thesaurus/

Comment 14

[Myk Melez [:myk] [@mykmelez]]

*** Bug 191724 has been marked as a duplicate of this bug. ***

Comment 15

[Bradley Baetz (:bbaetz)]

See my comments in bug 191724.

postgres does stemming, which would probably be useful

Comment 16

[Myk Melez [:myk] [@mykmelez]]

Attachment: work in progress

Implementation of full text search.  No UI yet.

Comment 17

[Bradley Baetz (:bbaetz)]

Hmm. I'm not sure if releveance is that useful. Wouldn't a boolean match be
better? That way it can just be a replacement for 'containsanywords', which is
what I thought was going to happen.

Also, does the product_and_status index make much difference if you're searching
on more than one status (which is the common case)?

Comment 18

[Myk Melez [:myk] [@mykmelez]]

>Hmm. I'm not sure if releveance is that useful. Wouldn't a boolean match be
>better?

I think they're both useful.  Relevance, like at Google, is great when looking
for a specific bug.  It should be invokable from a simple search form (perhaps
at the top of query.cgi, perhaps overloading the find form at the bottom of
every page).

>Also, does the product_and_status index make much difference if you're 
>searching on more than one status (which is the common case)?

It seems to help in some cases, although more testing is needed.

What I'd really like to do is join in duplicates bugs and do fulltext searches
on their summaries and description/comments as well, but that doesn't seem to
work because the MATCH() condition doesn't seem to accept a table qualifier on
the column name to search, at least according to my tests and an annotation from
November 19 2002 on this page:

http://www.mysql.com/doc/en/Fulltext_Search.html

Comment 19

[Bradley Baetz (:bbaetz)]

The problem is that 'relevance' needs a reasonable ammount of text to be useful.
The bug summary isn't enough, and neither are most comments. Itwould be useful
to compute a relevance index over _all_ of the comments of a bug, but mysql
can't do that. (I _think_ Pg can, if you fudge it a bit)

MATCH is a hack in mysql - its a lookup in an index. You can only apply it if
the index is present, and you can only index complete columns.

Comment 20

[Myk Melez [:myk] [@mykmelez]]

>The problem is that 'relevance' needs a reasonable ammount of text to be useful.

Is that really per record rather than overall?  So far my tests have shown that
it's reasonably effective.

>MATCH is a hack in mysql

Still, you should be able to say MATCH(foo.column).

Comment 21

[Bradley Baetz (:bbaetz)]

Well, it is per record. If you're searching for a bug about <whatever>, the bugs
most likely to be of interest are those which mention <whatver> in all of the
comments, rather than just one. If you combine the relevance via an aggregate,
then you may get an interesting result, but its not a linear scale, since all of
the comments sizes are different.

Well, you should be able to do table.column. What happens if you have two
columns in different tables with the same name, and want to fulltext on one of
them? What does mysql do? Is the problem if you are using a table alias? I could
beleive that, given how tied this all is to the underlying index.

Comment 22

[Myk Melez [:myk] [@mykmelez]]

Attachment: work in progress #2

This version divides the sum of the comment relevance by the number of comments
to account for variations in the number of comments per bug; it also multiplies
summary relevance by two to give it extra weight.  If comment size really
matters we could account for it in the calculation (f.e. perhaps dividing by
the total length of all comments instead of the number of comments), but I
suspect that this is unnecessary.

This version also adds UI in the form of a form at the top of the query page
with fields for status, product, and descriptive words that we search against
the full text index (work in progress; the form appears to break the regular
query form).

I said:

>What I'd really like to do is join in duplicates bugs and do fulltext searches

>on their summaries

I've decided this is better done as a separate bug (bug 24957).

bbaetz said:

>Well, you should be able to do table.column.

After more experimentation I think the problem may be the traditional one that
MySQL can't use two indexes for the same table.  Since fulltext indexes can't
be combined with traditional indexes, this makes it hard to join a table via a
key and then use the fulltext index to search it.  My experiments found that
the fulltext index could be used, but then the key index wouldn't be used, and
performance is bad.  Thus my suggestion in bug 24957, comment 30 that we use
UNION to run two queries, one on original bugs and one on duplicates.

Comment 23

[Bradley Baetz (:bbaetz)]

Hmm. You also need to take into account the relative lengths of teh comments,
and stuff.

If its abstracted enough so that postgres can use 'real' indexing, though (we
can update the index on the bug table via a trigger whenever the longdescs table
changes) then I guess we can try it and see.

Where did you get teh limit 10 from? I don't see that in the SQL.

Does the joined product/status field actually help? I'd be surprised if it did
help on any significant number of the queries people do, since there are other
fields involved also.

Comment 24

[Myk Melez [:myk] [@mykmelez]]

>Hmm. You also need to take into account the relative lengths of teh comments,
>and stuff.

Taking the relative lengths of comments into account is a database function, not
a function of our code.  Our manipulation of the relevance should be limited to
things that the database can't or won't do (i.e. aggregating relevance values).
 It's also icing on the cake, since comment sizes don't vary widely on average,
so full text search would work pretty well even without it.  Thus, even if we
wanted to do it ourselves, it could (and should, unless it's easy to do) go into
a separate bug.

What other stuff did you have in mind?

>If its abstracted enough so that postgres can use 'real' indexing, though (we
>can update the index on the bug table via a trigger whenever the longdescs 
>table changes) then I guess we can try it and see.

I don't think we should impose too many (or any) limitations on code going into
the tree to maintain compatibility with PostGreSQL when no code to support that
database system has gone in yet.

Nevertheless, I think the interface is pretty well abstracted.  Users search
"content" for words that "match", and Search.pm gets to decide what that means.

>Where did you get teh limit 10 from? I don't see that in the SQL.

Search.pm sets a variable on the query object, and buglist.cgi sees that and
adds the LIMIT.  It should probably be higher than ten, though, perhaps 100.

>Does the joined product/status field actually help? I'd be surprised if it did
>help on any significant number of the queries people do, since there are other
>fields involved also.

I think it does help, since although people search other fields, they regularly
search those two as well, and I often see one or the other of those indexes
being used by the query optimizer to optimize queries.

Comment 25

[Myk Melez [:myk] [@mykmelez]]

Attachment: patch v1: implementation

Here's a cleaned-up version of the work in progress.  I took out the extra
weight given to summary relevance (couldn't justify it; seems like we should
try the simpler approach first) and removed the product/summary combined index
(I'll file a separate bug on this).  I also cleaned up the UI a bit.

Comment 26

[Myk Melez [:myk] [@mykmelez]]

Attachment: patch v2: fix for syntax error

This patch fixes a syntax error with the last patch.

Comment 27

[Bradley Baetz (:bbaetz)]

> Taking the relative lengths of comments into account is a database function, not
> a function of our code. 

True, but of you index each comment separately, the db can't do comparisns
across multiple bugs, given teh length of the comments. Relevance is a % ting,
not an absolute value.

Anyway, I'll look at this later today.

Comment 28

[Asa Dotzler [:asa]]

a few comments (this rocks!).

1. buglist columns still contain clickable headers (for sorting) but resorting
the list doesn't work. Could either disable the links so the user doesn't think
he sorted columns or make them actually sort (and maybe add the "relevance"
column for searches made from the fulltext field)
2. "open" includes Unconfirmed bugs and probably shouldn't (my opinion, for bmo,
at least). If it should contain Unconfirmed then the status select on query.cgi
needs to  reflect that.
3. not sure if it's possible (without js?) but it would be nice if selecting
status or product from the new field immediately updated the matching listboxes
visible lower on the page. It's kind of confusing to select closed and still see
the default "New, Assigned, Reopened" still selected lower in the page. 

Comment 29

[Bradley Baetz (:bbaetz)]

Comment on attachment 122525 [details] [diff] [review]
patch v2: fix for syntax error

This looks fine (sory for the long delay...) Is this running with bmo's data
anywhere, so that I can test the effectiveness?

Comment 30

[Myk Melez [:myk] [@mykmelez]]

Attachment: patch v3: fixes for issues raised and others

This version fixes some issues:

1. it's hard to distinguish between the "find a specific bug" search form and
the regular search form, plus the interaction between the two is tricky (f.e.
auto-updating one status field when the other changes)

-> The new version puts the specific search form into a new format file and
adds a tabbed interface to the top of query.cgi for selecting the kind of
search to do.

2. you can't resort, but the links for it are there

-> The new version removes the links.  In the long term we should make it
possible to see the "relevance" number and sort by it or by other headers, but
that's a v2 feature.

3. "open" shouldn't include unconfirmed bugs

-> I chatted with Asa in person about it, and he changed his mind.  Unconfirmed
is not useful for some searches but is for others.  Perhaps we want to add an
"open-unconfirmed" option to the "status" drop-down, but I'd like to gauge user
demand post-upgrade before doing this.

The new version also modifies the guided bug entry form to use full text search
instead of quick search.

Comment 31

[Myk Melez [:myk] [@mykmelez]]

Comment on attachment 122525 [details] [diff] [review]
patch v2: fix for syntax error

bbaetz- Sorry, I don't have a public playground set up yet with b.m.o data. 
I'll try to set one up, but I'm not sure how soon I'll be able to (I'd like to
do it on the new servers, but they won't be in place until next Thursday, and
I'm on vacation from next Friday).

Asa's been using the the internal playground I set up, and he reports that it's
highly effective at finding the bugs he is looking for. FWIW.  Do you need this
testing for your review or just out of curiousity?

Comment 32

[Bradley Baetz (:bbaetz)]

(I never got the review mail - acm.org was bouncing me for some reason).

More for curiousity than anything else. I'll try to look at this this weekend,
but I have lots of other stuff on.

Comment 33

[Bradley Baetz (:bbaetz)]

Comment on attachment 126091 [details] [diff] [review]
patch v3: fixes for issues raised and others

>Index: Bugzilla/Search.pm

>@@ -364,6 +379,43 @@
>              push(@supptables, "longdescs $table");
>              push(@wherepart, "$table.bug_id = bugs.bug_id");
>              $term = "$table.bug_when > " . &::SqlQuote(SqlifyDate($v));
>+         },
>+         "^content,matches" => sub {

>+             my $term1 = "MATCH(bugs.short_desc) AGAINST(".&::SqlQuote($v).")";
>+             my $term2 = "MATCH($table.thetext) AGAINST(".&::SqlQuote($v).")";
>+             $term = "$term1 AND $term2";

This only finds something if its mentioned in the title and the longdesc.
Also, because this is part of the where clause and not HAVING, the averaging
calculation which you do averages teh score of all teh comments where this text
is mentioned, rather than averaging all comments. That doesn't seem right.



>Index: template/en/default/search/search-specific.html.tmpl

>+<table>
>+  <tr>
>+    <td align="right" valign="baseline">
>+      <b><label for="bug_status">Status:</label></b>

Add a space after the :, and do the same for the other fields.

Should there be an 'all products' option?

Comment 34

[Jack Tanner]

I don't know if this is still relevant to the current implementation, but in
comment 12 Myk asked for Porter's Stemmer in Perl. See
http://www.tartarus.org/~martin/PorterStemmer/.

Comment 35

[Myk Melez [:myk] [@mykmelez]]

>This only finds something if its mentioned in the title and the longdesc.

Hmm, right, that is a problem.  Unfortunately "title OR longdesc" is prone to
performance problems with larger data sets, since it can't use the full text
indexes to limit rows.  Probably the best thing we can do for now is only search
comments, which is almost as good, since words in bug summaries tend to be
repeated in comments.  Later we might want to find some way of distinguishing
between small and large searches and add summaries back into the small ones.

>Also, because this is part of the where clause and not HAVING, the averaging
>calculation which you do averages teh score of all teh comments where this text
>is mentioned, rather than averaging all comments. That doesn't seem right.

Averaging against all comments would be better, but HAVING takes place after the
GROUP BY clause, which means individual comments are not available to the HAVING
clause, so we can't search them there.

We could remove the MATCH statements from the WHERE clause, leave them in the
SELECT clause, average against all comments in the SELECT clause, and limit the
result set to bugs with relevance via the HAVING clause, but that'll run into
performance problems.

In theory we should be able to join the longdescs table again to get the count
of all comments, but MySQL does strange things when you join a table twice in
one query and then apply group functions to it.  It makes COUNT(matching
longdescs) equal COUNT(all longdescs) and SUM(longdescs relevance) be equal to
SUM(longdescs relevance) * COUNT(all longdescs).

I think the best we can do here is keep the current calculation despite its
limitations.  Again, down the road we might distinguish between large and small
queries and do something better for the small ones.


>Should there be an 'all products' option?

Yeah, I wonder where that went.  I'll add it back.

Comment 36

[Myk Melez [:myk] [@mykmelez]]

Attachment: patch v4: review updates

Per my remarks in comment 35, this version searches only in comments (but it
still uses a summary search in calculating relevance).

>Add a space after the :, and do the same for the other fields.

Why?  There's space after the </b> tag, and there's plenty of space in the UI
after the label (nor does adding a space there add any space in one common
standards-compliant browser I tested).

>Should there be an 'all products' option?

Yes, it's the first option in the "Products" drop-down menu.

Comment 37

[Myk Melez [:myk] [@mykmelez]]

I've got a test version up now at:

http://mecha.mozilla.org/webtools/bztest/query.cgi?format=specific

Comment 38

[Bradley Baetz (:bbaetz)]

Comment on attachment 130450 [details] [diff] [review]
patch v4: review updates

If I search for 'ftp uri format', I get back bugs (at the top of the list),
which just mention 'format' a lot (as in 'format toolbar'), and don't contain
the other terms. I suspect that this is a mysql thing we can't fix. Is there
syntax for 'all words', or does that require mysql 4?

If I search for 'proxy wrong page', I get 85188 bugs returned. We need to cap
the results from this search to (say) 200 or so.

I tired searching for our mostdupe bug, with 'table image space', and that
worked well (except for returning 28005 bugs)

Do you remember how long the index took to create, BTW?

tab.description needs to be html escaped, and it wouldn't hurt to do that to
the name, too.

This is all an evil hack, but if you add the limit, r=bbaetz. The only other
requirement is that this page may end up looking different under different
databases, due to additional features. This means that the query params may be
different, so don't bookmark them (and we shouldn't let them be saved queries
either, really)

Comment 39

[Myk Melez [:myk] [@mykmelez]]

Attachment: patch v5: review updates

>If I search for 'ftp uri format', I get back bugs (at the top of the list),
>which just mention 'format' a lot (as in 'format toolbar'), and don't contain
>the other terms. I suspect that this is a mysql thing we can't fix.

Actually it's a configuration option (ft_min_word_len, i.e. minimum length of
words to index).  I've lowered it to two characters on b.m.o and rebuilt the
indexes.  Try your search again.

>Is there syntax for 'all words', or does that require mysql 4?

It requires MySQL 4.0.1+ and the addition of "IN BOOLEAN MODE" to appropriate
parts of the query.  I've done it on b.m.o, which is running 4.0.14, but we'll
need to wait to check that into the trunk, since the trunk currently only
requires 3.23.something.

The syntax, by the way, is a plus sign before the words that have to be there
and a minus sign before the words that can't be there.	There are other
operators too; read the following page for more info:

http://www.mysql.com/doc/en/Fulltext_Search.html


>If I search for 'proxy wrong page', I get 85188 bugs returned. We need to cap
>the results from this search to (say) 200 or so.

Done, and I've updated the message at the top of the bug list to say that.


>Do you remember how long the index took to create, BTW?

mysql> repair table bugs quick;
+-----------+--------+----------+----------+
| Table     | Op     | Msg_type | Msg_text |
+-----------+--------+----------+----------+
| bugs.bugs | repair | status	| OK	   |
+-----------+--------+----------+----------+
1 row in set (34.60 sec)

mysql> repair table longdescs quick;
+----------------+--------+----------+----------+
| Table 	 | Op	  | Msg_type | Msg_text |
+----------------+--------+----------+----------+
| bugs.longdescs | repair | status   | OK	|
+----------------+--------+----------+----------+
1 row in set (5 min 20.84 sec)


>tab.description needs to be html escaped, and it wouldn't hurt to do that to
>the name, too.

Why?  The template defines those names in a data structure and never uses any
others.


>This is all an evil hack, but if you add the limit, r=bbaetz.

Not sure why it's all a hack, since it does most of what we want exactly as we
want it, but limit added.  It looks like you mean for me to carry your review
forward, so doing so.


>The only other requirement is that this page may end up looking different
under 
>different databases, due to additional features. This means that the query 
>params may be different, so don't bookmark them (and we shouldn't let them be 

>saved queries either, really).

The parameters on this page are pretty basic.  There are some useful standard
ones (query_format, product, and bug_status) and then "content," which contains
the words to search on.  It's hard to imagine implementations that take any of
these parameters away (is there a web search engine that doesn't ask you for
some words to search for?), and installations won't be changing their database
backends regularly, but in any case these queries are unlikely to be saved,
since they aren't useful except for finding that one bug you are looking for,
at which point it's the bug that needs bookmarking.

Comment 40

[Myk Melez [:myk] [@mykmelez]]

Comment on attachment 130797 [details] [diff] [review]
patch v5: review updates

Carrying forward bbaetz' review.

Comment 41

[Myk Melez [:myk] [@mykmelez]]

a=myk

Checking in buglist.cgi;
/cvsroot/mozilla/webtools/bugzilla/buglist.cgi,v  <--  buglist.cgi
new revision: 1.234; previous revision: 1.233
done
Checking in checksetup.pl;
/cvsroot/mozilla/webtools/bugzilla/checksetup.pl,v  <--  checksetup.pl
new revision: 1.247; previous revision: 1.246
done
Checking in query.cgi;
/cvsroot/mozilla/webtools/bugzilla/query.cgi,v  <--  query.cgi
new revision: 1.120; previous revision: 1.119
done
Checking in Bugzilla/Search.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Search.pm,v  <--  Search.pm
new revision: 1.45; previous revision: 1.44
done
Checking in template/en/default/filterexceptions.pl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/filterexceptions.pl,v 
<--  filterexceptions.pl
new revision: 1.8; previous revision: 1.7
done
Checking in template/en/default/bug/create/create-guided.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/bug/create/create-guided.html.tmpl,v
 <--  create-guided.html.tmpl
new revision: 1.11; previous revision: 1.10
done
Checking in template/en/default/global/messages.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/global/messages.html.tmpl,v
 <--  messages.html.tmpl
new revision: 1.13; previous revision: 1.12
done
Checking in template/en/default/list/list.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/list/list.html.tmpl,v 
<--  list.html.tmpl
new revision: 1.13; previous revision: 1.12
done
Checking in template/en/default/list/table.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/list/table.html.tmpl,v 
<--  table.html.tmpl
new revision: 1.15; previous revision: 1.14
done
Checking in template/en/default/search/boolean-charts.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/search/boolean-charts.html.tmpl,v
 <--  boolean-charts.html.tmpl
new revision: 1.6; previous revision: 1.5
done
RCS file:
/cvsroot/mozilla/webtools/bugzilla/template/en/default/search/search-specific.html.tmpl,v
done
Checking in template/en/default/search/search-specific.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/search/search-specific.html.tmpl,v
 <--  search-specific.html.tmpl
initial revision: 1.1
done
Checking in template/en/default/search/search.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/search/search.html.tmpl,v
 <--  search.html.tmpl
new revision: 1.15; previous revision: 1.14
done
RCS file:
/cvsroot/mozilla/webtools/bugzilla/template/en/default/search/tabs.html.tmpl,v
done
Checking in template/en/default/search/tabs.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/search/tabs.html.tmpl,v 
<--  tabs.html.tmpl
initial revision: 1.1
done

Comment 42

[Bradley Baetz (:bbaetz)]

Repair table quick doesn't rebuild the indexes, does it? I thought that that was
the difference between quick and normal. If it takes 5 minutes to rebuild the
indexes for all of bugzilla's data, then mothra is way too fast :)

Can that mysql param be docmented in the docs somewhere?

I think its a bit too early to require mysql 4 - no distribution ships with it
in stable as far as I know (and even if some do, its not the majority). I also
don't want to add conditionals in there at this stage. Lets look at it again in
a few months, or if we want mysqlv4 for other reasons.

The 'hack' bit is the limited fields we do this on, due to the
one-index-per-table-per-query issue mysql has.

Comment 43

[Myk Melez [:myk] [@mykmelez]]

"REPAIR TABLE tablename QUICK" is the recommended way to rebuild a full-text
index per the following doc:

http://www.mysql.com/doc/en/Fulltext_Fine-tuning.html

Bug 218206 has been filed for documenting this feature.

I agree we shouldn't require MySQL 4 now; I think we should require it in 2.19,
which will be the new development branch and which won't get out of development
as 2.20 until long after MySQL 4 becomes the universal standard.

Comment 44

[Myk Melez [:myk] [@mykmelez]]

*** Bug 79748 has been marked as a duplicate of this bug. ***

Comment 45

[Myk Melez [:myk] [@mykmelez]]

*** Bug 121762 has been marked as a duplicate of this bug. ***