1122434 - Add full text search capabilities using Plucene

Status: RESOLVED WONTFIX

(Bugzilla :: Query/Bug List, enhancement)

Description

[Jochen Wiedmann]

Currently full text search capabilities have to be provided by the database. This is a bad thing (TM), because it leads to inconsistencies between varying Bugzilla installations, depending on the underlying database. Besides: If the database doesn't provide such capabilities, there is no full text search available. (Althiough the UI suggests differently.)

A better thing (TM) would be to have a full text index created with Plucene (I'd prefer the Pure-Perl variant over the native Lucene module, so that it can really be assumed to be present and use that index. Necessary steps:

- Have enter_bug add the bug to the index.
- Have save_bug update the index.
- If a full_text search is present, add a filter to the query, which removes those bugs from the query result that don't match the full text search expression.
- Add an upgrade to checksetup.pl that adds all bugs to the index, that aren't present.

I don't know where the index must be stored. I am sure that a local file, or directory, say data/Plucene could be used. However, for reasons of maintainability, it would certainly be better to store the index in the database. (For example, we've got a daily backup for that anyways, do we?)
However, I have no idea whether Plucene does support that.

Comment 1

[Frédéric Buclin]

(In reply to Jochen Wiedmann from comment #0)
> Currently full text search capabilities have to be provided by the database.
> This is a bad thing (TM), because it leads to inconsistencies between
> varying Bugzilla installations, depending on the underlying database.
> Besides: If the database doesn't provide such capabilities, there is no full
> text search available. (Althiough the UI suggests differently.)

MySQL, PostgreSQL, SQLite and Oracle all natively support fulltext searches. We don't support any other database. So the "if" seems pretty irrelevant to me at this point. Moreover, I wouldn't want to depend on a module (Plucene) which hasn't been updated since 2005 (already 10 years!). Some of the bugs reported against Plucene suggest that it doesn't support Unicode correctly, which is IMO a showstopper.

Sphinx already supports MySQL, Pg and Oracle, see bug 342856 and http://sphinxsearch.com/docs/current.html#intro. There is no need for another indexer.

Reopen if you disagree.

Comment 2

[Jochen Wiedmann]

I understand your concerns about the state of Plucene. OTOH, I don't understand why one would reject using a module, and rather depend on an external Server, as suggested by bug 342856.

Question: The presence of said bug suggests that there is, in general, the desire for more, or better capabilities in full text search. Should we rephrase the summary as "Add an API for support of external/internal full text indices, other than the database"?

Btw, another alternative would be Apache Lucy. Unlike Plucene, it seems to be quite active:

http://mail-archives.apache.org/mod_mbox/lucy-dev/201412.mbox/browser