517699 - index unCamelized and UnSmushedText in search index

Status: RESOLVED WONTFIX

(addons.mozilla.org Graveyard :: Public Pages, defect)

Description

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

For any addon which has camelCasing or smushedText for the title, we want to normalize it and store those terms in a new table.

E.g. If a new addon named MonkeyMania is created, we'd make a new entry in our search_extras table for "monkey mania".

We'll want this in a separate table as this table will only affect a few addons - and will only be accessed by sphinx's translated_addons view.

Comment 1

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

I explain the issue in more detail here:

http://spindrop.us/2009/09/19/mysql-and-the-grand-regexp-retardedness-with-lettercasing/

It's a more general solution to:
https://bugzilla.mozilla.org/show_bug.cgi?id=517344.

I don't think this should be done for 5.1 - but it should be done shortly thereafter.

Comment 2

[Wil Clouser [:clouserw]]

We have to do this manually on our end?  Sphinx isn't smart enough to figure this out itself? :-/

Comment 3

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

Well I'm sure sphinx is extendable, I could try to write a plugin for it too - perhaps we should make this bug title to be a request for a more generic solution to the problem?

Comment 4

[Wil Clouser [:clouserw]]

Would it benefit SUMO?  Also, maybe someone else already wrote one? ;)

I think more generic would be good.

Comment 5

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

Doubtful if this would help SUMO - the SmushedText is a nature of what we index, whereas SUMO indexes supporty-text-heavy stuff.

e.g. FireBug is an important search term for us, whereas SUMO I would think would have large text strings "how do I use private browsing"

I asked the community forum about this, and I'm happy pushing this off until we talk to Percona ~ Oct 12 as they may have some clue.  I'll look into extending Sphinx - if it's worth doing.

Comment 6

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

Using substring searchs seems to eliminate this problem.

FireBug now contains infixes: Fire and Bug so FireBug will be a hit for "Fire Bug"