525953 - Create custom word list for SUMO search suggestion

Status: VERIFIED FIXED

(support.mozilla.org :: Search, defect, P3)

Description

[James Socol [:jsocol, :james]]

We'll need to find a method to build a custom word list to handle spelling suggestions for things like "Fierfox" and "Faecbook".

Comment 1

[James Socol [:jsocol, :james]]

I keep getting "Fairfax" when I try "fierfox", so we'll probably also need a custom replacement word list.

http://us2.php.net/manual/en/function.pspell-config-repl.php

Comment 2

[Stephen Donner [:stephend] Not actively reading bugmail]

You can get pretty crazy with this, but here's a starting point; we should somehow allow for simple transpositions (and letter omissions):

firfox
firefix
furefox
addon
adon
bokmarks
crashs
crashws
facebok, etc.

Comment 3

[Paul Craciunoiu [:paulc]]

I'm wondering if we could specify a list of words that are preferred and really important -- for which more than usual # of mistakes would still bring them up. Basically same thing as what Stephen said above :)

Comment 4

[Chris Ilias [:cilias]]

I've filed bug 529347 separate from this to cover cases in which a space is used. (i.e. home page, book mark, web site, zone alarm)

Comment 5

[James Socol [:jsocol, :james]]

FTR, bug 529347 is about the indexer, which is not related to the spelling suggestions.

Comment 6

[James Socol [:jsocol, :james]]

Attachment: 30 key words in English

These are key names, brands, and other words I gleaned from talking with #sumodev and the first 20-ish pages of the forums. I also tweaked the add_to_dict script to make it easier to change the default language. This patch does not implement a custom replacement list (yet).

And I really need to clarify what this bug is and what it needs: this is about the spelling suggestion feature only. If you're worried that a result should appear but doesn't--if you're worried about results at all--that's not this bug. Check the wordforms bug (bug 529347) or file a new one.

This needs two things:
1) Words common to SUMO that would not be in a typical English dictionary--eg: "Firefox", "Google", "JavaScript", "username".
2) Likely or common misspellings that are generating the wrong suggestion.

An example of the latter: if you enter "firefix" and the site suggests "Fairfax" but not "firefox", that would be relevant here. If, on the other hand, you search for "addon" and expect results that contain "add-on", please head to bug 529347.

Comment 7

[Paul Craciunoiu [:paulc]]

Comment on attachment 413474 [details] [diff] [review]
30 key words in English

WFM

Comment 8

[James Socol [:jsocol, :james]]

r56613.

Obviously we'll need to add more terms over time and a bunch of new locales (bug 529903).

QA: take a look at the patch and look at the words with a +. Those should no longer trigger suggestions and near misspellings should suggest correctly.

Comment 9

[Vishal Kamdar [:vish_moz]]

Verified, FIXED.
words with + in the patch do no trigger suggestions, however it still does not suggest correctly for misspellings. Filed Bug 532002 for misspellings.