485300 - [newsearch] "Did you mean?" suggests the same search terms

Status: VERIFIED FIXED

(support.mozilla.org :: General, defect)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

Steps to Reproduce:

1. Load http://support-stage.mozilla.org/tiki-newsearch.php?locale=ja&q=%E8%B3%AA%E5%95%8F&where=d&l=ja&filter_lang=1&author=&filter_author=0&en_too=1&type=0&answered=0&lastmodif=0
2. Click on the "Did you mean '質問'?" link: (http://support-stage.mozilla.org/tiki-newsearch.php?locale=ja&q=%E8%B3%AA%E5%95%8F&where=d&l=ja&filter_lang=1&author=&filter_author=0&en_too=1&type=0&answered=0&lastmodif=0)

Expected Results:

After clicking that link once, the correct characters/phrase should be found and the user should be redirected to that term's page

Actual Results:

It keeps suggesting the same string

Comment 1

[Paul Craciunoiu [:paulc]]

I'll take a look tomorrow morn ;)

Comment 2

[Laura Thomson :laura]

This is needed for 1.0.2

Comment 3

[Paul Craciunoiu [:paulc]]

I don't get this specific URL locally (maybe it's the reindexing?)
http://support-stage.mozilla.org/tiki-newsearch.php?locale=ja&q=%E8%B3%AA%E5%95%8F&where=d&l=ja&filter_lang=1&author=&filter_author=0&en_too=1&type=0&answered=0&lastmodif=0
In other words, there's no "Did you mean?" for me.

I'd need a new case to reproduce this. I'm playing around locally but I don't get any yet.

Comment 4

[E. Cooper [:ecooper]]

FWIW, I don't experience this locally, either. I suggest we wait until bug 488374 is resolved since that's the big piece missing between the local and staging instances.

Comment 5

[Paul Craciunoiu [:paulc]]

Agree with you, Eric.

Comment 6

[Stephen Donner [:stephend] Not actively reading bugmail]

https://support-stage.mozilla.org/tiki-newsearch.php?locale=en&q=VG+safe&where=d&l=en&filter_lang=1&author=&filter_author=0&en_too=1&type=0&answered=0&lastmodif=0, which is a search for "VG Safe", keeps suggesting the same search over and over, without ever messaging the user that no search result exists for that phrase

Comment 7

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #6)
> https://support-stage.mozilla.org/tiki-newsearch.php?locale=en&q=VG+safe&where=d&l=en&filter_lang=1&author=&filter_author=0&en_too=1&type=0&answered=0&lastmodif=0,
> which is a search for "VG Safe", keeps suggesting the same search over and
> over, without ever messaging the user that no search result exists for that
> phrase

Correction; sorry, I mean that it does, below, "No pages matched the search criteria", but then suggests the same term, as already noted above (it's not very noticeable, in comparison to the "Did you mean..." text.

Comment 8

[E. Cooper [:ecooper]]

So, there are a few problems at play here. 

Firstly, there seems to be a problem with query encoding. A search for 'この' in the ja locale should give you a ton of articles, but only gives articles for the English translation. Using mb_convert_encoding with 'UTF-8' and 'auto' can fix this.

Secondly, "Did you mean" doesn't work with languages that don't use a 'space' as a delimiter for words. Not so sure about a fix for this. I need to look at the 'Did you mean' code still.

Comment 9

[Paul Craciunoiu [:paulc]]

(In reply to comment #6)
> https://support-stage.mozilla.org/tiki-newsearch.php?locale=en&q=VG+safe&where=d&l=en&filter_lang=1&author=&filter_author=0&en_too=1&type=0&answered=0&lastmodif=0,
> which is a search for "VG Safe", keeps suggesting the same search over and
> over, without ever messaging the user that no search result exists for that
> phrase
First, I see it on staging, but still doesn't happen locally. Eric, do you want to proceed with the mb_convert_encoding fixes? I can help out.

Secondly, "Did you mean" will probably be better to implement using an open source spell checker, or something of the sort. A few examples are in this thread:
http://www.sphinxsearch.com/forum/view.html?id=526
OR, even better, Media Wiki already implemented this from SphinxSearch, and we could try to use their code:
http://www.mediawiki.org/wiki/Extension:SphinxSearch#Did_You_Mean

Laura, what do you think?

Comment 10

[E. Cooper [:ecooper]]

I filed bug 489277 to handle the encoding issues.

Comment 11

[Paul Craciunoiu [:paulc]]

Attachment: patch, v1

This is a very simple patch that should take care of it.

Comment 12

[Paul Craciunoiu [:paulc]]

Attachment: patch, v2

Oops, of course I meant "!=" not "==" :(

Comment 13

[Paul Craciunoiu [:paulc]]

Attachment: patch, v2

I need to watch my attaching skills.

Comment 14

[[:Cww]]

Can just this one go in with 1.3?  It's frustrating/confusing to users.

Comment 15

[Paul Craciunoiu [:paulc]]

r48549 / r48550

Comment 16

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED on https://support-stage.mozilla.org/tiki-newsearch.php?locale=en&q=VG+safe&where=d&l=en&filter_lang=1&author=&filter_author=0&en_too=1&type=0&answered=0&lastmodif=0; I regression-tested a bit as well (will keep testing).