498119 - SUMO search returns no results for forum

Status: VERIFIED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Paul Craciunoiu [:paulc]]

STR:
1. Go to http://support.mozilla.com/en-US/kb/Support+Website+Forums
2. Search for anything in the search box, e.g. "browser vista"

Actual results:
No search results on the page.

Expected results:
Search results from the forum.

Comment 1

[Paul Craciunoiu [:paulc]]

sphinx search uses two indexes: for KB (done by scripts/sphinx/indexer.php) and for forums (done by scripts/sphinx/indexer-forums.php), with "documents" and "forum" as the index names in sphinx.conf

The KB index seems to work, so perhaps a good solution is to reindex for forums only/restart searchd.

Comment 2

[chizu]

Running the reindex manually, documents threw the error:
indexing index 'documents'...
ERROR: index 'documents': source 'documents': XML parse error: no element found (line=20232, pos=19, docid=24735).

What would you need to further debug based on that error?

Comment 3

[E. Cooper [:ecooper]]

So, this looks like /scripts/sphinx/indexer.php is producing bogus XML in a brand new way. I'm not sure if a failed index will stop the indexer all together, but I'm guessing it does if the forum threads aren't showing up.

The simplest thing to do would be to just look at the XML being outputted. Could the results of the following be attached to the bug?

sed -n '20232 p' /usr/local/sphinx/index/data.xml

Or head can be piped into tail since data.xml is going to be huge and sed slow. All we need is that one line.

As an aside, I'm not entirely sure if the indexes are kept at /usr/local/sphinx/index/ on staging. The sphinx.conf can be checked for reference.

Comment 4

[chizu]

That's the last line in the file (with the four preceding lines):
                <keywords><![CDATA[]]></keywords>
                <pollyes>6</pollyes>
                <pollcoeff>0.5000</pollcoeff>
                <type>18</type>
        </sphinx:document>

Comment 5

[E. Cooper [:ecooper]]

If that is the last line of the file, the error is because the closing tag for '</sphinx:docset>' is missing. It should directly follow the last '<sphinx:document>' element. With that missing, I can reproduce the 'no element found' error locally. I'm not sure why that would be missing, though.

Unfortunately, the error in comment 2 is unrelated to our original problem. I'll file a bug shortly for us to investigate this issue.

The good news is that our original bug seems to have been solved, and the cron seems to be working since I can get forum threads from yesterday with targeted queries.

<http://support.mozilla.com/tiki-newsearch.php?where=f&q=Why+do+I+keep+getting+this+message%3AThe+network+link+was+interrupted+while+negotiating+a+connection.+Please+try+again.++&sa=>

The STR also produces proper results if you don't search for 'browser vista', but that's a separate issue involving the default boolean choice for multiple words (uses AND and not OR).

Comment 6

[Stephen Donner [:stephend] Not actively reading bugmail]

The URL in comment 5 WFM too; verified FIXED.