Closed Bug 405028 Opened 13 years ago Closed 11 years ago
Switch search engine on sumo
We want to switch to a different search engine on sumo. Criteria and possible candidates to come.
This doesn't have a target yet, I think it's important for making the site easier to use, as well as giving better control of search results, especially now that we're dealing with many different locales.
I'm not sure if this is feasible for 0.6, as we've yet to find a better search engine. Fwiw, the google search can filter results based on language settings, so that in itself shouldn't be a problem.
what happened with the search into lucene?
May be of interest: bug 419676
Assignee: nobody → nelson
Target Milestone: --- → 0.8
With this patch, the only things blocking new search on production is load testing, configuring of sphinx to use cluster (for maximum performance), and indexing cron job. Note that to activate on production requires modification of "if clause" in searchbar.tpl (see bug 468090 for why we are blocking the functionality so strongly before hand).
Attachment #352096 - Flags: review?(laura)
Attachment #352096 - Flags: review?(laura) → review-
Comment on attachment 352096 [details] [diff] [review] new search ui touchup, to make it look like old search (also block use on production until later) Generally ok, but we won't need to block use after the next push (quite the opposite), so I think we need to delete that part. Matthew's JS has fixed this for the time being.
Attachment #352256 - Flags: review?(laura) → review+
Comment on attachment 352256 [details] [diff] [review] same patch as before except with no blocking of access in r20678/r20679. Bug remains open for remaining blocking bugs.
In trying to deploy Sphinx to production (bug 460213) in tonight's 0.8 push, we discovered a blocking problem (bug 470381) which was not caught on support-stage because support-stage was not configured to disallow writes on the slave db. This has been rectified so we can now test properly on support-stage. However, to fix this problem needs code refactoring and repush, so we have either to schedule this for 0.8.1 or if you want to be aggressive with timeline, to schedule a "0.8.0.5" sometime sooner. There is also a problem that appeared on support-stage (bug 470386) that is preventing reindexing of forum content that should be fixed before we switch.
sumo.loadtest.mozilla.com should be ready for testing now. Sphinx searchd is running on ml-app10 alone. Just visually - seems much faster, especially forum search. Toggle search method here http://sumo.loadtest.mozilla.com/tiki-admin_newsearch.php https://bugzilla.mozilla.org/show_bug.cgi?id=470386 has a patch on it too (which is applied on the loadtest cluster). Indexing from scratch took: [root@ml-app10 webroot]# php manual_index.php Data retrieved from the db 1147 wiki pages successfully parsed in 880.358 s [root@ml-app10 webroot]# php manual_index_forums.php Data retrieved from the db 96647 topics successfully parsed in 823.146 s [root@ml-app10 webroot]# /usr/bin/indexer --all --rotate Sphinx 0.9.8.1-release (r1533) Copyright (c) 2001-2008, Andrew Aksyonoff using config file '/etc/sphinx.conf'... indexing index 'documents'... collected 1147 docs, 11.9 MB sorted 2.3 Mhits, 100.0% done total 1147 docs, 11850011 bytes total 3.050 sec, 3884645.89 bytes/sec, 376.01 docs/sec indexing index 'forums'... collected 96647 docs, 110.2 MB collected 0 attr values sorted 0.1 Mvalues, 100.0% done sorted 18.0 Mhits, 100.0% done total 96647 docs, 110204194 bytes total 28.663 sec, 3844795.72 bytes/sec, 3371.81 docs/sec rotating indices: succesfully sent SIGHUP to searchd (pid=1781). Also, in order to make forums index the first time properly, I had to restart searchd. Don't know why..., anyway, might be a good idea note to review /usr/local/sphinx/index/searchd.log if things are not working. We might also want to have some log management on /usr/local/sphinx/index/query.log (in terms of it growing? I'm not sure what's built in to sphinx).
The actual switch will not happen during the 1.0 push, but somewhere around that time.
Target Milestone: 1.0 → ---
Depends on: 488429
Depends on: 488432
Depends on: 489044
Depends on: 489046
Depends on: 489379
Depends on: 489386
Depends on: 489414
Depends on: 489380
Depends on: 490056
Depends on: 490069
Finally done, as of Friday. Hallelujah!
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.