576667 - Need to set up Sphinx Server and push Fx Input staging to live on 7/5

Status: VERIFIED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Aakash Desai [:aakashd]]

We've learned that our current search engine doesn't scale well (thanks to users grabbing our Fx4 Beta 1 candidate builds and sending us a ton of feedback). We need a push before Fx4 Beta 1 as there's going to be a lot of people using this App at the Summit and elsewhere. Dave Dash is currently implementing Sphinx onto our Input App and we want to get a push out on Monday night (or after he's completed the code changes).

Stuff to do from my knowledge:

- set up the sphinx indexing on the sphinx cluster
- get the sphinx server running on the sphinx cluster
- point the input app code at it
- ``git pull origin master`` from input.stage.mozilla.com  
- Input needs an apache reload after it goes live 

I think that's enough, but I'll defer to Frederic and Dave here. Will someone be avilable to get this done?

Comment 1

[Justin Dow [:jabba]]

Dropping the priority to normal for now, so that it doesn't page on-call. Please bump it back up to critical once we have the code and then the on-call person will get paged about it and can take care of.

Comment 2

[Fred Wenzel [:wenzel]]

Raising this back to critical. The code using Sphinx has landed on trunk, and presumably input has auto-pulled it in the mean time.

We are still ironing out some things, so no push to production yet. Still, on staging we need the following:
- a running sphinx server, whether you hook us up to an existing one or it's a new local instance
- if the SPHINX_* settings in settings.py are not correct, copy them to settings_local.py and change them.
- make sure that ``./manage.py update_index -r`` works by running it manually. Then update the cron job (it did not include "-r" before). If Sphinx is not local, consult with Dave Dash what cron job to set up instead.

Comment 3

[Aravind Gottipati [:aravind]]

What website is this for?  It appears that we are re-using the existing sphinx cluster to index multiple sites production.  I would like to reuse the same setup if this is for a new site.

Comment 4

[Aakash Desai [:aakashd]]

This is for http://input.mozilla.com and http://input.stage.mozilla.com

Comment 5

[Aravind Gottipati [:aravind]]

Where do I get the sphinx configs from, I tried looking under http://svn.mozilla.org/projects/, but I didn't see an input directory there.  Is this checked into a git repo somewhere else?

Comment 6

[Aravind Gottipati [:aravind]]

(In reply to comment #5)
> Where do I get the sphinx configs from, I tried looking under
> http://svn.mozilla.org/projects/, but I didn't see an input directory there. 
> Is this checked into a git repo somewhere else?

Nevermind, I see the staging repo is checked out from git://github.com/fwenzel/reporter.git

Will look for the configs there.

Comment 7

[Fred Wenzel [:wenzel]]

Yes, here: http://github.com/fwenzel/reporter/tree/master/configs/sphinx/

Sorry, if I had worked with sphinx before I'd probably have given you those straight away ;)

Comment 8

[Aravind Gottipati [:aravind]]

Okay, staging should be set, can we get some eyes on it and test it out?

Comment 9

[Aakash Desai [:aakashd]]

On it!

Comment 10

[Aakash Desai [:aakashd]]

Alright, everything looks good for a push to production.

Comment 11

[Aravind Gottipati [:aravind]]

Prod should be good to go now.

Comment 12

[Aakash Desai [:aakashd]]

verified FIXED