Closed Bug 464136 Opened 16 years ago Closed 16 years ago

Push SUMO 0.7.2 on Thursday November 13th

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

All
Other
task
Not set
normal

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: laura, Assigned: chizu)

References

Details

This will involve 
- doing an svn up
- changing the DocumentRoot
- installing Sphinx
- running some other database changes.

Downtime will be required.

I'll add more details as we go on.
Depends on: 463708
Flags: needs-downtime+
Assignee: server-ops → oremj
The tag is https://svn.mozilla.org/projects/sumo/tags/0.7/0.7.2_20081113_r19814/

The DocmentRoot needs to be changed to
[currentdir]/webroot

Trevor, can you please post instructions on installing the Sphinx packages you built for us?
Blocks: 464743
One thing that's going to be different about production versus staging is the app runs on multiple web-heads. Can sumo use one sphinx searchd for all the web-heads?
Assignee: oremj → thardcastle
Trevor: not sure, and I can't raise Nelson (who would know).  Let's try it with one.  None of the sphinx stuff is publicly visible in this release, so if that doesn't work it's no big deal.
Running on one means a single point of failure.  

I feel like we missed some step on making sure this is cluster aware.
Agreed.  (Although Nelson may be able to advise otherwise)

I don't think it should block release though, since the Sphinx stuff is not available on any public pages yet.  We're just trying to stage the various bits of search over the different releases.
I'd expect the best setup to be like what we do with memcached. Sphinx runs on a few of the web-heads, but not all of them.
If it's going to make you really uncomfortable, we can hold off on
the release until next week.
(In reply to comment #8)
> If it's going to make you really uncomfortable, we can hold off on
> the release until next week.

Uh, I'd rather not. This release fixes a ton of security vulns.
(In reply to comment #5)
> Running on one means a single point of failure.  

We only run one search thingy for www.mozilla.com, so it's not like we haven't done this before. ;)
Well the answer is that right now given my current configuration, the search will look for localhost port 3312, which means it needs to be running on all webheads to work consistently (BUT even if it does not work on production right now it does not matter in view of comment #6). Maybe it is simply wiser to not have on one right now - and then for us to test it using only that webhead (is there a way to force a request to use a particular webhead?) My thinking is that (in line with comment #6), we just want to make sure we are at a "working state" up to what we have so far, so as to spread things out over several releases.

Eventually though,  we should decide to run searchd on as many webheads depending on our performance/reliability requirements. I will have to make some changes to have correct logic (to load balance across the webheads) but no bug has been filed for this yet. See http://www.sphinxsearch.com/doc.html#distributed for long answer.
(In reply to comment #11)

> now it does not matter in view of comment #6). Maybe it is simply wiser to not
> have on one right now - and then for us to test it using only that webhead (is
> there a way to force a request to use a particular webhead?) My thinking is

Ugh, I mean "maybe it is simply wiser to simply have it installed on one right now - and then....
Code changes are pushed. There was a brief outage because of mismatched permissions on the new docroot. The site looks fine to me, but could use some more verification.

I'll deploy sphinx on one web head for now.
(In reply to comment #13)
> Code changes are pushed. There was a brief outage because of mismatched
> permissions on the new docroot. The site looks fine to me, but could use some
> more verification.
> 
> I'll deploy sphinx on one web head for now.

I've tested a bit; logged in, logged out, viewed my account, searched, etc., and it looks fine to me.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
Verified FIXED; deployed.
Status: RESOLVED → VERIFIED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.