Last Comment Bug 522923 - sumo / GLB deployment in Amsterdam/Singapore
: sumo / GLB deployment in Amsterdam/Singapore
Status: RESOLVED FIXED
10/22/2009 @ 9pm
:
Product: mozilla.org Graveyard
Classification: Graveyard
Component: Server Operations (show other bugs)
: other
: All Other
-- minor (vote)
: ---
Assigned To: Arzhel Younsi [:XioNoX]
: matthew zeier [:mrz]
:
Mentors:
Depends on: 524155
Blocks:
  Show dependency treegraph
 
Reported: 2009-10-17 20:19 PDT by matthew zeier [:mrz]
Modified: 2015-03-12 08:17 PDT (History)
9 users (show)
mzeier: needs‑downtime+
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description User image matthew zeier [:mrz] 2009-10-17 20:19:13 PDT
About ready to add sumo into GLB and serve from Amsterdam.

It's staged @ 63.245.213.88 & 63.245.213.89.  Would like some QA eyes on this.

Possible to target Tuesday (10/20) or Thursday (10/21)?
Comment 1 User image matthew zeier [:mrz] 2009-10-17 20:37:58 PDT
And staged @ 180.92.184.28 in Singapore for those in that region.
Comment 2 User image Shyam Mani [:fox2mike] (AFK until March 10) 2009-10-17 21:14:23 PDT
(In reply to comment #1)
> And staged @ 180.92.184.28 in Singapore for those in that region.

Works fine. I clicked around a bit, searches etc seem to fine :)
Comment 3 User image Stephen Donner [:stephend] 2009-10-19 10:01:03 PDT
mrz, Thursday would be better for me, if that's still doable.
Comment 4 User image Arzhel Younsi [:XioNoX] 2009-10-19 14:34:38 PDT
mrz, the GLB is ready for sumo, with HTTP and HTTPS checks on the 3 datacenters.
Comment 5 User image Stephen Donner [:stephend] 2009-10-19 22:12:01 PDT
I've been getting somewhat frequent "Service Unavailable" when on 63.245.213.89; can you guys check that host?  It could also be that it's with the load balancer, and I just gut unlucky -- I'll keep looking off and on tonight.
Comment 6 User image Pedro Alves 2009-10-20 13:50:39 PDT
As a side note, this won't impact metrics, as we collect data from db and not log parsing
Comment 7 User image matthew zeier [:mrz] 2009-10-20 14:04:13 PDT
daniel, does this mean we're okay to do this tonight from your end?
Comment 8 User image matthew zeier [:mrz] 2009-10-20 21:43:00 PDT
Pushing to Thursday.  Was unaware of support.mozilla.com's MX RRs.

Over to justdave for postfix massaging.
Comment 9 User image Dave Miller [:justdave] (justdave@bugzilla.org) 2009-10-20 22:55:19 PDT
CNAME's target domain added to postfix.
Comment 10 User image Dave Miller [:justdave] (justdave@bugzilla.org) 2009-10-21 01:09:18 PDT
I reverted the DNS changes, looks like we screwed outgoing mail, too (see bug 523533)
Comment 11 User image Dave Miller [:justdave] (justdave@bugzilla.org) 2009-10-21 01:11:22 PDT
The correct fix:

1) remove all records for support.mozilla.com from the mozilla.com zone.
2) add NS records for support.mozilla.com in the mozilla.com zone pointing at the glb.mozilla.net NS servers
3) tell the GLB servers to handle the entire support.mozilla.com zone
4) set up SOA and MX records in the support.mozilla.com zone on the GLB servers
Comment 12 User image matthew zeier [:mrz] 2009-10-21 08:57:55 PDT
(In reply to comment #11)
> The correct fix:
> 
> 1) remove all records for support.mozilla.com from the mozilla.com zone.
> 2) add NS records for support.mozilla.com in the mozilla.com zone pointing at
> the glb.mozilla.net NS servers
> 3) tell the GLB servers to handle the entire support.mozilla.com zone
> 4) set up SOA and MX records in the support.mozilla.com zone on the GLB servers

This got more complicated.  

Over to Arzhel to stage.
Comment 13 User image Dave Miller [:justdave] (justdave@bugzilla.org) 2009-10-22 19:42:00 PDT
This got pushed about 40 minutes ago, DNS Trace looks good.
http://pastebin.mozilla.org/678392

Note You need to log in before you can comment on or make changes to this bug.