dm-mailman01 has been a VM for a long time... it's our only mail-related box that isn't a real machine. I think we should move it to a blade, as it's having a hard time keeping up lately with its load, and it just seems like it should be a real machine anyway. A better CPU and more RAM would both be good help for it.
We could give it one of the old DL360 G4s, too... a blade might be a waste of resources/overkill. The thing to watch out for here is disk space. The mailing list archives for @mozilla.org lists are on the main drive on this machine, and we've had to grow the hard drive twice in the last 2 years to make room for their continued growth. If we put this on physical hardware, it'd probably be a good idea to move the mailman archives to networked storage so it can still be grown easily in the future.
chizu, can you get this lined up for Thursday's window?
If we're going to make tonight's window, we need an NFS or iSCSI share in place ASAP to have time to rsync the content over before the window.
shyam, something you can look at?
Sure, are we targetting this thursday?
If possible, sure.
The 360 in the URL would be a good host for this but there don't appear to be any drives in there. Needs an onsite visit.
That server should have a pair of 72 GB drives. They just need to be inserted in all the way.
Will swing by on the way in.
Drives are seated, chassis is connected but not powered on. Fire at will.
dm-mailman02 is up - where's the nfs (or iscsi?) volume you made for this?
Started an rsync from dm-mailmail01 -> dm-mailman02 (/usr/lib/mailman & /var/lib/mailman).
I'm not sure how you guys want to best schedule this. I've copied over all the data but it takes a long time to get the file list to pick up deltas.
Only "clean" way to do it is going to be to take an extended outage window. And if "du -sh" took 8 hours to calculate how much disk space it was using, I'm sure rsync will take at least that long to figure out what's changed. If we disable postfix on the box, the mail will get queued. It'll be offline long enough for people to get 4 hour delay notifications though, so it'll best be announced.
This is probably the kind of thing that would be good to do on a Sunday afternoon or a holiday. I'll quite likely be around on Monday, most of my holiday activity is planned for Saturday and Sunday, if you'd like to schedule this for late morning/early afternoon on Monday.
Timed rsync: real 344m36.094s user 3m29.479s sys 5m4.212s About 6 hours. Dave, you want to tackle this on Sunday/Monday? I'll send something out if you'll be able to handle this and do the actual switch.
Yeah, late morning Monday is probably good. I won't be around much on Sunday (during the day at least). 10am pacific is probably a good time for it. Schedule 8 just to be safe.
all done. There was a MESS of stuff we forgot about that I discovered as I was moving things... - multiple local user accounts (complete with accompanying cron jobs) for autoresponder scripts running procmail - configuration for htdig - nrpe config and check scripts - lots of perl modules for majordomo (which are all available via rpmforge these days) that had been installed via CPAN on the old box, and thus didn't show up in the package list. - apache configuration - cgi scripts and web apps related to the bugzilla.org website I'm *pretty* sure we got everything. I even copied over the ssh host keys just to be safe, since I ended up swapping the IPs on the two servers in lieu of having to screw with DNS and firewall.