924716 - two new aws instances like vcssync-dev

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Aki Sasaki (not active)]

These will be production instances, so we may want to do more:

* inventory?
* dns?
* nagios?
* puppet?

Otherwise I can set these up by hand like I did vcssync-dev, and document.


mkfs.ext4 /dev/xvdj
/etc/fstab: /dev/xvdj  /opt      ext4    defaults,noatime 1 2
Block size=4096 (log=2)
6553600 inodes, 26214400 blocks

newuser vcs2vcs

yum install sudo
yum install git
yum install screen
yum install gcc
yum remove ssmtp
yum install postfix  # edit main.cf
/etc/init.d/postfix start
installed https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz

Comment 1

[Rail Aliiev [:rail]]

Attachment: puppet

Comment 2

[Rail Aliiev [:rail]]

DNS:

invtool A create --ip 10.132.48.15 --fqdn vcssync1.srv.releng.usw2.mozilla.com --private  --description "bug 924716: two new aws instances like vcssync-dev"
invtool PTR create --ip 10.132.48.15 --target vcssync1.srv.releng.usw2.mozilla.com --private --description "bug 924716: two new aws instances like vcssync-dev"
invtool  A create --ip 10.132.49.15 --fqdn vcssync2.srv.releng.usw2.mozilla.com  --private  --description "bug 924716: two new aws instances like  vcssync-dev"
invtool  PTR create --ip 10.132.49.15 --target  vcssync2.srv.releng.usw2.mozilla.com --private --description "bug  924716: two new aws instances like vcssync-dev"

Comment 3

[Rail Aliiev [:rail]]

Comment on attachment 815568 [details] [diff] [review]
puppet

https://hg.mozilla.org/build/puppet/rev/f2db5877faf0

Comment 4

[Rail Aliiev [:rail]]

python aws_create_instance.py -c configs/vcssync-linux64 -r us-west-2 -i us-west-2.instance_data_tests.json -s aws-releng -k secrets/aws-secrets.json vcssync{1-2}

$ ssh [e-mail]
$ sudo -s
# mkfs.ext4 /dev/xvdj
# echo "/dev/xvdj  /opt      ext4    defaults,noatime 1 2" >> /etc/fstab
# mount -a
# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvde1            9.9G  1.1G  8.3G  12% /
none                  7.4G     0  7.4G   0% /dev/shm
/dev/xvdj              99G  188M   94G   1% /opt

# yum install git gcc # screen and sudo are already installed, no need to adjust postfix

### to trick puppet needed to use uid/gid < 500
# groupadd --system vcs2vcs
# useradd --system -g vcs2vcs vcs2vcs
# cp -a /etc/skel /home/vcs2vcs                                                                    
# chown -R vcs2vcs: /home/vcs2vcs

Comment 5

[Aki Sasaki (not active)]

So far:

sudo mkdir /opt/vcs2vcs
sudo chown vcs2vcs /opt/vcs2vcs
scp'ed ssh keys over to ~vcs2vcs/.ssh/
installed https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
cloned https://github.com/escapewindow/mozharness.git and checked out the vcs_multi_repo branch # only until bug 847727 is reviewed+landed
yum install python-devel

scp'ed initial3.tar.bz2 up (initial cvs history stub)
# as vcs2vcs
mkdir -p /opt/vcs2vcs/build; cd /opt/vcs2vcs/build; tar xjvf /opt/vcs2vcs/initial3.tar.bz2
# on vcssync2 we'll need to move the conversion/beagle dir to conversion/project-branches


# I'll need to loop this via cron with the run_beagle.sh script later, but I'd like to see it run once successfully first
python mozharness/scripts/vcs-sync/vcs_sync.py -c mozharness/configs/vcs_sync/beagle.py

Comment 6

[Rail Aliiev [:rail]]

Back to Aki to figure out what's left here.

Comment 7

[Aki Sasaki (not active)]

I think we're done.

Rail: do you know why I can't ssh passwordless into vcs2vcs?
Same authorized_keys changes, I can ssh into asasaki@vcssync{1,2} but can't to vcs2vcs@vcssync{1,2} (asks for password).

I was able to work around this (scp stuff into my account, then sudo mv it into vcs2vcs dirs).

Comment 8

[Rail Aliiev [:rail]]

sshd was refusing to authenticate a user "locked" (no password set) in /etc/shadow
I copied /root/.ssh/authorized_keys to /home/vcs2vcs/.ssh, chowned it and ran passwd to set password (the same with cltbld). It worked for me.

Comment 9

[Aki Sasaki (not active)]

sudo yum install mercurial

# so we can pull hg.m.o/build/mozharness production, rather than github.com/escapewindow/mozharness vcs_multi_repo

Thanks Rail!