1118294 - generate AMI for Socorro EC2 instances from CI

Status: RESOLVED FIXED

(Socorro :: Infra, task)

Description

[Robert Helmer [:rhelmer]]

We should create AMI(s) for use by Socorro. These can be based on an official base image (e.g. Ubuntu cloud AMI) and should be provisioned to be able to run Socorro and all required services that we'll run on EC2:

Collector, Processor, Crontabber, PostgreSQL, ES, etc.

We can generate separate AMIs for separate services if necessary, but for starters it's probably simplest to have a single AMI.

Comment 1

[Daniel Maher [:phrawzty]]

We already deploy to Red Hat so it's a known situation.  In the interest of reducing friction I think that we should stay the course for now.  The problem is that RH AMIs cost money, so we probably want to target CentOS for now since that leap is much smaller than going straight to Ubuntu.

> 08:11:16 < rhelmer> phrawzty: yeah let's just push forward with centos,
>                     will be super easy to swap in rhel if we change our minds
> 08:11:23 < phrawzty> done and done.

Comment 2

[Daniel Maher [:phrawzty]]

The working model: https://gist.github.com/phrawzty/97470ae758bddd9d4796

Comment 3

[Daniel Maher [:phrawzty]]

See https://github.com/mozilla/socorro-infra/pull/1

Comment 4

[Daniel Maher [:phrawzty]]

The Packer (AMI) profile for the default box is now available in the socorro-infra repo[0].

[0] https://github.com/mozilla/socorro-infra/tree/master/packer

Comment 5

[Daniel Maher [:phrawzty]]

08:44:32 < phrawzty> rhelmer: what would you consider the condition(s) for resolution to be ?
08:47:20 < rhelmer> phrawzty: so re: your question, I guess that when we make changes to the packer config in socorro-infra then a CI job builds and pushes a new AMI
08:48:07 < rhelmer> phrawzty: what I was typing before was trying to think through the mechanics of bootstrapping the buildbox ... it needs to build the env/consul RPMs that go into the packer box right? and we want the buildbox to use the packer-generated AMI, which this box is also building :)
08:49:08 < rhelmer> that should all be fine once we've run everything once
08:49:43 < rhelmer> phrawzty: I guess right now we can't easily trigger changes to only terraform or only packer since they're in the same repo though
08:51:25 < rhelmer> phrawzty: anyway I think when we push changes to socorro-infra that involve packer changes, packer should run in CI and generate a new AMI, and that resolves bug 1118294
08:51:55 < rhelmer> phrawzty: terraform picking up that new AMI and running "terraform apply" is a next step, but separate from that bug
08:53:29 < phrawzty> rhelmer: in this case, does "CI" refer to Travis, or to whatever we're running (Jenkins, DeadCI) ?
08:53:54 < rhelmer> phrawzty: I was thinking DeadCI on our buildbox would be the place

Comment 6

[Robert Helmer [:rhelmer]]

This happens on Jenkins for now.