1603594 - Install docker in the production environment

Status: RESOLVED FIXED

(Developer Services :: General, task)

Description

[u587052]

We want to improve the speed and maintainability of our end-to-end tests while maintaining a close similarity between the testing environment and production.

---

To prepare for 1598958, we need to have the production environment be able to run docker containers. This ticket involves the ansible configuration to install docker, as well as the actual deployment of that installation to our production machines.

To test the docker-installation ansible job, Connor mentioned that we might be able to spool up a temporary EC2 instance and point ansible at it.

Comment 1

[u587052]

I'm interested in handling this if nobody else already has their eyes on slurping this up

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

Looking into our ansible logic in version-control-tools, I've gone ahead and added geerlingguy.docker to our vendor-roles.
Theoretically, this could be as easy as adding geerlingguy.docker to the roles: section of docker-hgweb and docker-hgmaster. However, these machines are already associated with roles with "docker" in the name (docker-hg-web, docker-hg-ssh). This is confusing to me, because:

1. Due to the name, it sounds like these existing roles already install/require docker
2. When looking in the existing docker-... roles, I couldn't find any logic to install docker. So, they don't use docker yet?

Finally, I'd like to test the docker vendor role that I installed. Can I get access to our Amazon account so I can spool up an EC2 machine with the right AMI?

TL;DR, NI'd for two reasons:

1. Do the hg machines already have docker?
2. Can I get access to amazon?

Comment 3

[Connor Sheehan [:sheehan]]

(In reply to Mitchell Hentges [:mhentges] from comment #2)

Looking into our ansible logic in version-control-tools, I've gone ahead and added geerlingguy.docker to our vendor-roles.
Theoretically, this could be as easy as adding geerlingguy.docker to the roles: section of docker-hgweb and docker-hgmaster. However, these machines are already associated with roles with "docker" in the name (docker-hg-web, docker-hg-ssh). This is confusing to me, because:

1. Due to the name, it sounds like these existing roles already install/require docker
2. When looking in the existing docker-... roles, I couldn't find any logic to install docker. So, they don't use docker yet?

Finally, I'd like to test the docker vendor role that I installed. Can I get access to our Amazon account so I can spool up an EC2 machine with the right AMI?

TL;DR, NI'd for two reasons:

1. Do the hg machines already have docker?
2. Can I get access to amazon?

docker-* is the Ansible role applied to docker containers in the docker based test environment. When you build the docker based test images you basically apply these roles to the base Centos7 docker image. You'll notice in the meta that these roles apply the production role (ie docker-hg-web applies hg-web as part of it's dependencies). Any test-environment specific changes (adding test files, changing production config values to test values, etc) happen there as well. Hope that clears things up.

How difficult would it be to install docker in our own role? I would think it's rather straightforward (probably just yum install docker plus add a few users). Are there advantages to using the third-party role?

I'll set up AWS access for you tomorrow. :)

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

I saw that we had two other third-party roles (geerlingguy.java, geerlingguy.jenkins), so I cargo-culted the third in.
I don't think that it will be as easy as a yum install docker since there's a systemd service and group management involved, but it shouldn't be too spicy, either.

I'll write a minimal docker role tomorrow :+1:

(note: are the other third-party roles used? I couldn't find a usage, so I'll happily trash them in a separate patch)

Comment 5

[Connor Sheehan [:sheehan]]

(In reply to Mitchell Hentges [:mhentges] from comment #4)

(note: are the other third-party roles used? I couldn't find a usage, so I'll happily trash them in a separate patch)

Doesn't look like it - the annotate for ansible/vendor-roles/requirements.yml suggests they were added in the mozreview days and weren't torn out after.

Comment 6

[Connor Sheehan [:sheehan]]

Sent Mitch his credentials securely via Slack (FF-Send link).

Comment 7

[Mitchell Hentges [:mhentges] 🦀]

Attachment: ansible: adds docker for hg-web and hg-ssh (Bug 1603594) r?sheehan

Adds role to install docker community edition and enable the systemd service. I tested it against the CentOS 7 AMI.

The role was also added to be a dependency of hg-web and hg-ssh

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/d5ce3e62e9f3
ansible: adds docker for hg-web and hg-ssh r=sheehan

Comment 9

[Mitchell Hentges [:mhentges] 🦀]

Attachment: ansible: replace vct_dir stat check with "is_testing" variable (Bug 1603594) r?sheehan

It was previously implicit that, if "/vct" existed, we were running in a test environment.
By using a dedicated variable to represent this fact ("is_testing"), we can simplify the logic here

Depends on D58173

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/a966d39e1d34
ansible: replace vct_dir stat check with "is_testing" variable r=sheehan

Comment 11

[Mitchell Hentges [:mhentges] 🦀]

Attachment: ansible: name all testing-related roles/playbooks accordingly (Bug 1603594) r?sheehan

Some of our ansible roles are only used in local testing. We applied a convention to label these as docker-$thing,
but that is becoming confusing as we install Docker in production ("does this role apply to a place where we have
docker, or where we're running in docker, like our tests?").

By naming these as testing-$thing, we get two benefits:

1. It's easier to tell at a glance that a role/playbook is only for the testing environment
2. When attempting to use a role in production, you'll know if it's been written to assume a testing environment by its name

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/a8be0f5bf052
ansible: name all testing-related roles/playbooks accordingly r=sheehan