1321304 - ATMO V2: Fix docker/docker-compose setup to follow dockerflow, and/or fix dockerflow?

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)

Description

[Jannis Leidel [:jezdez]]

While working on bug 1311634 I noticed that I wasn't able to follow the dockerflow patterns for using an unprivileged user account.

Previously ATMO has used a Dockerfile (co-authored by cloudops) that used the user/group ID 1000 (which matches the default "ubuntu" user account in the used image).

When changing the user/group ID to 10001 as required by dockerflow the step in our continuous integration setup to collect static files stopped working with a generic Python error "OSError [Errno 13]: Permission denied".

To facilitate dev/prod parity we're using docker-compose to spin up a set of docker containers both in the development environment as well as on CI services Travis and Circle CI.

During that step the Django collectstatic management command tries to copy files to a directory called /app/static. That directory was not able to be written to.

I've done extensive research to find the cause of this issue, e.g. https://github.com/docker/docker/issues/1295, https://github.com/docker/docker/issues/7511#issuecomment-69261847 and was not able to fix the issue.

To reproduce the issue do the following using the bug1311634 git branch:

make test

That will run:

docker-compose run web python manage.py collectstatic --noinput
docker-compose run web pytest

On the first command it fails with the error described above.

What I noted is that building the "web" container with docker instead of docker-compose allowed me to run collectstatic without problems. So my current theory is that it has something to do with how docker-compose mounts the curren working directory as a volume in the docker container, which for some reason overwrites the permissions set in the image before.

I'd appreciate if someone would look at this since this is a blocker and frankly a pretty glaring omission in the dockerflow practices. I did check many other Mozilla Dockerfiles and was astounded how many were actually running the apps with root permissions.

Comment 1

[Jannis Leidel [:jezdez]]

BTW, https://github.com/mozilla/telemetry-analysis-service/pull/86 tries to do that more

Comment 2

[GitHub Autolander Bot]

Attachment: [telemetry-analysis-service] mozilla:bug1321304 > mozilla:master

Comment 3

[Jannis Leidel [:jezdez]]

Closing as this has been deployed and we're actively working on python-dockerflow to fill in the gabs for Python projects that dockerflow isn't specific enough.