Closed Bug 1171057 Opened 11 years ago Closed 11 years ago

Docker project should mount source in current directory, not clone from Github

Categories

(bugzilla.mozilla.org :: Developer Box, defect)

Product:
bugzilla.mozilla.org
Component:
Developer Box
Version:
Production
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1174967

People

(Reporter: emorley, Unassigned)

Details

1) Clone the BMO repo 2) Follow the docker/boot2docker instructions in contrib/docker/README.md 3) Modify a file in the host source dir Expected: The running docker image reflects the changes made to the source dir. Actual: The running docker image cloned its own src dir which is only inside the VM and not accessible from the host to modify. IMO fig.yml should be in the root of the source repo (and point to the files in contrib/docker/), so (a) someone can run a |fig build| from the root of the BMO repo, and (b) we can easily mount that directory in the app. I'm going to take a quick shot at this. (Which I'm probably going to end up regretting, like every other time I've touched Docker whilst using Windows... but oh well!).
(In reply to Ed Morley [:emorley] from comment #0) > IMO fig.yml should be in the root of the source repo (and point to the files > in contrib/docker/), so (a) someone can run a |fig build| from the root of > the BMO repo, and (b) we can easily mount that directory in the app. That's an interesting idea. I will take a look at moving the fig.yml file to the document root specifically for BMO. Then in fig.yml: volumes: - '.:/home/bugzilla/devel/htdocs/bmo' dkl
One caveat would be you would need to run checksetup.pl before anything would work. checksetup.pl is originally ran as part of the image build process and fig.yml will be mapping the local git clone over the images git clone. Could just document this I suppose and people will just need to do it once. Thought? dkl
We shouldn't be cloning the repo inside the dockerfile at all IMO. There should be a way to do this (perhaps we need to use ONBUILD?) I just need to do more reading. Noting these before I forget: (many of these are from http://docs.docker.com/articles/dockerfile_best-practices/) * We should switch to using docker-compose rather than fig, since it now replaces it * We should use one container per service (rather than everything in one) - we can just use the mysql image, this will drastically simplify the DB parts * We should use WORKDIR rather than continually 'cd'ing * We should use USER rather than 'su' * COPY is preferred over ADD * The 'RUN wget' line shouldn't be on it's own line, since it causes additional layers and increases image size * The yum update line shouldn't be on it's own line, since otherwise the cache won't invalidate when it ought to
Also it sounds like we should be using nsenter (if we even need to get inside the container) rather than sshd: https://jpetazzo.github.io/2014/06/23/docker-ssh-considered-evil/
(In reply to Ed Morley [:emorley] from comment #4) > Also it sounds like we should be using nsenter (if we even need to get > inside the container) rather than sshd: > https://jpetazzo.github.io/2014/06/23/docker-ssh-considered-evil/ Or exec/attach, as of docker 1.3+ http://askubuntu.com/questions/505506/docker-how-to-get-bash-ssh-inside-runned-container-run-d/543057#543057
Unfortunately the design of checksetup.pl and the way requirements are defined is making this quite hard to do 'the Docker way' without always resulting in cache invalidation. In say a Python project, the Dockerfile would do something like: COPY requirements.txt /tmp RUN pip install -r /tmp/requirements.txt COPY . /app ...that way so long as requirements.txt hasn't changed, the |RUN pip install| step can be used from the cached image, since it's not invalidated - even when other files change in the source directory. This makes rebuilds extremely fast, and is one of the core principals behind Docker images. Unfortunately with Bugzilla, even if I change the Dockerfile to use the source repo the current directory (vs cloning), we still have to do something like: COPY . /app WORKDIR /app RUN ./checksetup.pl --cpanfile && $CPANM ... Since |checksetup.pl --cpanfile| uses Bugzilla/Install/Requirements.pm, which needs other modules in turn - so we can't just move a single requirements file into the image. As such, changes to any source files (even if Requirements.pm was unchanged) invalidates the whole step, so we have to re-run checksetup and re-install the CPAN deps again (which won't be installed, since they were only present in that layer and onwards). As such, I think our options are: 1) Give up trying to run checksetup.pl as part of the build - and just do so on firstrun (either manually or using ENTRYPOINT and a script to tell if it's been run) 2) Try to refactor Requirements.pm so its more standalone (ha) 3) See if we could be really hacky and just COPY checksetup.pl, Requirements.pm and any of the deps of it - just so we can have a more stable layer that does the CPAN installs. Also another consideration, is that if we switch to using multiple containers (which is one of the main principals of using docker - currently we just use one container so docker compose isn't adding any value) then we definitely cannot run the DB parts of checksetup.pl during the build - since they'd touch the DB container, which wouldn't yet be running (build and run are separate steps).
All good points and for full disclosure, when I originally wrote the Dockerfile I was still learning the basics and there is much room for improvement. I have had plans to do most of the things you have mentioned once I get the full testing framework working in Treeherder/Taskcluster. Another idea I was thinking of was moving the whole system setup (apache, mysql, cpan) to Ansible configuration files and then just have the bash script to do the final setup (git clone, checksetup.pl, etc.) This way the same configuration scripts could be used outside of Docker such as Vagrant, bare metal, etc. The issue with running multiple containers is that TaskCluster does not support that method of test jobs yet. There is talk of supporting it in the future but right now I am limited to everything running in a single container. I actually have a github repo for doing Bugzilla in multiple containers now (https://github.com/dklawren/docker-bugzilla-fig). It works but container startup time is much slower, as you can imagine, each time you want to spin up a new one just to try some small change. I am definitely interested in refactoring the scripts to look to see if it using the git clone in the image and update it or if a Bugzilla tree is mounted from the host, then run checksetup.pl to get it working properly. dkl
(In reply to David Lawrence [:dkl] from comment #7) > All good points and for full disclosure, when I originally wrote the > Dockerfile I was still learning the basics and there is much room for > improvement. I have had plans to do most of the things you have mentioned > once I get the full testing framework working in Treeherder/Taskcluster. Yeah it's taking me a while to construct a mental model of how everything is supposed to fit together with Docker. > Another idea I was thinking of was moving the whole system setup (apache, > mysql, cpan) to Ansible configuration files and then just have the bash > script to do the final setup (git clone, checksetup.pl, etc.) This way the > same configuration scripts could be used outside of Docker such as Vagrant, > bare metal, etc. I like the avoiding-duplication part of that - the only problem is that the more steps that are combined in one Docker RUN step, the more gets invalidated when one of the pre-requisites changes. Also, we can reduce the amount of setup by using the official Docker images for things, rather than always starting with the base image. eg the DB Dockerfile can be as simple as: https://hg.mozilla.org/hgcustom/version-control-tools/file/8e8e4b728130/testing/docker/builder-bmodb/Dockerfile > The issue with running multiple containers is that TaskCluster does not > support that method of test jobs yet. Ah I didn't know. That's pretty limiting :-( Perhaps we just need two versions - one for local development and one for taskcluster? > It works but container startup time is much slower, as you can imagine, > each time you want to spin up a new one just to try some small change. Starting up the app shouldn't take too much longer if the container already exists; or is it rebuilding that's slow? I suspect if we use an official DB image many of these problems would go away (I'm presuming they've optimised for startup time etc). > I am definitely interested in refactoring the scripts to look to see if it > using the git clone in the image and update it or if a Bugzilla tree is > mounted from the host, then run checksetup.pl to get it working properly. Great - I'll dig into this more over the next few days - learning about Docker is useful for the Treeherder bug 1169263 anyway.
(In reply to David Lawrence [:dkl] from comment #7) > This way the same configuration scripts could be used outside of Docker > such as Vagrant, bare metal, etc. I'm not sure how long this has been the case - but Vagrant supports using Docker as a provisioner. Also Docker is "bare metal" IMO, since if you're on Linux, you don't need the Boot2Docker layer inbetween - and if you're not on Linux, then the Ansible configuration scripts wouldn't help people anyway. As such, perhaps the Dockerfile is already a suitable setup script for using via several means? :-)
Docker Compose 1.3.0 (about to be released) adds support for specifying the Dockerfile and build context separately in docker-compose.yml. You can also tell Docker Compose to use an alternative yml file. It also supposedly improves perf for multi-containers. As such, we could do something like: docker-compose.yml docker-compose-taskcluster.yml contrib/docker/<all the shared files> contrib/docker/dev/Dockerfile contrib/docker/taskcluster/Dockerfile Then for new contributors / someone wanting a quick slim Docker environment, they'd do: ~/bmo$ docker-compose build For taskcluster: ~/bmo$ docker-compose build -f docker-compose-taskcluster.yml In the dev environment we can use an official DB image in a separate container, to save rolling our own. It would also be good to consolidate some of the docker projects we have (I know some are for upstream, but there are now lots of diverging implementations): https://github.com/mozilla/webtools-bmo-bugzilla/tree/master/contrib/docker https://github.com/dklawren/docker-bugzilla https://github.com/dklawren/docker-bugzilla-bmo https://github.com/dklawren/docker-bugzilla-fig
An issue I'm hitting at the moment is that Elasticsearch is not available on MetaCPAN: ==> Found dependencies: ElasticSearch, Apache2::SizeLimit Searching ElasticSearch on cpanmetadb ... -> FAIL Finding ElasticSearch on cpanmetadb failed. Searching ElasticSearch (0) on metacpan ... ! Could not find a release matching ElasticSearch (0) on MetaCPAN. Searching ElasticSearch on mirror http://www.cpan.org ... Downloading index file http://www.cpan.org/modules/02packages.details.txt.gz ... -> FAIL Finding ElasticSearch (0) on mirror http://www.cpan.org failed. -> FAIL Couldn't find module or a distribution ElasticSearch (0) I can't see how the current contrib directory docker config is working around this? (or does the error just get ignored and it causes no problems?) The version-control-tools repo Dockerfile patches Elasticsearch out of Requirements.pm, but that seems suboptimal. I guess I can't even do a |cpanm --without-out feature FOO| since that requirement isn't annotated with a feature name: https://github.com/mozilla/webtools-bmo-bugzilla/blob/014dc8595070533eb7a14771e98640f74037ff78/Bugzilla/Install/Requirements.pm#L393-L398 David, what do you think the best way around this is? (I guess this is partly a "should a lean dev environment support every extension in BMO, or should we have a more fully featured dockerfile that adds support for the edge-case features" decision).
Flags: needinfo?(dkl)
Depends on: 1172386
(In reply to Ed Morley [:emorley] from comment #12) > (or does the error just get ignored and it causes no problems?) my guess is this is what's happening. ES is currently only used for capturing detailed timing instrumentation and is disabled by default (and is almost always disabled in production too). > I guess I can't even do a |cpanm --without-out feature FOO| since that > requirement isn't annotated with a feature name: > https://github.com/mozilla/webtools-bmo-bugzilla/blob/ > 014dc8595070533eb7a14771e98640f74037ff78/Bugzilla/Install/Requirements. > pm#L393-L398 i vote for fixing it via feature tagging and --without-feature : bug 1172386
Committed bug 1172386, so like glob mentioned, we can just use --without-feature elasticsearch now for docker builds.
Flags: needinfo?(dkl)
No longer depends on: 1172386
I've started with the fresh bug 1174967, since the scope has now increased considerably, and I wanted a comment 0 that matched reality / a clean start.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.