Last week, we discussed intentions and end goals with this migration: see the Friday notes in [this doc](https://docs.google.com/document/d/1Zkghx-mbCldn-Ng6kYsCLPdzAdEjLm4kJCQAPtTj9j4). TL;DR: We want to put each of our hg server toolchains into their own independent docker images. This should ease incremental building (only one image should need to be updated when a change occurs), will remain relatively identical to production, and should be possible without requiring "host networking" in docker (linux-only). ----- There will be the following docker images and containers: * (HG SSH) `pash` (python 2) * Runs Pash (tool to absorb SSH connections and only serve mercurial-related requests) * (HG SSH) `venv_bundles` (python 3) * Generates HG bundles and throws them on S3 * (HG SSH) `tools` (python 2) * Aggregator replicator daemon (push data aggregator) (push data aggregator-pending) * Notifier daemons * Unified Process * Heartbeat * (HG Web) `hgweb` (python 2) * UWSGI and hg.m.o web application * (HG Web) `tools3` (python 3) * IP Scaper (x3) * (HG Web) `replication` (python 2) * Replicate (x8) head (x1) * (HG Web) `tools2` (python 2) * FF Scraper * (HG Web) Apache (not python) * (both) Kafka (not python) ----- Work we need to do: 1. Confirm with "enterprise security" that this is a good path forward [will link bug once created] 2. Migrate custom deployment/testing scripts to use `docker-compose` [will link bug once created] 3. Migrate ansible setup to individual docker images and containers as shown ^ [will link bug once created]
Bug 1598958 Comment 5 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Last week, we discussed intentions and end goals with this migration: see the Friday notes in [this doc](https://docs.google.com/document/d/1Zkghx-mbCldn-Ng6kYsCLPdzAdEjLm4kJCQAPtTj9j4). TL;DR: We want to put each of our hg server toolchains into their own independent docker images. This should ease incremental building (only one image should need to be updated when a change occurs), will remain relatively identical to production, and should be possible without requiring "host networking" in docker (linux-only). ----- There will be the following docker images and containers: * (HG SSH) `pash` (python 2) * Runs Pash (tool to absorb SSH connections and only serve mercurial-related requests) * (HG SSH) `venv_bundles` (python 3) * Generates HG bundles and throws them on S3 * (HG SSH) `tools` (python 2) * Aggregator replicator daemon (push data aggregator) (push data aggregator-pending) * Notifier daemons * Unified Process * Heartbeat * (HG Web) `hgweb` (python 2) * UWSGI and hg.m.o web application * (HG Web) `tools3` (python 3) * IP Scaper (x3) * (HG Web) `replication` (python 2) * Replicate (x8) head (x1) * (HG Web) `tools2` (python 2) * FF Scraper * (HG Web) Apache (not python) * (both) Kafka (not python) ----- Work we need to do: 1. Confirm with "enterprise security" that this is a good path forward [will link bug once created] 2. [Migrate custom deployment/testing scripts to use `docker-compose`](https://bugzilla.mozilla.org/show_bug.cgi?id=1600827) 3. Migrate ansible setup to individual docker images and containers as shown ^ [will link bug once created]
Last week, we discussed intentions and end goals with this migration: see the Friday notes in [this doc](https://docs.google.com/document/d/1Zkghx-mbCldn-Ng6kYsCLPdzAdEjLm4kJCQAPtTj9j4). TL;DR: We want to put each of our hg server toolchains into their own independent docker images. This should ease incremental building (only one image should need to be updated when a change occurs), will remain relatively identical to production, and should be possible without requiring "host networking" in docker (linux-only). ----- There will be the following docker images and containers: * (HG SSH) `pash` (python 2) * Runs Pash (tool to absorb SSH connections and only serve mercurial-related requests) * (HG SSH) `venv_bundles` (python 3) * Generates HG bundles and throws them on S3 * (HG SSH) `tools` (python 2) * Aggregator replicator daemon (push data aggregator) (push data aggregator-pending) * Notifier daemons * Unified Process * Heartbeat * (HG Web) `hgweb` (python 2) * UWSGI and hg.m.o web application * (HG Web) `tools3` (python 3) * IP Scaper (x3) * (HG Web) `replication` (python 2) * Replicate (x8) head (x1) * (HG Web) `tools2` (python 2) * FF Scraper * (HG Web) Apache (not python) * (both) Kafka (not python) ----- Work we need to do: 1. [Confirm with "enterprise security" that this is a good path forward](https://bugzilla.mozilla.org/show_bug.cgi?id=1600828) 2. [Migrate custom deployment/testing scripts to use `docker-compose`](https://bugzilla.mozilla.org/show_bug.cgi?id=1600827) 3. Migrate ansible setup to individual docker images and containers as shown ^ [will link bug once created]
Last week, we discussed intentions and end goals with this migration: see the Friday notes in [this doc](https://docs.google.com/document/d/1Zkghx-mbCldn-Ng6kYsCLPdzAdEjLm4kJCQAPtTj9j4). TL;DR: We want to put each of our hg server toolchains into their own independent docker images. This should ease incremental building (only one image should need to be updated when a change occurs), will remain relatively identical to production, and should be possible without requiring "host networking" in docker (linux-only). ----- There will be the following docker images and containers: * (HG SSH) `pash` (python 2) * Runs Pash (tool to absorb SSH connections and only serve mercurial-related requests) * (HG SSH) `venv_bundles` (python 3) * Generates HG bundles and throws them on S3 * (HG SSH) `tools` (python 2) * Aggregator replicator daemon (push data aggregator) (push data aggregator-pending) * Notifier daemons * Unified Process * Heartbeat * (HG Web) `hgweb` (python 2) * UWSGI and hg.m.o web application * (HG Web) `tools3` (python 3) * IP Scaper (x3) * (HG Web) `replication` (python 2) * Replicate (x8) head (x1) * (HG Web) `tools2` (python 2) * FF Scraper * (HG Web) Apache (not python) * (both) Kafka (not python) ----- Work we need to do: 1. [Confirm with "enterprise security" that this is a good path forward](https://bugzilla.mozilla.org/show_bug.cgi?id=1600828) 2. [Migrate custom deployment/testing scripts to use `docker-compose`](https://bugzilla.mozilla.org/show_bug.cgi?id=1600827) 3. [Migrate ansible setup to individual docker images and containers as shown ^](https://bugzilla.mozilla.org/show_bug.cgi?id=1600834)