1567285 - Improve the need for |docker-compose run backend bash|

Status: RESOLVED WONTFIX

(Tree Management :: Treeherder, enhancement, P2)

Description

[Armen [:armenzg]]

We use |docker-compose run backend bash| for a lot of things. From running tests to running scripts that interact with our local TH instance.

In certain cases, too many containers end up getting Docker into an odd state where the containers start freezing up.

I'm considering investigating an image that is only used for running tests while another image OR simply a pipenv/poetry set up for normal development. The latter will also help to start Visual Studio with a proper Python environment.

Would this help you as well?

Comment 1

[Sarah Clements [:sclements]]

In certain cases, too many containers end up getting Docker into an odd state where the containers start freezing up.

Luckily I haven't run into this issue. I'm curious what circumstances have triggered this.

I'm considering investigating an image that is only used for running tests while another image OR simply a pipenv/poetry set up for normal development.

Would it be more confusing or tedious to have different images for different tasks and how would it work to switch between them? What would be the benefit/difference of a pipenv/poetry set up?

The latter will also help to start Visual Studio with a proper Python environment.

Visual Studio Code text editor or Visual Studio IDE? By this do you mean that it would have proper remote debugging set up? That is definitely something I'd like but it'd be great to have it set up for use with docker too.

Comment 2

[Armen [:armenzg]]

(In reply to Sarah Clements [:sclements] from comment #1)

In certain cases, too many containers end up getting Docker into an odd state where the containers start freezing up.

Luckily I haven't run into this issue. I'm curious what circumstances have triggered this.

To ingest data I need to have this set up:
https://treeherder.readthedocs.io/pulseload.html

After multiple stops and starts of the ingestion scripts the containers get stuck. I need to close all tabs and see if there's anything else that needs killing. This does not happen often, however, it happens once in a while.

I'm considering investigating an image that is only used for running tests while another image OR simply a pipenv/poetry set up for normal development.

Would it be more confusing or tedious to have different images for different tasks and how would it work to switch between them?

You would not need to deal with the testing image unless you want to reproduce the testing environment that Travis uses.
Both the dev image and the test image will have a shared based.

What would be the benefit/difference of a pipenv/poetry set up?

You could run etl commands within a poetry shell rather than having to start multiple shells with docker-compose.
That way you can just have |docker-compose up| with all services running and run commands from |poetry run <whatever_command_you_want>|.

The latter will also help to start Visual Studio with a proper Python environment.

Visual Studio Code text editor or Visual Studio IDE? By this do you mean that it would have proper remote debugging set up? That is definitely something I'd like but it'd be great to have it set up for use with docker too.

The IDE.

Are you referring to this?
https://docs.microsoft.com/en-us/visualstudio/debugger/remote-debugging?view=vs-2019

Or setting Python breaks within the IDE?
https://docs.microsoft.com/en-us/visualstudio/python/debugging-python-in-visual-studio?view=vs-2019

If the latter, I assume so since both the scripts (poetry run <script and args>) and the IDE (poetry run code .) will be started within the same virtual env. Notice that we would be able to get in-editor linting and possibly in-editor test running.

Comment 3

[Armen [:armenzg]]

This has not been an issue.