1359585 - Run tecken systemtests just prior to deployment

Status: RESOLVED FIXED

(Socorro :: Symbols, task)

Description

[Peter Bengtsson [:peterbe]]

See https://bugzilla.mozilla.org/show_bug.cgi?id=1359216#c1

Comment 1

[Peter Bengtsson [:peterbe]]

Do you think you can incorporate this into the build/deployment automation?

When you run `docker-compose run systemtest tests/systemtest/run_tests.sh` locally, it uses docker to run this. And then it's able to re-use the Python environment inside the docker image. If you run that `tests/systemtest/run_tests.sh` outside, it might not work because you need to install a couple more dependencies. But I can code around that if that's how you run things. 

Basically, help me help you help me :)

Comment 2

[Miles Crabill [:miles]]

The systemtest runs in dev, properly sends an email notification with information about test failures, etc. When we move to running tecken in stage the systemtest will be run by Jenkins sshing into the boxes rather than on puppet cron.

Comment 3

[Peter Bengtsson [:peterbe]]

Is that still working and being used during deployments? On dev *and* stage?

For the record, running the systemtests locally is currently broken. It's docker related and something to do with the web app not being reachable at all.

Comment 4

[Miles Crabill [:miles]]

No, it doesn't happen. The systemtest is broken in those environments as well, I think for the same reason.

Comment 5

[Miles Crabill [:miles]]

This + logging are todo for tecken but non-blocking to load-testing.

Comment 6

[Peter Bengtsson [:peterbe]]

Pretty sure this is in place already. At least in Dev and Stage.