951449 - Integration tests within Unittests confuse Jenkins

Status: VERIFIED FIXED

(Socorro :: General, task)

Description

[K Lars Lohn [:lars] [:klohn]]

right now we've got a bunch of integration tests in the unittest directory.  When run locally, they happily connect to localhost for the database, rabbit, etc.  However, when they're run in Jenkins, they have no way to know that localhost is not the right place to get access to Postgres, Rabbit, etc.

Devise a way to get the connection parameters appropriate for Jenkins into the Integration tests without spoiling it for running those same tests locally.

The solution: as with all problems, Configman solves everything.  The upcoming changes to Configman 1.1.15 enable an apps configuration to have a "resource branch".  Since the resource branch looks the same for _all_ socorro apps, one standard set of command line arguments or environment variables will work for any socorro app.  For example, if you want to set the postgres host name, "--resource.postgres.hostname=whatever" will work for all socorro apps.  

In build.sh, running the make file is augmented by injecting some variables into the Makefile at run time.  If we expand that to giving all the connection parameters for all the required resources, PG, ES, RMQ, etc, we can exploit those values within the Makefile.  If the connection parameters are invoked with make, the nose tests are invoked with those connection parameters in the environment styled in the manner of the Configman resource branch options.  Configman will grab those values and feed them to the Integration tests that are running under nosetests.  The Integration tests will then connect to the appropriate resources within the Jenkins environment.

If make is run with no externally injected parameters, then the Makefile does not make the environment variables and nosetests runs with the Integration tests running with their defaults.  The Integration tests will connect with the appropriate localhost resources.

This change depends on Configman 1.1.15 which is coming in PR 1737 and PR 1741

Comment 1

[K Lars Lohn [:lars] [:klohn]]

Attachment: PR to add environment vars to Makefile

Comment 2

[K Lars Lohn [:lars] [:klohn]]

adrian suggests that rhelmer and lonnen take a look at the PR, too.

Comment 3

[Lonnen :lonnen]

I looked. The bash portions are fine for now.

Comment 4

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/e09c8e55cbe3f005a1390e294e7b64bd1a3e0942
Merge pull request #1750 from twobraids/make_environment

fixes Bug 951449 (v71) -  Integration tests within Unittests confuse Jenkins

Comment 5

[Matt Brandt [:mbrandt]]

Bumping to verified as [qa-]