1386781 - [traceback] make dockersetup fails with '"build_type_enum" already exists'

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

g-k and claudijd are hitting errors when running "make dockersetup". That step in building a dev environment should set up postgres and then set up elasticsearch.

On the "set up postgres" step, it's dying with an error along these lines:

"""
$ ~/mozilla/socorro-zero/socorro> dc run webapp /app/docker/run_setup_postgres.sh
Starting socorro_rabbitmq_1 ... done
Starting socorro_elasticsearch_1 ... 
Starting socorro_postgresql_1 ... done
Starting socorro_localstack-s3_1 ... done
Setting up the db (breakpad)...
drop database breakpad [y/N]: y
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running stamp_revision  -> 15d9132d759e
2017-08-02 19:31:02,013 ERROR - setupdb -  - MainThread - Something went horribly wrong: (ProgrammingError) type "build_type_enum" already exists
 "CREATE TYPE build_type_enum\n    AS ENUM ('release', 'esr', 'aurora', 'beta', 'nightly');\nCREATE TYPE build_type\n    AS ENUM ('release', 'esr', 'aurora', 'beta', 'nightly');\n" {}
Traceback (most recent call last):
  File "./scripts/socorro", line 5, in <module>
    SocorroWelcomeApp.run()
  File "/app/socorro/app/socorro_app.py", line 181, in run
    values_source_list=values_source_list
  File "/app/socorro/app/socorro_app.py", line 278, in _do_run
    return_code = fix_exit_code(app_to_run.main())
  File "/app/socorro/app/socorro_app.py", line 530, in main
    return requested_app.main()
  File "/app/socorro/external/postgresql/setupdb_app.py", line 390, in main
    db.load_raw_sql('types')
  File "/app/socorro/external/postgresql/postgresqlalchemymanager.py", line 76, in load_raw_sql
    self.session.execute(raw_sql)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 991, in execute
    bind, close_with_result=True).execute(clause, params or {})
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
    context)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1159, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
    context)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (ProgrammingError) type "build_type_enum" already exists
 "CREATE TYPE build_type_enum\n    AS ENUM ('release', 'esr', 'aurora', 'beta', 'nightly');\nCREATE TYPE build_type\n    AS ENUM ('release', 'esr', 'aurora', 'beta', 'nightly');\n" {}
Setting up the db for Django...
"""

This bug covers figuring out what's going on here and fixing whatever else prevents people from building functional dev environments.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Grabbing this to work on now.

Comment 2

[Greg G]

OK I got something working on this branch https://github.com/g-k/socorro/tree/docker-debug

I haven't cleaned up and tried to run through these steps but on that branch something like:

1. create oauth creds on https://console.developers.google.com/apis/credentials with authorized js origin http://localhost:8000/ and callback url http://localhost:8000/oauth2/signin/
2. copy oauth client id and secret to docker/config/docker_common.env
3. make dockerstop; docker-compose rm
4. docker-compose run postgresql # to start postgres before running the webapp migrations so db setup script can connect to it
5. make dockersetup
6. make dockerrun
7. login to the webapp with google (which creates a user) and should say something like "you don't have permission to view annnything"
8. docker-compose run postgresql psql -h postgresql -U postgres breakpad
9. enter aPassword on prompt
10. UPDATE auth_user SET is_superuser = true;
11. with authed user browser open http://localhost:8000/admin/
12. create a product named Firefox version 1.0 
13. create a release for product: Firefox, Version: 1.0, recentish release data
14. http://localhost:8000/home/product/Firefox should load now with a db error saying there aren't crashes

Haven't tried submitting crashes.

Shouldn't need to, but might need to nuke all docker containers and images (affects other projects too) with:

docker container prune
docker system prune
make dockerbuild

I think the problem is that the first time the db starts up it's not listening by the time the db script runs causing the "unable to connect error" then the db setup script continues with the django migrations which make the build_type_enum (didn't confirm this). Subsequent make dockersetup runs use a running or recently stopped postgres image that start fast enough, but the build_type_enum already exists by that point.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

So I have a few thoughts on this.

First, I had forgotten about the oauth stuff. We shouldn't be editing the docker_common.env file, but instead have a "this is where I put variable overrides that are specific to my machine" env file or something like that. I think I thought about this, but it looks like if I did, I didn't document it anywhere. Tomorrow, I'll figure out if that's the case.

Second, I'm not sure what step 3 is about. That seems curious.

Third, I think you're right in that the problems you and claudijd hit were related to postgres not being ready and then everything getting all messed up. I fixed that in my PR (more in a bit).

Steps 8, 9, and 10 should be handled by "make dockersetup". It should create a superuser. I'm not sure why it's not doing that. I'll fix that tomorrow.

Step 12 and 13 are covered by the first comment I added to my PR. That should get handled by "make dockersetup", too. Basically, you should be able to do "make dockersetup" and then it sets everything up to have a basic functional Socorro system running on your local machine. I'll talk with Peter about this tomorrow and see what we can do.

My PR so far is here: https://github.com/mozilla-services/socorro/pull/3885

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

I created a tracker to covers improving the docker environment and setup process for a functional webapp and processor. That's bug #1387104.

I created bug #1387096 to cover improving setupdb_app so it's easier to use and reason about and create a superuser for the webapp.

I created bug #1387100 to cover adding required lookup table data (like the Firefox product) so the webapp and processor work.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 3885 - fix run_setup_postgres.sh issues

Comment 6

[[github robot]]

Commit pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/9655c07cb8d90683e86755eeb3f7ca41670b9e1c
fixes bug 1386781 - fix issues in run_setup_postgres.sh (#3885)

* bug 1386781 - fix issues in run_setup_postgres.sh

This moves the alembic_config to docker_common.env so it's available in both the
processor and webapp containers.

This adds a urlwait to docker/run_setup_postgres.sh to force it to wait for
postgres db to be ready for connections before trying to setup tables and run
migrations.

This moves the database name to a variable that exists already so it's not
hard-coded.

This removes the code in setupdb_app that prevents logging lines from happening
because that makes understanding what happened and debugging it much much
harder.

* Add -e and adjust run_setup_postgres.sh script

This changes the script so that it always drops the db and creates a new one.
Further, it will die if it errors out. It also updates the comments.

* Fix handling of "no, don't drop the db"

This fixes the logging to use the right method (.warning() vs. .warn()).

This also fixes the socorro script to return the non-zero exit code if there is
one as an exit code. That allows run_setup_postgres.sh to drop out.

Fixes the makefile dockersetup rule to stop if there's an error and updates the
comment for that rule which doesn't look right anymore.