938927 - bugzilla_simple_shim dies after a little while

Status: RESOLVED FIXED

(Webtools :: Pulse, defect)

Description

[Mark Côté [:mcote]]

Running the bugzilla_simple_shim on my bmo box with a period of 1 second, and it dies after a few minutes:

Traceback (most recent call last):
  File "bugzilla_simple_shim.py", line 149, in <module>
    rc = main(args[0], options.repeat, options.verbose)
  File "bugzilla_simple_shim.py", line 122, in main
    shim.update()
  File "bugzilla_simple_shim.py", line 46, in update
    table = getattr(db, self.db_table)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlsoup.py", line 455, in __getattr__
    return self.entity(attr)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlsoup.py", line 452, in entity
    return self.map_to(attr, tablename=attr, schema=schema)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlsoup.py", line 335, in map_to
    schema=schema or self.schema)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/schema.py", line 332, in __new__
    table._init(name, metadata, *args, **kw)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/schema.py", line 396, in _init
    self._autoload(metadata, autoload_with, include_columns)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/schema.py", line 413, in _autoload
    self, include_columns, exclude_columns
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1594, in run_callable
    with self.contextual_connect() as conn:
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1661, in contextual_connect
    self.pool.connect(),
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/pool.py", line 272, in connect
    return _ConnectionFairy(self).checkout()
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/pool.py", line 431, in __init__
    rec = self._connection_record = pool._do_get()
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/pool.py", line 788, in _do_get
    con = self._create_connection()
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/pool.py", line 225, in _create_connection
    return _ConnectionRecord(self)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/pool.py", line 318, in __init__
    self.connection = self.__connect()
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/pool.py", line 379, in __connect
    connection = self.__pool._creator()
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/engine/strategies.py", line 80, in connect
    return dialect.connect(*cargs, **cparams)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/sqlalchemy/engine/default.py", line 283, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/vagrant/pulseshims/lib/python2.6/site-packages/MySQLdb/connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
sqlalchemy.exc.OperationalError: (OperationalError) (1040, 'Too many connections') None None

The connection object should be falling out of scope and be deleted after every iteration, so I'm not sure what's happening here.  sqlsoup must be holding a reference to the objects.

Comment 1

[Mark Côté [:mcote]]

Attachment: bug-938927.diff

This should be a lot more robust now.  Tested by shutting down MySQL and Pulse at various points; it now always recovers.

Comment 2

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8334836 [details] [diff] [review]
bug-938927.diff

Review of attachment 8334836 [details] [diff] [review]:
-----------------------------------------------------------------

Nice

Comment 3

[Mark Côté [:mcote]]

https://hg.mozilla.org/automation/pulseshims/rev/dc7e9d3a4474