1125112 - Fix datasource's closing connections that no longer exist (AttributeError: 'NoneType' object has no attribute 'open')

Status: RESOLVED WONTFIX

(Tree Management :: Treeherder, defect, P3)

Description

[Ed Morley [:emorley]]

https://rpm.newrelic.com/accounts/677903/applications/5585473/traced_errors/2954081717

Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/newrelic-2.40.0.34-py2.6-linux-x86_64.egg/newrelic/hooks/application_celery.py", line 66, in wrapper return wrapped(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__ return self.run(*args, **kwargs)
File "/data/www/treeherder.allizom.org/treeherder-service/treeherder/model/tasks.py", line 38, in cycle_data call_command('cycle_data')
File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 161, in call_command return klass.execute(*args, **defaults)
File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 255, in execute output = self.handle(*args, **options)
File "/usr/lib/python2.6/site-packages/newrelic-2.40.0.34-py2.6-linux-x86_64.egg/newrelic/api/function_trace.py", line 110, in literal_wrapper return wrapped(*args, **kwargs)
File "/data/www/treeherder.allizom.org/treeherder-service/treeherder/model/management/commands/cycle_data.py", line 73, in handle jm.disconnect()
File "/data/www/treeherder.allizom.org/treeherder-service/treeherder/model/derived/base.py", line 118, in disconnect dhub.disconnect()
File "/usr/lib/python2.6/site-packages/datasource/bases/SQLHub.py", line 245, in disconnect if self.connection[host_type]['con_obj'].open:
AttributeError: 'NoneType' object has no attribute 'open'

Comment 1

[Ed Morley [:emorley]]

This exception has since gone away, however we should still fix datasource to report these properly.

Broken out from bug 1134140...

(In reply to Ed Morley [:edmorley] from bug 1134140 comment #1)
> This corresponds to this datasource line:
> 
> https://github.com/mozilla/treeherder-service/blob/
> 52a653911e450fb5c5e9359438fd90a87c85b82a/vendor/datasource/bases/SQLHub.
> py#L245
> 
>             if self.connection[host_type]['con_obj'].open:
>                 self.connection[host_type]['con_obj'].commit()
>                 self.connection[host_type]['con_obj'].close()
> 
> It seems like datasource should be:
> (a) Explicitly checking to see if self.connection[host_type]['con_obj'] is
> not None first, and raising an exception that's actually clear eg
> "Connections lost".
> (b) Handling the exception in such a way that we don't stop trying to close
> the remaining connections for other host types.

Comment 2

[Ed Morley [:emorley]]

We're moving away from datasource.