1213295 - Pulse GenericConsumer does not catch IOError from amqp - this is due to incorrect pulse credentials

Status: RESOLVED DUPLICATE of bug 1079515

(Webtools :: Pulse, defect)

Description

[Armen [:armenzg]]

Attachment: pulse.diff

It fails under _build_consumer():
> exchange(self.connection).declare(passive=True)
https://hg.mozilla.org/automation/mozillapulse/file/e6d10b16c666/mozillapulse/consumers.py#l166


Traceback (most recent call last):
  File "pulse_actions/worker.py", line 69, in run_pulse
    pulse.listen()
  File "build/bdist.linux-x86_64/egg/mozillapulse/consumers.py", line 148, in listen
    on_connect_callback=on_connect_callback
  File "build/bdist.linux-x86_64/egg/mozillapulse/consumers.py", line 166, in _build_consumer
    exchange(self.connection).declare(passive=True)
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/kombu-3.0.26-py2.7.egg/kombu/abstract.py", line 66, in __call__
    return self.bind(channel)
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/kombu-3.0.26-py2.7.egg/kombu/abstract.py", line 70, in bind
    return copy(self).maybe_bind(channel)
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/kombu-3.0.26-py2.7.egg/kombu/abstract.py", line 75, in maybe_bind
    self._channel = maybe_channel(channel)
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/kombu-3.0.26-py2.7.egg/kombu/connection.py", line 1054, in maybe_channel
    return channel.default_channel
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/kombu-3.0.26-py2.7.egg/kombu/connection.py", line 756, in default_channel
    self.connection
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/kombu-3.0.26-py2.7.egg/kombu/connection.py", line 741, in connection
    self._connection = self._establish_connection()
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/kombu-3.0.26-py2.7.egg/kombu/connection.py", line 696, in _establish_connection
    conn = self.transport.establish_connection()
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/kombu-3.0.26-py2.7.egg/kombu/transport/pyamqp.py", line 116, in establish_connection
    conn = self.Connection(**opts)
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/amqp-1.4.7-py2.7.egg/amqp/connection.py", line 180, in __init__
    (10, 30),  # tune
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/amqp-1.4.7-py2.7.egg/amqp/abstract_channel.py", line 67, in wait
    self.channel_id, allowed_methods)
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/amqp-1.4.7-py2.7.egg/amqp/connection.py", line 240, in _wait_method
    self.method_reader.read_method()
  File "/home/armenzg/venv/pulse_actions/lib/python2.7/site-packages/amqp-1.4.7-py2.7.egg/amqp/method_framing.py", line 189, in read_method
    raise m
IOError: Socket closed

Comment 1

[Mark Côté [:mcote]]

Comment on attachment 8671910 [details] [diff] [review]
pulse.diff

Review of attachment 8671910 [details] [diff] [review]:
-----------------------------------------------------------------

Also feel free to push to MozReview if you like. ;)

::: mozillapulse/consumers.py
@@ +4,5 @@
>  
>  import logging
>  import uuid
>  from socket import timeout as socket_timeout
> +from socket import IOError

Needs to be sorted properly (move up a line).

@@ +60,5 @@
>          if connect:
> +            try:
> +                self.connect()
> +            except IOError:
> +                logging.error("Your Pulse user and password are likely to be incorrect")

Hm.  Could you do some tests and see what other conditions will raise an IOError?  What about if the host is unreachable, or the port closed, or the port not serving amqp?  I wouldn't want to send the user down the wrong path, and there is probably a better way to solve this, although it may require going down to the amqp module.

Comment 2

[Ed Morley [:emorley]]

This kombu/py-amqp unhelpful behaviour caused us confusion recently in bug 1287404 too (we're not using mozillapulse). I've filed an upstream issue to see if handling can be improved there:
https://github.com/celery/kombu/issues/611

Comment 3

[Ed Morley [:emorley]]

This is now fixed in newer py-amqp (used by Kombu).