856717 - Pulsetranslator died with "connection timed out"

Status: RESOLVED FIXED

(Webtools :: Pulse, defect)

Description

[Jonathan Griffin (:jgriffin)]

Pulsetranslator died again, this time with this stack trace:

Traceback (most recent call last):
  File "/home/webtools/apps/pulsetranslator/bin/runtranslator", line 8, in <module>
    load_entry_point('pulsetranslator==0.1', 'console_scripts', 'runtranslator')()
  File "/home/webtools/apps/pulsetranslator/src/pulsetranslator/pulsetranslator/runservice.py", line 35, in
main
    service.start()
  File "/home/webtools/apps/pulsetranslator/src/pulsetranslator/pulsetranslator/pulsetranslator.py", line 54, in start
    self.pulse.listen()
  File "build/bdist.linux-i686/egg/mozillapulse/consumers.py", line 136, in listen
  File "/home/webtools/apps/pulsetranslator/lib/python2.6/site-packages/carrot-0.10.7-py2.6.egg/carrot/messaging.py", line 446, in wait
    it.next()
  File "/home/webtools/apps/pulsetranslator/lib/python2.6/site-packages/carrot-0.10.7-py2.6.egg/carrot/backends/pyamqplib.py", line 300, in consume
    self.channel.wait()
  File "build/bdist.linux-i686/egg/amqplib/client_0_8/abstract_channel.py", line 95, in wait
  File "build/bdist.linux-i686/egg/amqplib/client_0_8/connection.py", line 202, in _wait_method
  File "build/bdist.linux-i686/egg/amqplib/client_0_8/method_framing.py", line 221, in read_method
socket.error: [Errno 110] Connection timed out

Comment 1

[Jonathan Griffin (:jgriffin)]

Attachment: Make the pulsetranslator reconnect on network error

Borrowed some code from the pulsesbuildmonitor for this.

Comment 2

[Mark Côté [:mcote]]

Comment on attachment 739557 [details] [diff] [review]
Make the pulsetranslator reconnect on network error

Review of attachment 739557 [details] [diff] [review]:
-----------------------------------------------------------------

Nice.  This bug means that the translator would stop receiving messages, but its process would keep running, right?  Just want to make sure, since when I've reset it before, I've noticed that it's still running, just not doing anything.

Comment 3

[Jonathan Griffin (:jgriffin)]

Yes, exactly.

Comment 4

[Jonathan Griffin (:jgriffin)]

https://github.com/mozilla/pulsetranslator/commit/c0d0e723d94f7c33f3f0d4c2ec6022e0eef2552f and deployed

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Jonathan, just a question for this landing. I assume we are safe in not missing any pulse notification giving that we run the daemon with --durable by default, right?

Comment 6

[Jonathan Griffin (:jgriffin)]

Yes, that's correct.  Since we run with --daemon, all messages are queued when the translator goes down, and are delivered when it gets restarted.

Comment 7

[Jonathan Griffin (:jgriffin)]

s/--daemon/--durable/