823510 - Celery needs to retry on another node

Status: RESOLVED FIXED

(Marketplace Graveyard :: General, defect, P4)

Description

[Tarek Ziadé (:tarek)]

See https://wiki.mozilla.org/Marketplace/HAResults#RabbitMQ

And:

14:56 < tarek> hi Celery! is there a way in kombu to replay a call to the broken to a fallback node when the connection errors out ?
15:14 < asksol_> tarek: in kombu 2.5 you can have multiple url's
15:14 < asksol_> since kombu 2.5 i should say, since it's already the latest
15:15 < asksol_> this is functionality that augments Connection.ensure .autoretry
15:16 < asksol_> connection.ensure(obj, method, **retry_policy)(*args, **kwargs) will retry in the event of connection failure, and obj.__revive__(channel) is called when a new 
                 connection/channel has been made
15:17 < asksol_> Producer, Queue, Exchange,etc all support __revive__
15:19 < asksol_> Producer.publish also supports a retry argument: Producer.publish(message, retry=True, declare=[queue1], retry_policy={…})
15:20 < asksol_> I should really find the time to document this, kombu docs are really incomplete

Comment 1

[Allen Short [:ashort]]

Celery 3.x uses the new version of kombu. Failover hosts can be specified in BROKER_URL as URLs separated by semicolons.

Comment 2

[Wil Clouser [:clouserw]]

(In reply to Allen Short [:ashort] from comment #1)
> Celery 3.x uses the new version of kombu. Failover hosts can be specified in
> BROKER_URL as URLs separated by semicolons.

What is the status of upgrading?

Comment 3

[Allen Short [:ashort]]

Upgrading celery to 3.x caused some inexplicable tracebacks on -dev. Postponing this until we have a better way to diagnose/reproduce these problems.

Comment 4

[Allen Short [:ashort]]

zamboni now uses Celery 3.x.

Comment 5

[Victor Carciu]

Please add STRs or mark it as [qa-] if no QA is needed for this bug.