Hooks should handle nonexistent exchanges more gracefully
Categories
(Taskcluster :: Services, defect)
Tracking
(Not tracked)
People
(Reporter: dustin, Assigned: dustin)
Details
If you make a hook and create a binding to an exchange that doesn't exist, queue will report an error on every re-sync run. This isn't an error the operations team needs to know about, though. It's probably best to log it at an informational level and forget about it.
|
Assignee | |
Comment 1•7 years ago
|
||
Ah, when there are >1 hooks with invalid exchanges, each one triggers a reconnect to pulse, and they just bounce back and forth generating lots of errors. I saw a huge storm of errors in staging when I created all of the Firefox hg-push hooks, since exchange/hgpushes doesn't exist on the staging AMQP server.
|
|
Assignee | |
Comment 2•7 years ago
|
||
I think that the behavior of simply never firing is unsurprising behavior from the user's perspective, and it would be difficult to thread the nonexistence of the exchange back to the user (especially if the exchange is deleted after the hook is created). So I'll just take that approach.
And I'll also update this reconciliation code with some of the more recent learnings about AMQP's behavior with respect to errors -- an invalid exchange need not terminate the connection!
|
|
Assignee | |
Comment 3•7 years ago
|
||
|
|
Assignee | |
Comment 4•7 years ago
|
||
Much better:
2019-04-19T19:58:29.336154+00:00 app[listeners.1]: 2019-04-19T19:58:29.336Z listeners garbage/beta: updating bindings to [{"exchange":"exchange/foo","routingKeyPattern":"#"}]
2019-04-19T19:58:29.336382+00:00 app[listeners.1]: 2019-04-19T19:58:29.336Z listeners garbage/delhi: creating listener (and queue if necessary)
2019-04-19T19:58:29.466334+00:00 app[listeners.1]: 2019-04-19T19:58:29.466Z listeners error binding exchange exchange/foo with #: Error: Operation failed: QueueBind; 404 (NOT-FOUND) with message "NOT_FOUND - no exchange 'exchange/foo' in vhost '/'" (ignored)
2019-04-19T19:58:29.467995+00:00 app[listeners.1]: 2019-04-19T19:58:29.467Z base:entity Return modify trivially, as no changes was made by modifier
2019-04-19T19:58:29.602121+00:00 app[listeners.1]: 2019-04-19T19:58:29.602Z listeners garbage/delhi: updating bindings to [{"exchange":"exchange/bar","routingKeyPattern":"#"}]
2019-04-19T19:58:29.727571+00:00 app[listeners.1]: 2019-04-19T19:58:29.727Z listeners error binding exchange exchange/bar with #: Error: Operation failed: QueueBind; 404 (NOT-FOUND) with message "NOT_FOUND - no exchange 'exchange/bar' in vhost '/'" (ignored)
Description
•