pulse-publisher didn't emit error when pulse connection was closed

RESOLVED FIXED

Status

Taskcluster
Platform Libraries
RESOLVED FIXED
11 months ago
3 months ago

People

(Reporter: jonasfj, Assigned: jonasfj)

Tracking

Details

(Assignee)

Description

11 months ago
See: https://sentry.io/taskcluster/taskcluster-queue/issues/205875529/

This caused 5min downtime, and we fixed by restarting the queue.

Long term solution:
  Fix pulse-publisher

We probably upgraded the underlying AMQP library and didn't check semantics changes for emit 'error'.
Or something like that. Or maybe tc-monitor failed to crash the process.

TODO: Investigate why and when errors are emitted.
(Assignee)

Comment 1

11 months ago
Looks like forced connection shutdown messages now issue a 'close' event, see:
  https://github.com/squaremo/amqp.node/issues/110

Hence, we didn't crash and by implication didn't reconnect.
(Assignee)

Comment 2

11 months ago
https://github.com/taskcluster/pulse-publisher/pull/16

Comment 3

3 months ago
This has been merged.
Assignee: nobody → jopsen
Status: NEW → RESOLVED
Last Resolved: 3 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.