Closed Bug 2003883 Opened 15 hours ago Closed 6 minutes ago

500 errors from firefoxci tc queue service [Error: PulsePublisher.sendDeadline exceeded]

Categories

(Cloud Services :: Operations: Taskcluster, defect, P1)

Product:
Cloud Services
Component:
Operations: Taskcluster
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: jcristau, Assigned: jcristau)

Details

No description provided.

The outage started at 2025-12-03 15:09Z, firefox-ci-tc.services.mozilla.com started sending lots of 500 errors, with logs from the queue service showing errors publishing messages to pulse.

Severity: -- → S1
tracking-firefox146: --- → ?
tracking-firefox147: --- → ?
Priority: -- → P1

The queue/taskclusteretl-dev/task-completed rabbitmq queue was crashing, and apparently blocking everything else (why???); deleting that queue appears to have resolved things for now.

Assignee: nobody → jcristau
Status: NEW → RESOLVED
Closed: 6 minutes ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.