support-rabbit[12].webapp.phx1:Webops Rabbit SUMO Unread Messages

RESOLVED FIXED

Status

Infrastructure & Operations
MOC: Problems
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: jedi, Unassigned)

Tracking

Details

(Reporter)

Description

a year ago
03:10 < nagios-phx1> Fri 07:10:58 UTC [1005] support-rabbit2.webapp.phx1.mozilla.com:Rabbit Unread Messages is CRITICAL: RABBITMQ_OVERVIEW CRITICAL - messages CRITICAL (1803), messages_ready WARNING (1435), messages_unacknowledged OK (368) (http://m.mozilla.org/Rabbit+Unread+Messages)
03:11 < nagios-phx1> Fri 07:11:57 UTC [1006] support-rabbit1.webapp.phx1.mozilla.com:WebOps Rabbit SUMO Unread Messages is CRITICAL: Current value: 1290.0, warn threshold: 800.0, crit threshold: 1000.0 (http://m.mozilla.org/WebOps+Rabbit+SUMO+Unread+Messages)
03:15 < nagios-phx1> Fri 07:15:37 UTC [1007] support-rabbit1.webapp.phx1.mozilla.com:Rabbit Unread Messages is CRITICAL: RABBITMQ_OVERVIEW CRITICAL - messages CRITICAL (2213) messages_ready CRITICAL (1845), messages_unacknowledged OK (368) (http://m.mozilla.org/Rabbit+Unread+Messages)
03:18 < nagios-phx1> Fri 07:18:52 UTC [1009] support-rabbit2.webapp.phx1.mozilla.com:WebOps Rabbit SUMO Unread Messages is CRITICAL: Current value: 1908.0, warn threshold: 800.0, crit threshold: 1000.0 (http://m.mozilla.org/WebOps+Rabbit+SUMO+Unread+Messages)
(Reporter)

Comment 1

a year ago
[mdevney@support-celery1.webapp.phx1 ~]$ sudo supervisorctl status
celery-kitsune-prod              RUNNING   pid 28281, uptime 1 day, 22:53:21
[mdevney@support-celery1.webapp.phx1 ~]$ sudo supervisorctl restart celery-kitsune-prod
celery-kitsune-prod: stopped
celery-kitsune-prod: started
[mdevney@support-celery1.webapp.phx1 ~]$



[mdevney@support-celery2.webapp.phx1 ~]$ sudo supervisorctl status
celery-kitsune-prod              RUNNING   pid 21236, uptime 1 day, 22:53:20
[mdevney@support-celery2.webapp.phx1 ~]$
[mdevney@support-celery2.webapp.phx1 ~]$ sudo supervisorctl status
celery-kitsune-prod              RUNNING   pid 21236, uptime 1 day, 22:54:04
[mdevney@support-celery2.webapp.phx1 ~]$ sudo supervisorctl restart celery-kitsune-prod
celery-kitsune-prod: stopped
celery-kitsune-prod: started
[mdevney@support-celery2.webapp.phx1 ~]$




[mdevney@support-celery3.webapp.phx1 ~]$ sudo supervisorctl status
celery-kitsune-prod              RUNNING   pid 13301, uptime 1 day, 22:53:21
[mdevney@support-celery3.webapp.phx1 ~]$ sudo supervisorctl restart celery-kitsune-prod
celery-kitsune-prod: stopped
celery-kitsune-prod: started
[mdevney@support-celery3.webapp.phx1 ~]$
(Reporter)

Comment 2

a year ago
03:25 < nagios-phx1> Fri 07:25:32 UTC [1010] support-rabbit1.webapp.phx1.mozilla.com:Rabbit Unread Messages is OK: RABBITMQ_OVERVIEW OK - messages OK (656)
                     messages_ready OK (369) messages_unacknowledged OK (287) (http://m.mozilla.org/Rabbit+Unread+Messages)
03:25 < nagios-phx1> Fri 07:25:32 UTC [1011] support-rabbit2.webapp.phx1.mozilla.com:WebOps Rabbit SUMO Unread Messages is OK: Current value: 213.0, warn
                     threshold: 800.0, crit threshold: 1000.0 (http://m.mozilla.org/WebOps+Rabbit+SUMO+Unread+Messages)
03:25 < nagios-phx1> Fri 07:25:33 UTC [1012] support-rabbit2.webapp.phx1.mozilla.com:Rabbit Unread Messages is OK: RABBITMQ_OVERVIEW OK - messages OK (652)
                     messages_ready OK (368) messages_unacknowledged OK (284) (http://m.mozilla.org/Rabbit+Unread+Messages)
(Reporter)

Updated

a year ago
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
(Assignee)

Updated

a year ago
Component: MOC: Incidents → MOC: Problems
Product: Infrastructure & Operations → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.