Closed Bug 1288641 Opened 8 years ago Closed 8 years ago

support-rabbit[12].webapp.phx1:Webops Rabbit SUMO Unread Messages

Categories

(Infrastructure & Operations :: MOC: Problems, task)

Product:
Infrastructure & Operations
Component:
MOC: Problems
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: mdevney, Unassigned)

Details

03:10 < nagios-phx1> Fri 07:10:58 UTC [1005] support-rabbit2.webapp.phx1.mozilla.com:Rabbit Unread Messages is CRITICAL: RABBITMQ_OVERVIEW CRITICAL - messages CRITICAL (1803), messages_ready WARNING (1435), messages_unacknowledged OK (368) (http://m.mozilla.org/Rabbit+Unread+Messages)
03:11 < nagios-phx1> Fri 07:11:57 UTC [1006] support-rabbit1.webapp.phx1.mozilla.com:WebOps Rabbit SUMO Unread Messages is CRITICAL: Current value: 1290.0, warn threshold: 800.0, crit threshold: 1000.0 (http://m.mozilla.org/WebOps+Rabbit+SUMO+Unread+Messages)
03:15 < nagios-phx1> Fri 07:15:37 UTC [1007] support-rabbit1.webapp.phx1.mozilla.com:Rabbit Unread Messages is CRITICAL: RABBITMQ_OVERVIEW CRITICAL - messages CRITICAL (2213) messages_ready CRITICAL (1845), messages_unacknowledged OK (368) (http://m.mozilla.org/Rabbit+Unread+Messages)
03:18 < nagios-phx1> Fri 07:18:52 UTC [1009] support-rabbit2.webapp.phx1.mozilla.com:WebOps Rabbit SUMO Unread Messages is CRITICAL: Current value: 1908.0, warn threshold: 800.0, crit threshold: 1000.0 (http://m.mozilla.org/WebOps+Rabbit+SUMO+Unread+Messages)
[mdevney@support-celery1.webapp.phx1 ~]$ sudo supervisorctl status
celery-kitsune-prod              RUNNING   pid 28281, uptime 1 day, 22:53:21
[mdevney@support-celery1.webapp.phx1 ~]$ sudo supervisorctl restart celery-kitsune-prod
celery-kitsune-prod: stopped
celery-kitsune-prod: started
[mdevney@support-celery1.webapp.phx1 ~]$



[mdevney@support-celery2.webapp.phx1 ~]$ sudo supervisorctl status
celery-kitsune-prod              RUNNING   pid 21236, uptime 1 day, 22:53:20
[mdevney@support-celery2.webapp.phx1 ~]$
[mdevney@support-celery2.webapp.phx1 ~]$ sudo supervisorctl status
celery-kitsune-prod              RUNNING   pid 21236, uptime 1 day, 22:54:04
[mdevney@support-celery2.webapp.phx1 ~]$ sudo supervisorctl restart celery-kitsune-prod
celery-kitsune-prod: stopped
celery-kitsune-prod: started
[mdevney@support-celery2.webapp.phx1 ~]$




[mdevney@support-celery3.webapp.phx1 ~]$ sudo supervisorctl status
celery-kitsune-prod              RUNNING   pid 13301, uptime 1 day, 22:53:21
[mdevney@support-celery3.webapp.phx1 ~]$ sudo supervisorctl restart celery-kitsune-prod
celery-kitsune-prod: stopped
celery-kitsune-prod: started
[mdevney@support-celery3.webapp.phx1 ~]$
03:25 < nagios-phx1> Fri 07:25:32 UTC [1010] support-rabbit1.webapp.phx1.mozilla.com:Rabbit Unread Messages is OK: RABBITMQ_OVERVIEW OK - messages OK (656)
                     messages_ready OK (369) messages_unacknowledged OK (287) (http://m.mozilla.org/Rabbit+Unread+Messages)
03:25 < nagios-phx1> Fri 07:25:32 UTC [1011] support-rabbit2.webapp.phx1.mozilla.com:WebOps Rabbit SUMO Unread Messages is OK: Current value: 213.0, warn
                     threshold: 800.0, crit threshold: 1000.0 (http://m.mozilla.org/WebOps+Rabbit+SUMO+Unread+Messages)
03:25 < nagios-phx1> Fri 07:25:33 UTC [1012] support-rabbit2.webapp.phx1.mozilla.com:Rabbit Unread Messages is OK: RABBITMQ_OVERVIEW OK - messages OK (652)
                     messages_ready OK (368) messages_unacknowledged OK (284) (http://m.mozilla.org/Rabbit+Unread+Messages)
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Component: MOC: Incidents → MOC: Problems
You need to log in before you can comment on or make changes to this bug.