1313819 - Pingdom Alert: Incident #27450 for support.mozilla.org

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Community Platform, task)

Description

[Ashlee Nguyen [:ashlee]]

User reported support.mozilla.org 

<jgmize> ashlee: having troubles on support.mozilla.org new relic monitor failed, not loading for me

Then recovered

Then became unavailable again

<jgmize> the site is loading now and the new relic monitor has recovered, so not urgent just tracking and whatever alert you got

Alert came in from Pingdom at 7:06 PM, it hasn't alerted since. Will update here if more alerts come in.

Comment 1

[Ryan Watson [:w0ts0n]]

from bug 1213817:
Tracking these alerts going critical then recovering.

(IRC) Fri 19:29:18 PDT [5896] nagios2.private.scl3.mozilla.com:Zeus Max Queue Times is CRITICAL: CRITICAL: zeus.zlb1_external_private_phx1_mozilla_com.pools.sumo:81.max_queue_time=4138.0ms

(IRC) Fri 19:31:18 PDT [5899] nagios2.private.scl3.mozilla.com:Zeus Max Queue Times is OK: OK: All queue times less than 2000 ms

(IRC) Fri 19:49:20 PDT [5908] nagios2.private.scl3.mozilla.com:Zeus Max Queue Times is CRITICAL: CRITICAL: zeus.zlb1_external_private_phx1_mozilla_com.pools.sumo:81.max_queue_time=10993.0ms

(IRC) Fri 19:50:20 PDT [5910] nagios2.private.scl3.mozilla.com:Zeus Max Queue Times is OK: OK: All queue times less than 2000 ms

(IRC) Fri 20:09:17 PDT [5916] nagios2.private.scl3.mozilla.com:Zeus Max Queue Times is CRITICAL: CRITICAL: zeus.zlb1_external_private_phx1_mozilla_com.pools.sumo:81.max_queue_time=11705.0ms

(IRC) Fri 20:12:16 PDT [5919] nagios2.private.scl3.mozilla.com:Zeus Max Queue Times is OK: OK: All queue times less than 2000 ms

Comment 2

[:Atoll]

Whatever *was* happening, it seems to have either been fixed or was fixed in a separate bug.

Comment 3

[Shyam Mani [:fox2mike]]

My apologies for all the spam. Sometimes, automation can be a pain, this is clearly one of those times. We'll try and fix this before we try and go through this process again.