addons.mozilla.org knocked offline.

RESOLVED FIXED

Status

Cloud Services
Operations: Marketplace
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: w0ts0n, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
As per the time stamps:
[10:55:05]  nagios-phx1	 Tue 02:55:04 PDT [1330] addons.dynect.mozilla.net is DOWN :PING CRITICAL - Packet loss = 100%
[10:55:08]  nagios-phx1	 Tue 02:55:07 PDT [1331] marketplace.firefox.com is DOWN :PING CRITICAL - Packet loss = 100%
[10:56:53]  nagios-phx1	 Tue 02:56:53 PDT [1332] geo.marketplace.firefox.com is DOWN :PING CRITICAL - Packet loss = 100%
[10:57:28]  nagios-phx1	 Tue 02:57:27 PDT [1333] services.addons.mozilla.org is DOWN :PING CRITICAL - Packet loss = 100%
[10:57:30]  nagios-phx1	 Tue 02:57:29 PDT [1334] static.addons.mozilla.net is DOWN :PING CRITICAL - Packet loss = 100%

Confirmed via browser/curl

I rang oremj and he came online. It appears to have recovered a few minutes later.
[11:06:06]  nagios-phx1	 Tue 03:06:06 PDT [1336] addons.mozilla.org is UP :PING OK - Packet loss = 0%, RTA = 0.89 ms

Comment 1

3 years ago
It appears both netscaler units went offline.

mktns1 is still offline, it has a remote PDU (tripp-lite-03894.console.phx1.mozilla.com), but none of the typical oob credentials work, so I've been unable to access it.

Will look in to the root cause further in the morning.
Filed Support ticket with Citrix #63967137.

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.