Closed Bug 1351153 Opened 8 years ago Closed 8 years ago

releases.mozilla.org (54.192.18.20) is DOWN :PING CRITICAL - Packet loss = 100%

Categories

(Cloud Services :: Operations: Product Delivery, task)

Product:
Cloud Services
Component:
Operations: Product Delivery
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: ryanc, Unassigned)

Details

No description provided.
Hello, Had this come in, nagios-scl3> Tue 02:47:26 UTC [5645] releases.mozilla.org (54.192.18.20) is DOWN :PING CRITICAL - Packet loss = 100% Site looks accessible, and curl returns 200 according according to docs, https://mana.mozilla.org/wiki/display/SVCOPS/Product+Delivery#ProductDelivery-Troubleshooting $ curl -v -o /dev/null https://archive.mozilla.org/pub/firefox/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 52.84.124.6... * TCP_NODELAY set * Connected to archive.mozilla.org (52.84.124.6) port 443 (#0) * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 * Server certificate: ftp.mozilla.org * Server certificate: DigiCert SHA2 Secure Server CA * Server certificate: DigiCert Global Root CA > GET /pub/firefox/ HTTP/1.1 > Host: archive.mozilla.org > User-Agent: curl/7.51.0 > Accept: */* > < HTTP/1.1 200 OK Traceroute from nagios3.private.scl3 looks good, Host Loss% Snt Last Avg Best Wrst StDev 1. fw1.private.scl3.mozilla.net 0.0% 10 0.7 0.7 0.7 0.8 0.0 2. 63.245.214.53 0.0% 10 2.7 2.7 2.3 2.9 0.0 3. 63.245.214.46 0.0% 10 1.1 1.1 1.0 1.2 0.0 4. xe-1-2-3.border1.sjc2.mozilla.net 0.0% 10 1.9 1.9 1.8 2.0 0.0 5. xe-1-2-0.border1.pao1.mozilla.net 0.0% 10 2.2 2.1 2.0 2.3 0.0 6. ip67-92-171-49.z171-92-67.customer.algx.net 0.0% 10 2.6 4.5 2.5 21.7 6.0 7. 207.88.13.224.ptr.us.xo.net 0.0% 10 41.7 41.6 41.4 42.1 0.0 8. 207.88.12.143.ptr.us.xo.net 0.0% 10 41.6 43.4 41.5 52.9 4.0 9. 207.88.12.140.ptr.us.xo.net 0.0% 10 41.7 44.3 41.4 54.4 5.0 10. 207.88.12.180.ptr.us.xo.net 0.0% 10 41.5 44.4 41.3 71.2 9.4 11. 207.88.14.189.ptr.us.xo.net 0.0% 10 41.4 41.7 41.3 42.7 0.3 12. 52.95.217.244 0.0% 10 41.4 41.4 41.3 41.5 0.0 13. 176.32.125.152 0.0% 9 94.4 91.8 78.9 98.9 5.9 14. 176.32.125.155 0.0% 9 75.8 75.3 74.4 76.0 0.4 15. 54.239.43.198 0.0% 9 75.3 75.3 75.1 75.8 0.0 16. 54.239.104.151 25.0% 9 84.0 78.2 76.6 84.0 2.8 17. ??? 18. server-54-192-18-20.iad12.r.cloudfront.net 0.0% 9 75.6 75.7 75.6 75.8 0.0
Update, Site is back up (according to nagios in scl3), 19:58:10 <@nagios-scl3> Tue 02:58:10 UTC [5652] releases.mozilla.org (54.192.18.20) is UP :PING OK - Packet loss = 0%, RTA = 75.50 ms
Summary: nagios-scl3> Tue 02:47:26 UTC [5645] releases.mozilla.org (54.192.18.20) is DOWN :PING CRITICAL - Packet loss = 100% → releases.mozilla.org (54.192.18.20) is DOWN :PING CRITICAL - Packet loss = 100%
releases/archive is just Cloudfront -> s3. I'm not sure Cloudfront officially support ICMP, so a ping check probably isn't needed. It's possible that whatever Cloudfront node nagios was talking to went down or just doesn't respond to icmp. All other monitors are green, so I will close this out.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
I added a simple https uptime check in pingdom for this, to check every 5 minutes and Ryan will yank the nagios ping check.
You need to log in before you can comment on or make changes to this bug.