214521 - stage.mozilla.org down

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Roland Mainz]

stage.mozilla.org is down now since a while, making it impossible to upload any
nightlies or release builds

Some context info:
-- snip --
% /usr/local/openssh/bin/slogin -l gisburn stage.mozilla.org
ssh: connect to host stage.mozilla.org port 22: Connection timed out
% ping stage.mozilla.org 60
no answer from hemosaur.mozilla.org
% nslookup stage.mozilla.org
Server:  ns1.hrz.uni-giessen.de
Address:  134.176.2.5
Non-authoritative answer:
Name:    hemosaur.mozilla.org
Address:  67.72.78.219
Aliases:  stage.mozilla.org
% traceroute 67.72.78.219
traceroute to 67.72.78.219 (67.72.78.219), 30 hops max, 40 byte packets
 1  ro-natur.uni-giessen.de (134.176.2.40)  1.927 ms  1.355 ms  1.339 ms
 2  ro-win.uni-giessen.de (134.176.247.1)  1.405 ms  1.474 ms  1.397 ms
 3  ar-marburg2.g-win.dfn.de (188.1.42.133)  2.458 ms  2.305 ms  2.309 ms
 4  ar-marburg1-ge4-0.g-win.dfn.de (188.1.81.1)  2.419 ms  2.773 ms  2.435 ms
 5  cr-frankfurt1-po8-3.g-win.dfn.de (188.1.80.53)  4.795 ms  8.243 ms  4.256 ms
 6  so-6-0-0.ar2.FRA2.gblx.net (208.48.23.141)  7.771 ms  7.872 ms  7.823 ms
 7  pos6-0-2488M.cr1.FRA2.gblx.net (67.17.65.77)  4.052 ms  4.045 ms  3.987 ms
 8  so3-0-0-2488M.ar2.FRA3.gblx.net (67.17.65.82)  8.073 ms  7.882 ms  7.897 ms
 9  gigabitethernet7-2.Frankfurt1.Level3.net (195.122.136.245)  13.207 ms 
12.254 ms  12.254 ms
10  ae0-53.mp1.Frankfurt1.Level3.net (195.122.136.65)  13.344 ms  12.761 ms 
12.993 ms
11  so-0-0-0.mp1.London2.Level3.net (212.187.128.61)  20.358 ms  20.239 ms 
20.351 ms
12  so-1-0-0.bbr1.Washington1.level3.net (212.187.128.138)  98.674 ms  97.102
ms  96.914 ms
13  so-0-1-0.mpls1.Raleigh1.Level3.net (209.247.11.117)  111.306 ms  112.234 ms 
111.311 ms
14  gige6-0.hsipaccess1.Raleigh1.Level3.net (209.244.22.34)  108.082 ms  107.732
ms  107.734 ms
15  64.158.236.2 (64.158.236.2)  105.248 ms  107.099 ms  106.085 ms
16  br1.cr0.rdu.redundant.com (67.72.72.102)  105.029 ms  105.010 ms  104.701 ms
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
-- snip --

Comment 1

[Myk Melez [:myk] [@mykmelez]]

Redhat's whole network may be down.  blizzard logged out this morning and hasn't
logged back in all day.

Comment 2

[Dawn Endico]

reassigning to blizzard. i am also unable to connect to stage.

Comment 3

[Christopher Blizzard (:blizzard)]

Argh.  The running theory is that someone put a network cable in the wrong
place.  It was physically moved yesterday.  They are working on it now.

Comment 4

[Matthias Versen [:Matti]]

seems to work again