Closed Bug 914248 Opened 11 years ago Closed 11 years ago

mdn can't connect to elasticsearch-zlb.webapp.scl3.mozilla.com

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P1)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Platform:
x86
macOS
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: groovecoder, Assigned: bburton)

Details

Started yesterday around noon?

https://errormill.mozilla.org/mdn/mdn/group/91489/
priority bump - ongoing production stacktraces
Severity: normal → major
We've been seeing some networking issues in SCL3 since then, including one a few minutes ago, which netops is working on

As of right now the ES's ZLB is reachable from all the prod mdn hosts

[root@developeradm.private.scl3 ~]# issue-multi-command developer nc -z elasticsearch-zlb.webapp.scl3.mozilla.com 9200
[2013-09-09 09:35:56] [developer1.webapp.scl3.mozilla.com] running: nc -z elasticsearch-zlb.webapp.scl3.mozilla.com 9200
[2013-09-09 09:35:56] [developer2.webapp.scl3.mozilla.com] running: nc -z elasticsearch-zlb.webapp.scl3.mozilla.com 9200
[2013-09-09 09:35:56] [developer3.webapp.scl3.mozilla.com] running: nc -z elasticsearch-zlb.webapp.scl3.mozilla.com 9200
[2013-09-09 09:35:56] [developer1.webapp.scl3.mozilla.com] finished: nc -z elasticsearch-zlb.webapp.scl3.mozilla.com 9200 (0.227s)
[developer1.webapp.scl3.mozilla.com] out: Connection to elasticsearch-zlb.webapp.scl3.mozilla.com 9200 port [tcp/wap-wsp] succeeded!
[2013-09-09 09:35:56] [developer2.webapp.scl3.mozilla.com] finished: nc -z elasticsearch-zlb.webapp.scl3.mozilla.com 9200 (0.308s)
[developer2.webapp.scl3.mozilla.com] out: Connection to elasticsearch-zlb.webapp.scl3.mozilla.com 9200 port [tcp/wap-wsp] succeeded!
[2013-09-09 09:35:56] [developer3.webapp.scl3.mozilla.com] finished: nc -z elasticsearch-zlb.webapp.scl3.mozilla.com 9200 (0.308s)
[developer3.webapp.scl3.mozilla.com] out: Connection to elasticsearch-zlb.webapp.scl3.mozilla.com 9200 port [tcp/wap-wsp] succeeded!
Assignee: server-ops-webops → bburton
Severity: major → normal
Status: NEW → ASSIGNED
Priority: -- → P1
The cluster was also unhappy due to the networking partitions earlier. The cluster is now green again and all indexes are happy.
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.