ip-ns01 and ns02 are having disk volume issues

RESOLVED FIXED

Status

Infrastructure & Operations
Virtualization
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: rbryce, Assigned: lerxst)

Tracking

Details

(Whiteboard: PHX DNS Outage)

(Reporter)

Description

6 years ago
THere are currently 4 esx host down or unavailable.

Mainly ip-ns01-02 are down.
ip-ns01 and 02 being down broke all of internal dns for phx1.  I temporarily aliased 10.8.75.21 to natasha and restarted named for resolution to pick back up until the main nameservers come back up.
(Reporter)

Updated

6 years ago
Summary: esx1-4.private.corp outage → ip-ns01 and ns02 are having disk volume issues
(Reporter)

Comment 2

6 years ago
I was thinking this was on HCI. But its on IP-Ops02.
It appears the virtual disk needs more space, before they vms can start.  At least thats what Vmware is reporting
(Reporter)

Comment 3

6 years ago
ip-esx01.phx.mozilla.com  ip-esx01.phx.mozilla.com are the affected esx hosts
Adding the chain since this caused a PHX1 HCI outage
(Assignee)

Comment 5

6 years ago
This has been fixed
Assignee: server-ops → dparsons
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
bug 763328 is followup for removing this as a SPOF.
(Reporter)

Updated

6 years ago
Duplicate of this bug: 763293
(Reporter)

Updated

6 years ago
Group: infra
(Reporter)

Updated

6 years ago
Whiteboard: PHX DNS Outage
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.