Closed Bug 920799 Opened 11 years ago Closed 11 years ago

backup1.private.tor1.mozilla.com:LSI Raid is CRITICAL

Categories

(Infrastructure & Operations :: DCOps, task)

Product:
Infrastructure & Operations
Component:
DCOps
Platform:
x86_64
Linux
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: afernandez, Assigned: afernandez)

Details

(Whiteboard: RFK-159-86276) 

backup1.private.tor1.mozilla.com:LSI Raid is CRITICAL: CRITICAL:0:0:RAID-6:12 drives:511.999GB:Partially 0:1:RAID-6:12 drives:8.591TB:Partially Drives:12 online1 Bad Drives (15 Errors)

Following drive is to blame;
Enclosure Device ID: 4
Slot Number: 7
Device Id: 12
Sequence Number: 3
Media Error Count: 4
Other Error Count: 11
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA
Raw Size: 931.512 GB [0x74706db0 Sectors]
Non Coerced Size: 931.012 GB [0x74606db0 Sectors]
Coerced Size: 931.0 GB [0x74600000 Sectors]
Firmware state: Failed
SAS Address(0): 0x5003048000823ecb
Connected Port Number: 0(path0) 
Inquiry Data:             9QJ7ZJKSST31000340NS                            SN06    
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Foreign State: None 
Device Speed: 3.0Gb/s 
Link Speed: 3.0Gb/s 
Media Type: Hard Disk Device
Drive:  Not Certified
Assignee: server-ops → server-ops-dcops
Component: Server Operations → Server Operations: DCOps
QA Contact: shyam → dmoore
Ticket ID 	#RFK-159-86276
Whiteboard: RFK-159-86276
:jlin swapped the drive;
# sudo /opt/MegaRAID/MegaCli/MegaCli64 -PDList  -aALL | grep -B 1 -A 23 "Slot Number: 7"
Enclosure Device ID: 4
Slot Number: 7
Device Id: 12
Sequence Number: 11
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA
Raw Size: 931.512 GB [0x74706db0 Sectors]
Non Coerced Size: 931.012 GB [0x74606db0 Sectors]
Coerced Size: 931.0 GB [0x74600000 Sectors]
Firmware state: Rebuild
SAS Address(0): 0x5003048000823ecb
Connected Port Number: 0(path0) 
Inquiry Data:             9QJ6MC1EST31000340NS                            SN16    
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Foreign State: None 
Device Speed: 3.0Gb/s 
Link Speed: 3.0Gb/s 
Media Type: Hard Disk Device
Drive:  Not Certified

Check progress:
# /opt/MegaRAID/MegaCli/MegaCli64 -PDRbld -ShowProg -PhysDrv [4:7] -aALL
                                     
Rebuild Progress on Device at Enclosure 4, Slot 7 Completed 1% in 6 Minutes.

Exit Code: 0x00
Assignee: server-ops-dcops → afernandez
This recovered at 22:13 Pacific.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
hard drive shipment back to ix - fedex  796933817271
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.