If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

admin1a.private.scl3.mozilla.com failed disk

RESOLVED FIXED

Status

Infrastructure & Operations
DCOps
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: dgarvey, Unassigned)

Tracking

Details

(Reporter)

Description

3 years ago
https://inventory.mozilla.org/en-US/systems/show/5099/

Array P410i in Slot 0 (Embedded) array A logicaldrive 1 (136.7 GB, RAID 1, Interim Recovery Mode) physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 146 GB, Failed)



[dgarvey@admin1a.private.scl3 ~]$ sudo -i hpacucli ctrl all show config detail

Smart Array P410i in Slot 0 (Embedded)
   Bus Interface: PCI
   Slot: 0
   Serial Number: 5001438014135B00
   Cache Serial Number: PBCDH0CRH0J6WM
   RAID 6 (ADG) Status: Disabled
   Controller Status: OK
   Hardware Revision: C
   Firmware Version: 6.40
   Rebuild Priority: Medium
   Expand Priority: Medium
   Surface Scan Delay: 15 secs
   Surface Scan Mode: Idle
   Queue Depth: Automatic
   Monitor and Performance Delay: 60  min
   Elevator Sort: Enabled
   Degraded Performance Optimization: Disabled
   Inconsistency Repair Policy: Disabled
   Wait for Cache Room: Disabled
   Surface Analysis Inconsistency Notification: Disabled
   Post Prompt Timeout: 0 secs
   Cache Board Present: True
   Cache Status: OK
   Cache Ratio: 25% Read / 75% Write
   Drive Write Cache: Disabled
   Total Cache Size: 512 MB
   Total Cache Memory Available: 400 MB
   No-Battery Write Cache: Disabled
   Cache Backup Power Source: Capacitors
   Battery/Capacitor Count: 1
   Battery/Capacitor Status: OK
   SATA NCQ Supported: True


      Logical Drive: 1
         Size: 136.7 GB
         Fault Tolerance: RAID 1
         Heads: 255
         Sectors Per Track: 32
         Cylinders: 35132
         Strip Size: 256 KB
         Full Stripe Size: 256 KB
         Status: Interim Recovery Mode
         Caching:  Enabled
         Unique Identifier: 600508B1001CB703AE193E15971523C0
         Disk Name: /dev/sda
         Mount Points: /boot 512 MB
         OS Status: LOCKED
         Logical Drive Label: A0175A675001438014135B00D7F5
         Mirror Group 0:
            physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 146 GB, Failed)
         Mirror Group 1:
            physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 146 GB, OK)
         Drive Type: Data
   Array: A
      Interface Type: SAS
      Unused Space: 0  MB
      Status: Failed Physical Drive
      Array Type: Data

      One of the drives on this array have failed or has been removed.




      physicaldrive 1I:1:1
         Port: 1I
         Box: 1
         Bay: 1
         Status: Failed
         Last Failure Reason: Write retries failed
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 146 GB
         Rotational Speed: 15000
         Firmware Revision: HPD4
         Serial Number: 63I0A0BWFRP91325
         Model: HP      EH0146FCBVB     
         Current Temperature (C): 26
         Maximum Temperature (C): 35
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown

      physicaldrive 1I:1:2
         Port: 1I
         Box: 1
         Bay: 2
         Status: OK
         Drive Type: Data Drive
         Interface Type: SAS
         Size: 146 GB
         Rotational Speed: 15000
         Firmware Revision: HPDJ
         Serial Number: 6TB2KYTW0000B2109XU2
         Model: HP      EH0146FAWJB     
         Current Temperature (C): 32
         Maximum Temperature (C): 39
         PHY Count: 2
         PHY Transfer Rate: 6.0Gbps, Unknown


   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 250
      Device Number: 250
      Firmware Version: RevC
      WWID: 5001438014135B0F
      Vendor ID: PMCSIERA
      Model:  SRC 8x6G       

[dgarvey@admin1a.private.scl3 ~]$

Updated

3 years ago
colo-trip: --- → scl3

Comment 1

3 years ago
pinged oncall and swapped drive. will close bug when rebuild completes.

$ sudo !!
sudo hpacucli ctrl all show config

Smart Array P410i in Slot 0 (Embedded)    (sn: 5001438014135B00)


      logicaldrive 1 (136.7 GB, RAID 1, Recovering, 3% complete)
   array A (SAS, Unused Space: 0  MB)


      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 146 GB, Rebuilding)
      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 146 GB, OK)

Comment 2

3 years ago
[vle@admin1a.private.scl3 ~]$ sudo hpacucli ctrl all show config

Smart Array P410i in Slot 0 (Embedded)    (sn: 5001438014135B00)


      logicaldrive 1 (136.7 GB, RAID 1, OK)
   array A (SAS, Unused Space: 0  MB)


      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 146 GB, OK)
      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 146 GB, OK)

   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 250 (WWID: 5001438014135B0F)

[vle@admin1a.private.scl3 ~]$
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.