1317815 - Increase in storage on xiph1

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Thomas Daede [:TD-Linux]]

I would like to add at least another 10TiB of storage to xiph1. 

It's possible its RAID array is currently maxed out (not sure how to check, apparently it's hwraid), so this might involve replacing disks.

Another option is to move to a cloud provider, though my brief calculations showed that this probably wouldn't be economical.

Comment 1

[Van Le [:van]]

the HP d2000 disk enclosure has all of its bays (12) populated by 3.5'' (LFF) 3TB drives. if you want to increase the storage, we will need to upgrade these to 4TB, 6TB, or 8TB drives. what is the current RAID configuration if there is any? 4TB drives will only work if you are running RAID 0 or JBODs but it would still probably make more sense to get the 6TB or 8TB upgrades depending on your configuration. let me know what you decide and i can request a quote from our VAR.


https://www.hpe.com/us/en/product-catalog/servers/server-hard-drives.filters-facet_formfactor_url:LFF.html

Comment 2

[Thomas Daede [:TD-Linux]]

The current RAID configuration is RAID 1, however it's not visible from software (though I'm only used to mdadm, maybe there are some special HP tools to see it?). Jumping to 6TB or 8TB seems to make much more sense. We could also use this opportunity to change RAID modes, e.g. use a less redundant mode, combined with offsite backups.

Comment 3

[Van Le [:van]]

i've emailed our VAR for quotes on 6TB and 8TB drives. 

>The current RAID configuration is RAID 1, however it's not visible from software (though I'm only used to mdadm, maybe there are some special HP tools to see it?).

hmmm... mdadm is the linux utility for software raid. is the d2000 configured with a hardware RAID0 or as JBODs then? can you try to install hpacucli to confirm the hardware configuration on the ds2000? perhaps there are unused disks or conflicting configs? a hardware raid1 + a software raid1 might be overkill.

http://h20564.www2.hpe.com/hpsc/swd/public/detail?swItemId=MTX_e0ab8f2d629a4748ad073e2943

the controller on the d2000 should be supported by the utility per https://www.hpe.com/h20195/v2/getpdf.aspx/c04111697.pdf?ver=23

Comment 4

[Ralph Giles (:rillian)]

Attachment: hpacucli output

Thanks for the pointer, I didn't know how to talk to the raid controller. hpacucli confirms 12x3TB arranged in a 16TB RAID 1+0, matching the discussion in bug 740629. Upgrading those to 8TB seems the way to go.

The system uses pure hardware raid with the aggregate volumes presenting as /dev/sda and /dev/sdb. Neither mdadm nor /proc/mdstat has ever had useful information.

Comment 5

[Van Le [:van]]

per our vendor (yes we have the AJ940A model):

Just confirming that the enclosure is the HP D2600 # AJ940A.   
If so this is the largest capacity HP drive I show as an option.

SKU: 4M6060 HP 6TB 6G SAS 7.2K rpm LFF (3.5-inch) Midline Hard Drive                     $895


asking our vendor why the 8TB isn't compatible since it should only be a sled issue which we can swap with the 3TB.

Comment 6

[Van Le [:van]]

per our vendor:

Van

I spoke to HP and the largest HP option for the D2600 is the one I sent earlier, the 2600. 

Not sure if this is BIOS, sled, or otherwise related – I can research further if you wish

:TD-Linux, let me know how you'd like to proceed. looks like we can double the drive size at a cost of $895 per drive.

Comment 7

[Thomas Daede [:TD-Linux]]

Hmm, that seems like the best option. I'm going to assume other options, like switching to a different array, are worse? I ran additional pricing on cloud providers (900 per drive is a hefty premium), but this still makes the most sense.

Comment 8

[Thomas Daede [:TD-Linux]]

I found a couple of other options:

1. Buy a license key upgrade to add RAID 6 to the P411 storage controller, and migrate the array to that. That's a "free" 10TB upgrade. I don't think this can be done in place, though.

2. Migrate array to RAID 5. Same as 1, doesn't require buying a license key but is less reliable.

3. Set the RAID controller to one logical volume per physical volume, and do software RAID 6.

All of these definitely will need a backup first - is there any onsite backup that can hold 18TB of data, or should I do a backup locally to the MV office?

Comment 9

[Ludovic Hirlimann [:Usul]]

(In reply to Thomas Daede [:TD-Linux] from comment #8)

> All of these definitely will need a backup first - is there any onsite
> backup that can hold 18TB of data, or should I do a backup locally to the MV
> office?

bacula can handle that, file a bug in infra:backups to have it done.

Comment 10

[Van Le [:van]]

the raid 6 seems the most appropriate. you'll have 30TB of storage at a loss of some disk performance but with a tolerance of 2 drive failure and you won't need to buy any drives. ive emailed our VAR for a quote on the HP Smart Array Advanced Pack. it looks like it'll cost about $400 or so. let me know if you want to go this route or if you decide to do the software raid.

please open up a ticket for bacula and they can handle the backups per #c9.

Comment 11

[Van Le [:van]]

asking our VAR to dig deeper, probably under a different name or service.


 
Looks like the problem is that the part is no longer available
 
516471-B21 - HP Smart Array Advanced Pack No Media 1 Server 1yr 24x7 Support Software
 
I can dig deeper if you wish

Comment 12

[Van Le [:van]]

our VAR can't find where to buy the license to enable to raid 6. i went ahead and opened a case with HP (case id 5315951308).



On Wed, Dec 21, 2016 at 7:02 PM, Rich Pomper <[e-mail]> wrote:
I have been unable to locate the product


On Dec 21, 2016, at 6:52 PM, Van Le <[e-mail]> wrote:

hi rich, 

any luck? did HP drop their support for raid 6?

thanks,
van 

On Mon, Dec 19, 2016 at 1:45 PM, Rich Pomper <[e-mail]> wrote:
Doesn’t look good but let me see what I can find

Comment 13

[Van Le [:van]]

server needs to have a cache battery and also under warranty to request a license key. the server isn't currently under service and i'm not sure if it has a cache battery (the output you provided didn't include that information). have you looked into a software RAID?


Please note that a license key and a minimum of 256MB cache with battery kit or a license key and FBWC are required to enable features of raid configuration
. For more information go to: http://www.hp.com/go/saap
The license entitlement certificate must be redeemed online or via fax in order to obtain the license activation key(s). Include one year of 24x7 HP Software Technical Support Services. NOTE: A license key and a minimum

Also please let us know incase you need any further information from our end .

Comment 14

[Thomas Daede [:TD-Linux]]

This is now doine, it's in RAID 50, still using the HW RAID. I coordinated with :gcox to backup and restore the data as live migration wasn't possible. Thanks!

Comment 15

[Ralph Giles (:rillian)]

Thanks, Thomas!