1503996 - [MDC1] Request for NAS attached to Vreplay Appliance eth1 interface

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Mark Richards [:freshness]]

Requesting a NAS solution for our Vidyo Replay appliance in MDC1.  I have purchased a physical NAS that "could" but installed to increase our Storage capacity for the Vreplay. That device is currently being shipped to MDC1 per van's request.  Link to the device description here: 

https://www.amazon.com/Buffalo-TeraStation-Desktop-Drives-Included/dp/B00ND1L3DQ/ref=pd_cp_147_4?pf_rd_m=ATVPDKIKX0DER&pf_rd_p=ef4dc990-a9ca-4945-ae0b-f8d549198ed6&pf_rd_r=1SNJBJY2XNX76Z16PEP6&pd_rd_wg=Xxhfo&pf_rd_s=desktop-dp-sims&pf_rd_t=40701&pd_rd_w=7T8k6&pf_rd_i=desktop-dp-sims&pd_rd_r=a6975029-de0f-11e8-9610-817859b593e5&pd_rd_i=B00ND1L3DQ&psc=1&refRID=1SNJBJY2XNX76Z16PEP6


The phsyical NAS option was not our first choice, but due to a few flaw's with Vidyo's infrastructure design, using VM-hosted storage introduced a multitude of issues.  History of those bugs are attached to this bug.

Comment 1

[Van Le [:van]]

:freshness has this been tested? do you have an IP you want me to configure before i attach it to eth1 of the vreplay appliance? are there credentials and such you also want configured?

Comment 2

[Van Le [:van]]

:freshness, if there's a rack mountable version that works for you guys, can you order that one? this is a the desktop version and we shouldn't be installing it in the data center.

Comment 3

[Corey Shields [:cshields]]

Can you hold on shipping anything for a bit please.  I don't feel that a 2 drive COTS NAS is going to be a good idea for us here.

Comment 4

[Greg Cox [:gcox]]

> The phsyical NAS option was not our first choice, but due to a few flaw's with Vidyo's infrastructure design, using VM-hosted storage introduced a multitude of issues.

A little history, if I may, since the other bug is locked down:
* Yes, Vidyo's simplistic idea of a NAS (must be SMB/CIFS or direct-connect) is the root issue.  NFS would be nice here and would open up easier integration with the filer.  That can be worked around with a middleware VM that bridges NFS to SMB (hacky) or a CIFS license (expensive).  "annoyance, but not blocker."
* turning vreplay into a VM runs afoul of the usual Vidyo idiocy that VMs must be fully CPU-reserved, which puts a small bind while we wait for gear to ship and be included in mdc1/mdc2.  "annoyance, but not blocker."
* The filer is currently down on space as shelves ship to mdc1/mdc2 from scl3.  The expansion therein would alleviate space concerns in the immediate, but we have a refresh pending, and not having a monster volume going into the refresh would make things easier.  "annoyance, blocker for now, but eventual not blocker."
* bug 1494407 comment 4 had us going into next year, so we've not looked at anything on the VM side.  I felt we had an understanding to look at this next year - has the needs/expectations/whatever gotten suddenly more critical?
* Same bug, there's also the possibility of an S3 bucket happening in the future.

Overall, yeah, if this needs to happen NOW then I don't think storage/virtualization can help you, and I would support van's approach of "a rackmount NAS" if that's what is needed.  I do worry about that device being a SPOF, but I don't have a better offer at the moment, without some acceleration to our filer work.  We'd be happy to approach the mapper-VM-and-filer-space solution when it's possible/appropriate.

Comment 5

[Mark Richards [:freshness]]

REQ0061116 opened for the rack mounted version as well to be delivered to MDC1.

Comment 6

[Mark Richards [:freshness]]

The urgency has been bumped up, since our vreplay recently hit 85% capacity, so we're trying to get ahead of this before it becomes a broader issue.  AVOPs was also investigating the Vidyo Cloud option to resolve our storage limitations, but the timeline for that project just got pushed out further down the road. 

To be quite frank, we really just need to pull these recordings off of the vidyo replay, convert them from .flv files to .mp4's, and eventually host them elsewhere. Since there is a bit of a time sensitivity in this area, direct-connect seems to be our best bet. 

  To Answer Van's questions: The AVOP's team has not configured a NAS for the vreplay before.  However, I do have preferences in how to set it up:

    8Tb Rack Mounted NAS in Raid 1 (to mitigate the SPOF)
    hostname: vstorage1.mdc1.mozilla.com
    IP: 63.245.208.172
    Subnet: 255.255.255.224
    GW: 63.245.208.161

I second gcox's preferences for a Hard Share password.

Comment 7

[Van Le [:van]]

freshness, if it's attached directly to the vreplay appliance, can't we use a private IP for it to talk to one another? do we need to use/waste a public IP? and i'm not sure if you'd be able to directly reach it from outside since your vreplay appliance shouldn't be doing any type of internet sharing/hotspot.

Comment 8

[Mark Richards [:freshness]]

Hello Van, yes, we can't definitely use a Private IP for the new rack-mounted NAS.  One has been ordered, and will be delivered to MDC1 on Friday Nov. 9th.

Comment 9

[Corey Shields [:cshields]]

I want to clarify the affirmative negative  :)   (yes, we can't definitely)..

Does it *have* to have a public IP?  If so that might be a show stopper, from a security POV.

Comment 10

[Mark Richards [:freshness]]

Apologies!

  We can certainly use a Private IP for the new rack-mounted NAS (and it is preferred)

Comment 11

[Van Le [:van]]

we received the hardware but it requires m5 or smaller screws. the standard screw size is m6. i checked and wasn't able to find any small screws. will order and do another return trip to mdc1.

Comment 12

[Van Le [:van]]

opened RITM0067887 for some m5 and 10-32 screws and nuts.

Comment 13

[Van Le [:van]]

installed, configured, and added to the CMDB.