833472 - Decommission or move to newer hardware vertica{1,2,3}.stage.metrics.scl3

Status: RESOLVED WONTFIX

(Infrastructure & Operations :: MOC: Service Requests, task)

Description

[Dumitru Gherman [:dumitru]]

vertica{1,2,3}.stage.metrics.scl3 are currently marked as being in production.
They run on old hardware (HP Generation 1 servers), and we have the following options:

a) decommission them.
b) move services to newer hardware.

Please let us know what the decision is, so we can get the ball rolling.
We recently renewed their warranty for one year only, so there's plenty of time to properly plan and schedule either options.
Thanks!

Comment 1

[Daniel Einspanjer [:dre] [:deinspanjer]]

We were planning on merging these machines into a cluster with the newer nodes (vertica{4,5,6}.  If we wish to decommission the hardware, we should replace it with newer hardware equivalent to the newer nodes.  That would likely involve ordering replacement hardware given the memory and disk layout.

Comment 2

[Shyam Mani [:fox2mike]]

(In reply to Daniel Einspanjer :dre [:deinspanjer] from comment #1)
> We were planning on merging these machines into a cluster with the newer
> nodes (vertica{4,5,6}.  If we wish to decommission the hardware, we should
> replace it with newer hardware equivalent to the newer nodes.  That would
> likely involve ordering replacement hardware given the memory and disk
> layout.

Yes, we'd like to decommission the hardware.

Comment 3

[Daniel Einspanjer [:dre] [:deinspanjer]]

So we agreed to decommission and replace the hardware, who is responsible for ordering the replacements?  We don't need to change the specs, just keep them roughly equivalent to vertica[4-6].

Comment 4

[Shyam Mani [:fox2mike]]

(In reply to Daniel Einspanjer :dre [:deinspanjer] from comment #3)
> So we agreed to decommission and replace the hardware, who is responsible
> for ordering the replacements?  We don't need to change the specs, just keep
> them roughly equivalent to vertica[4-6].

I can order the hardware. Would the exact same be sufficient? (newer hardware of course) or do you like to bump up RAM/HDD space etc?

Comment 5

[Daniel Einspanjer [:dre] [:deinspanjer]]

no special upgrades needed, just roughly the same as vertica6

Comment 6

[Shyam Mani [:fox2mike]]

Punting over to Dumitru. Please see if we can re-purpose something before we move to newer hardware. Also, probably hold on to this for a while before we take a call.

Comment 7

[Daniel Einspanjer [:dre] [:deinspanjer]]

Would like to pick this up again.  We need to get machines equivalent to vertica[4-6].metrics.scl3.mozilla.com to merge in to the cluster.  Do we have anything suitable that we can re-purpose or do we need to order?

Comment 8

[Ludovic Hirlimann [:Usul]]

Shyam what's the best way to find equivalent hardware so we can move on with this bug ?

Comment 9

[Shyam Mani [:fox2mike]]

(In reply to Ludovic Hirlimann [:Usul] from comment #8)
> Shyam what's the best way to find equivalent hardware so we can move on with
> this bug ?

Annie, do we still need a stage instance for vertica? If so, can these be virtualized or does it need to be hardware?

Comment 10

[Annie Elliott]

This discussion needs to hold for the Vertica contract renewal discussions.

Comment 11

[Sheeri Cabral [:sheeri]]

We're keeping Vertica, but we have vertica 4-6 in stage. Let's decom these.

You can coordinate with mpressman, but I believe that nobody uses these.

Comment 12

[Greg Cox [:gcox]]

mpressman: anything standing in the way of straight-up powering 1-3 off?

Comment 13

[Sheeri Cabral [:sheeri]]

Greg - we're likely going to migrate away from Vertica in 2015, so let's keep stage alive until we're sure we're going to do that, as we may need to test some stuff out.

Comment 14

[Sheeri Cabral [:sheeri]]

I'm going to resolve/wontfix - we are OK with taking the risk of having old hardware on stage.