931848 - New foopy requested on p10 for 11 pandas

Status: RESOLVED WONTFIX

(Infrastructure & Operations :: RelOps: General, task)

Description

[Pete Moore [:pmoore][:pete]]

For the pandas in https://bugzilla.mozilla.org/show_bug.cgi?id=892691 we'll need a new foopy on p10. We have one free foopy (foopy108) for the first 12 pandas, so only need one more for the remaining 11 pandas.

Is it possible to set one up for us?

Hope I'm parking this bug in the right place - shout if not!

Many thanks guys!
Pete

Comment 1

[Amy Rich [:arr] [:arich]]

The rack is limited to the 7 foopies that are already allocated.  There's no physical space left for more.

When we discussed this previously, there was speculation that the current panda->foopy mapping was underutilizing the foopies and that we could add more pandas to each foopy.  The current assumptions of 11 pandas per foopies were carried over form the days of tegras being attached to r3 minis, and nothing about that infrastructure really applies here since it's different hardware and a different datacenter all around.

At the time that discussion took place we didn't have any metrics on foopies (because they were running OS X), but we do now, and https://graphite.mozilla.org/ is claiming we only rarely see 50% utilization of the foopies in that rack and they're usually under 30% utilization.

foopy102 - foopy108 are available for use with those pandas and you can likely spread out the remaining ones across them.

Comment 2

[Pete Moore [:pmoore][:pete]]

Thanks Amy. That makes a lot of sense.

The current load is as follows:

foopy102: 12 pandas
foopy103: 12 pandas
foopy104: 12 pandas
foopy105: 13 pandas
foopy106: 19 pandas
foopy107: 8 pandas
foopy108: 12 pandas

I will add spread the additional load as follows:

foopy102: 13 (+1 new panda)
foopy103: 13 (+1 new panda)
foopy104: 13 (+1 new panda)
foopy105: 13
foopy106: 19
foopy107: 14 (+6 new pandas)
foopy108: 14 (+2 new pandas)

Thanks,
Pete