Closed Bug 689937 Opened 9 years ago Closed 8 years ago

install five remaining r4 minis in scl1

Categories

(Infrastructure & Operations :: RelOps: General, task)

x86
macOS
task
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bear, Assigned: arich)

References

Details

here is the math:

160 tegras / 15 tegras per foopy (the new foopies are more capable)

says that we need 10.6 new foopies

we already have 4 that were delivered in the previous push, so we need a total of 6 new mini's
We have minis from scl1 that are currently in mv because they have stripped chassis screws that need to be removed (and we need to get the asset tag info, serial numbers, and MACs off of them).  We can re-purpose 6 of those to match the other foopies we just set up.
Assignee: server-ops-releng → mlarrain
10.6 - 4 = 6.6, which means *7* new foopies, right?
^^^^ this number pops  up *way* too often
Also, I racked *six* foopies during allhands, not four.. or was there a further calculation done there?
(In reply to Dustin J. Mitchell [:dustin] from comment #3)
> Also, I racked *six* foopies during allhands, not four.. or was there a
> further calculation done there?

there was - of those 6 you racked, 2 were immediately used for the 40(ish) tegras that were needing a foopy.

yea, the number should be 7, i'll adjust the title
Summary: 6 new foopies needed for the pending 160 tegra install → 6^H7 new foopies needed for the pending 160 tegra install
Summary: 6^H7 new foopies needed for the pending 160 tegra install → 7 new foopies needed for the pending 160 tegra install
These are on the way to mtv1 with digipengi right now, and will be racked in 2.IDF with the 6 I installed during all-hands.
This math assumes rev4's.

rev4s can handle ~15 tegras per.
rev2s can handle 8-10 tegras per.

We have 4 rev4's delivered.
160 - (4 * 15) = 100

If we're going rev2's we need 10-12.5 rev2 foopies.
In case you didn't see my comments in irc yesterday... To be clear, the oldest gen mini we can use is r3.  The r2s are already out of Applecare warranty, and we're not talking about a single host going down if we lose one.  Each foopy would be handling 10 tegras and needs to be considered a core part of the infra like the buildbot masters.  I'd also like to point out that any r3s that we use for this have likely had a rough life as builders or testers and are much more susceptible to hardware failure from constant power cycles (we already experience this pretty regularly with the r2s and have been taking them out of service as they fail).
As of the IRC conversation yesterday, getting these installed is on hold until we hear a specific request.  The 7 minis, with Sonnet chassis and power, are in mtv1 under matt's desk.

Matt, when you're at your desk, if you can enumerate the asset tags here in the bug, I'll update inventory.
(In reply to Dustin J. Mitchell [:dustin] from comment #8)
> As of the IRC conversation yesterday, getting these installed is on hold
> until we hear a specific request.  The 7 minis, with Sonnet chassis and
> power, are in mtv1 under matt's desk.
> 
> Matt, when you're at your desk, if you can enumerate the asset tags here in
> the bug, I'll update inventory.

As per meeting with IT:

* we'll use rev4 minis as foopys for now. Once we have linux foopies available (bug 692688) and a 10.7 tester image ready, we can recycle these as 10.7 testers.
Let me know the asset tags and I'll update inventory
Per bear:

[16:28:19] <bear|buildduty> we don't need 7 protofoopies
[16:28:29] <bear|buildduty> so if that moved down to 3 I won't mind

this is going down to 3.  The other 4 will go to abillings in bug 694703.

I can has inventory datas kplzthx?
Summary: 7 new foopies needed for the pending 160 tegra install → 3 new foopies needed for the pending 160 tegra install
Severity: normal → minor
We have a substantial amount of capacity left on the foopies we already deployed into mtv1, so allocating more will not be necessary.  Please obtain the inventory information for these three minis and send them back to scl1 to join the general pool.
Summary: 3 new foopies needed for the pending 160 tegra install → install three remaining r4 (mtv1) minis in scl1
Severity: minor → normal
I have the inventory data for these now.  Please let me know where they end up.
colo-trip: --- → scl1
Okay, the following minis are destined to go back to scl1 and join the releng pool:

05328 lion-testing1
06201 lion-testing2
06244 foopy18
06051 foopy19
06236 foopy20

One of these needs to become talos-r4-lion-ref.
Summary: install three remaining r4 (mtv1) minis in scl1 → install five remaining r4 minis in scl1
Duplicate of this bug: 697192
the following additional minis have been added:

talos-r4-snow-081
talos-r4-snow-082
talos-r4-snow-083
talos-r4-snow-084
talos-r4-snow-085

talos-r4-lion-081
talos-r4-lion-082
talos-r4-lion-083
talos-r4-lion-084
talos-r4-lion-085
Assignee: mlarrain → arich
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Component: Server Operations: RelEng → RelOps
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.