Closed Bug 684253 Opened 9 years ago Closed 9 years ago

need more 2 foopies for current tegra pool in addition to 4 for new foopies

Categories

(Release Engineering :: General, defect)

ARM
Android
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bear, Assigned: bear)

References

Details

(Whiteboard: [android][tegra])

Attachments

(2 files, 1 obsolete file)

With the increase in test and project coverage, the working directory for a tegra build has increased from an average of 300M to 450M.

The foopies have always had small(ish) drives, so this increase means that the 13 tegras per foopy are now running out of space regularly (even with an aggressive cronjob to cull logs and temp files)

I would like to reduce the tegra count to 10 per foopy and that would mean find a new home for 18 Tegras.

18 Tegras would be 1.8 foopies
40 new Tegras would be 4 foopies

So we would need to bring online 6 foopies to handle the new and current requirements.
Whiteboard: [android][tegra]
Going to pull 6 rev4's from scl1 for this. This will also give me a sense of how much load a foopy puts on modern hardware.
Assignee: server-ops-releng → zandr
Blocks: 669415
Matt, bring back 6 minis from scl1, and we'll set the up in mtv1 (dns, dhcp, nagios) for this.
Assignee: zandr → mlarrain
I have brought the 6 mini's back to my desk in MTV as soon as I have back in the office I will rack them on the second floor.
Status: NEW → ASSIGNED
These are racked and running.  They have IPs from DHCP and are in inventory.  They should appear in DNS shortly (I just committed).

Bear, you can start setting these up, but I'll reassign to Amy for the moment to get nagios running.
Assignee: mlarrain → arich
Amy I don't think has any known good setup for nagios on foopies because nagios doesn't run on foopies and that's a bug i'm supposed to be fixing ;)

but yea, I'll start chewing on this - exact time to be posted when I figure out my time availability
Bear: When it comes to installing and configuring nrpe on OS X, it's already done via puppet on other releng servers, so it's just a matter of getting the foopies puppetized and using that bit of code, I think.
Assignee: arich → bear
I added ping checks to nagios.

I also ssh'd to all 6, so they're all up :)
Assignee: bear → nobody
Component: Server Operations: RelEng → Release Engineering
QA Contact: zandr → release
Assignee: nobody → bear
Oh, I missed a critical detail, the hostnames:

 foopy12.build.mozilla.org
 foopy13.build.mozilla.org
 foopy14.build.mozilla.org
 foopy15.build.mozilla.org
 foopy16.build.mozilla.org
 foopy17.build.mozilla.org

:)

Also, to save you a trip to inventory, these are mounted in 2.IDF in mountain view, at the top of the rack with all the tegras in it.
figure we should be storing these and tools/buildfarm/mobile/ looked to be a good spot.

these are what i've been working on the new foopies with and i've updated https://wiki.mozilla.org/ReferencePlatforms/Test/Foopy to reference them
Attachment #561690 - Flags: feedback?(aki)
Attachment #561690 - Flags: feedback?(aki) → feedback+
Attachment #562043 - Flags: review?(aki)
Comment on attachment 562043 [details] [diff] [review]
add new tegras to buildbot-configs

Hm.
It's good keeping things contiguous, but this means any try jobs running at the time of the reconfig will die.

If we keep 65,95 as try and add the new tegras in either pool, then the reconfig will have no negative effect.
Attachment #562043 - Attachment is obsolete: true
Attachment #562043 - Flags: review?(aki)
Attachment #562132 - Flags: review?(aki)
Comment on attachment 562132 [details] [diff] [review]
add new tegras to buildbot-configs

(Still not completely sold on this split, but that's not for this patch. r=me)
Attachment #562132 - Flags: review?(aki) → review+
Comment on attachment 562132 [details] [diff] [review]
add new tegras to buildbot-configs

committed changeset 4794:3961a546f29c
Attachment #562132 - Flags: checked-in+
Comment on attachment 561690 [details] [diff] [review]
scripts to setup a foopy after it has been imaged

committed changeset 1821:082274864dd6
Attachment #561690 - Flags: checked-in+
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.