Closed Bug 600899 Opened 9 years ago Closed 9 years ago

Install resistor dongles on all Snow Leopard minis

Categories

(mozilla.org Graveyard :: Server Operations, task, major)

All
Other
task
Not set
major

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: joe, Assigned: jlaz)

References

Details

Unfortunately, we have discovered after installing a dongle on a single Snow Leopard computer that we can't use hardware acceleration on these computers without resistor dongles.

This will need to be done as an atomic operation; because of the performance implications of hardware acceleration, we can't install the dongles piecemeal. It'll either require us to remove machines from the pool and donglify them, then switch to these machines and donglify the rest, or turn off all 10.6 test machines, install the dongles, then turn them back on.
This needs to be done soon, but not necessarily this-week soon.
IT, per Platform meeting today we need this in the next week or two.

* How many dongles do we currently have?
* How long of a downtime window would you guys need?
* When could we schedule this?

AFAIK there are 55 snow leopard testing machines.
Severity: normal → major
Assignee: server-ops → jlazaro
Flags: colo-trip+
We have plenty of dongle ingredients, but it would be good if we can pay someone to solder them together. Just plugging resistors into dongles isn't reliable as they fall out easily.
Good to know this information. We want those dongles to stay together *forever* :)

Any known contractor that has done this for us previously in the Bay area?
Who could be point of contact from IT to coordinate this?
zandr said he knew a guy. Not sure how soon he can make this happen. zandr is out of the office this week.

Sean, could you order 20 each of the products in comment 4, please?
Any updates?
20 of each of the components mentioned in comment 4 are on order and will be here this week.
Whiteboard: On order
Checking on 'the guy', I'll comment here when I hear back.

Is there a bug that describes the actual problem? If I can get something worthy of filing in RADAR, I can friends at Apple to wave it around under the right noses.
Justin, I dropped the items from monoprice off at your desk.
Assignee: jlazaro → server-ops
John, could you and Zandr connect next week to discuss if Zandr's contact would be able provide a solution for this issue in a timely manner?  If not, we'll most likely manually create these again.
Assignee: server-ops → jlazaro
Assignee: jlazaro → jdow
I've got the resistors soldered to connectors. I can install these when needed. joduinn mentioned to me yesterday that a downtime is needed for this. It will be difficult if all of them need it at the same time, since I think currently half the snow leopard minis are in Castro and half in Internap. Let me know how you want this handled.
jabba what do you think the ETAs for each option would be?
What do you think is safer and what do you think is more comfortable to do?
Can you think of anything that releng could do to make this easier for IT?
Is weekend or early PDT options available?

joe how are we doing with schedules?
Whiteboard: On order → [waiting for downtime coordination]
No particular hurry on schedules.
Setting needs-treeclosure flag as we will have to shut all the snow leopard minis down.

Everything is ready. We just need the time slot. A couple of hours to be safe.

releng should be around to make sure the machines come back to normal and trigger few jobs before re-opening the tree.

Jabba and I figured out that unlike what was mentioned in comment 12 only 8 minis are left in Castro. This makes things easier.
Flags: needs-treeclosure?
Jlazaro: I'm going to be out most of this week, so maybe you can coordinate this. If RelEng has a downtime this week that can be used to move minis (i.e. are shelves there yet?), then this can piggy-back onto that downtime. There are about 100 MozillaDongle Mark III (TM) in the office. Id suggest putting them on all Snow Leopard minis and use the 45ish left over to replace MozillaDongle Mark II (TM) with Mark III (TM) on the Leopard minis. There won't be enough for all, but it will be enough for most. The rest can live with the Mark II dongle.
Assignee: jdow → jlazaro
Ready to get these installed.  Waiting for a downtime, unless we want to get these installed before 9AM (not sure when the best time is, 6AM?) during the weekday.
joduinn and jhford-buildonduty could please one of you coordinate the time/day with IT?

any differences with MII and MIII that could affect the minis' performance?
None.  III is soldered on.  II isn't.
jabba has a box of these resistors and video connectors, ready to go.
Whiteboard: [waiting for downtime coordination]
Flags: needs-treeclosure? → needs-treeclosure+
these talos-r3-snow machines were at 650 castro and now have a dongle:

013
014
015
016
017
018
019
020
<dmoore> jhford-buildduty: ok, starting on the snow leopard minis
<jhford-buildduty> cool
<joduinn> dmoore: we're doing the snowleopard minis here in 650castro 
<jhford-buildduty> ok, got the ones in 650 done
<joduinn> dmoore: we have 2 spare dongles. hope you are not short !!
<dmoore> jhford-buildduty: joduinn: finished, with a few spares on this end as well
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Nice work, gents. Quick, too! Let's get the boxes back up and running!
That was easy (http://bit.ly/di2hgh)!

Thanks guys!
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.