600899 - Install resistor dongles on all Snow Leopard minis

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Joe Drew (not getting mail)]

Unfortunately, we have discovered after installing a dongle on a single Snow Leopard computer that we can't use hardware acceleration on these computers without resistor dongles.

This will need to be done as an atomic operation; because of the performance implications of hardware acceleration, we can't install the dongles piecemeal. It'll either require us to remove machines from the pool and donglify them, then switch to these machines and donglify the rest, or turn off all 10.6 test machines, install the dongles, then turn them back on.

Comment 1

[Joe Drew (not getting mail)]

This needs to be done soon, but not necessarily this-week soon.

Comment 2

[Armen [:armenzg]]

IT, per Platform meeting today we need this in the next week or two.

* How many dongles do we currently have?
* How long of a downtime window would you guys need?
* When could we schedule this?

AFAIK there are 55 snow leopard testing machines.

Comment 3

[Justin Dow [:jabba]]

We have plenty of dongle ingredients, but it would be good if we can pay someone to solder them together. Just plugging resistors into dongles isn't reliable as they fall out easily.

Comment 4

[Justin Dow [:jabba]]

http://www.monoprice.com/products/product.asp?c_id=104&cp_id=10419&cs_id=1041912&p_id=4850&seq=1&format=2

http://www.showmecables.com/viewItem.asp?idProduct=1641

We need about 20 more of each of those products. We have plenty of resistors.

Comment 5

[Armen [:armenzg]]

Good to know this information. We want those dongles to stay together *forever* :)

Any known contractor that has done this for us previously in the Bay area?
Who could be point of contact from IT to coordinate this?

Comment 6

[Justin Dow [:jabba]]

zandr said he knew a guy. Not sure how soon he can make this happen. zandr is out of the office this week.

Sean, could you order 20 each of the products in comment 4, please?

Comment 7

[Armen [:armenzg]]

Any updates?

Comment 8

[Sean Alamares]

20 of each of the components mentioned in comment 4 are on order and will be here this week.

Comment 9

[Zandr Milewski [:zandr]]

Checking on 'the guy', I'll comment here when I hear back.

Is there a bug that describes the actual problem? If I can get something worthy of filing in RADAR, I can friends at Apple to wave it around under the right noses.

Comment 10

[Sean Alamares]

Justin, I dropped the items from monoprice off at your desk.

Comment 11

[Justin Lazaro [:jlaz] (use needinfo)]

John, could you and Zandr connect next week to discuss if Zandr's contact would be able provide a solution for this issue in a timely manner?  If not, we'll most likely manually create these again.

Comment 12

[Justin Dow [:jabba]]

I've got the resistors soldered to connectors. I can install these when needed. joduinn mentioned to me yesterday that a downtime is needed for this. It will be difficult if all of them need it at the same time, since I think currently half the snow leopard minis are in Castro and half in Internap. Let me know how you want this handled.

Comment 13

[Armen [:armenzg]]

jabba what do you think the ETAs for each option would be?
What do you think is safer and what do you think is more comfortable to do?
Can you think of anything that releng could do to make this easier for IT?
Is weekend or early PDT options available?

joe how are we doing with schedules?

Comment 14

[Joe Drew (not getting mail)]

No particular hurry on schedules.

Comment 15

[Armen [:armenzg]]

Setting needs-treeclosure flag as we will have to shut all the snow leopard minis down.

Everything is ready. We just need the time slot. A couple of hours to be safe.

releng should be around to make sure the machines come back to normal and trigger few jobs before re-opening the tree.

Jabba and I figured out that unlike what was mentioned in comment 12 only 8 minis are left in Castro. This makes things easier.

Comment 16

[Justin Dow [:jabba]]

Jlazaro: I'm going to be out most of this week, so maybe you can coordinate this. If RelEng has a downtime this week that can be used to move minis (i.e. are shelves there yet?), then this can piggy-back onto that downtime. There are about 100 MozillaDongle Mark III (TM) in the office. Id suggest putting them on all Snow Leopard minis and use the 45ish left over to replace MozillaDongle Mark II (TM) with Mark III (TM) on the Leopard minis. There won't be enough for all, but it will be enough for most. The rest can live with the Mark II dongle.

Comment 17

[Justin Lazaro [:jlaz] (use needinfo)]

Ready to get these installed.  Waiting for a downtime, unless we want to get these installed before 9AM (not sure when the best time is, 6AM?) during the weekday.

Comment 18

[Armen [:armenzg]]

joduinn and jhford-buildonduty could please one of you coordinate the time/day with IT?

any differences with MII and MIII that could affect the minis' performance?

Comment 19

[matthew zeier [:mrz]]

None.  III is soldered on.  II isn't.

Comment 20

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

jabba has a box of these resistors and video connectors, ready to go.

Comment 21

[John Ford [:jhford] CET/CEST Berlin Time]

these talos-r3-snow machines were at 650 castro and now have a dongle:

013
014
015
016
017
018
019
020

Comment 22

[John Ford [:jhford] CET/CEST Berlin Time]

<dmoore> jhford-buildduty: ok, starting on the snow leopard minis
<jhford-buildduty> cool
<joduinn> dmoore: we're doing the snowleopard minis here in 650castro 
<jhford-buildduty> ok, got the ones in 650 done
<joduinn> dmoore: we have 2 spare dongles. hope you are not short !!
<dmoore> jhford-buildduty: joduinn: finished, with a few spares on this end as well

Comment 23

[Mike Beltzner [:beltzner, not reading bugmail]]

Nice work, gents. Quick, too! Let's get the boxes back up and running!

Comment 24

[Armen [:armenzg]]

That was easy (http://bit.ly/di2hgh)!

Thanks guys!