794184 - bring mac minis to certified tech in desktop

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Van Le [:van]]

talos-r4-lion-058 - host had memory issues and ram was replaced but looks like issues are persisting. will need to bring to apple certified tech in desktop to rerun diagnostics.

talos-r4-snow-030 - bad onboard NIC, confirmed switch port and cable works fine. will need to bring to apple certified tech to swap out motherboard if possible.

talos-r3-xp-048 - dead drive. if desktop can pick up the exact hard drive to replace, we can replace the drive otherwise we'll have to retire.

talos-r4-snow-050 - bad hardware, system does not stay powered on - flickers on and off.

Comment 1

[Salvador Espinoza [:sal]]

Dropped off the minis in the desktop area.

Comment 2

[Vinh Hua [:vinh]]

Lion-058:  Ran diagnostics several times again but did not find any errors. Will put back into production.

Snow-030:  Has bad onboard NIC. Hardware is out of warranty. Need to decommission.

XP-48:  Has bad hard drive. Will replace HDD with spare one in stock.

Snow-50:  Has bad logic board. Hardware is out of warranty. Need to decommission.

Comment 3

[Amy Rich [:arr] [:arich]]

Now that we have a certified tech, what does it cost us to repair those machines (presumably we only have to purchase parts)?

Comment 4

[Vinh Hua [:vinh]]

Hard drives and memory can be easily replaced.  As for other hardware parts, it's all integrated onto the motherboard. Replacing the motherboard is tedious on any apple products.

Comment 5

[Amy Rich [:arr] [:arich]]

Unfortunately, tedium doesn't factor into this much.  :}

We can't replace these, so we need to know what it costs for parts to fix them.  

This was part of our whole request of having a certified apple tech in house (so we had someone who could do the repairs as opposed to us sending it out to a third party).

Comment 6

[Vinh Hua [:vinh]]

talos-r3-xp-048 - Hard drive has been replaced and reimaged.

vhua$ sudo fping talos-r3-xp-048.build.scl1.mozilla.com
talos-r3-xp-048.build.scl1.mozilla.com is alive

[snow-30 & snow-50] - The cost to replace the logic board on the r4 mac mini's is $591.77 each.  

Let me know if you want to proceed with ordering the replacement boards.

Comment 7

[Amy Rich [:arr] [:arich]]

releng folks: We can't get any more r4s.  Should we pay the $600 to repair these two?

Comment 8

[Hal Wine [:hwine] use NI!]

I'm on the hook to get you an answer - please let me know if there are any deadlines we're approaching.

Comment 9

[Amy Rich [:arr] [:arich]]

No deadlines, no (since we're out of warranty), it just means that the machines sit out of commission longer.

Comment 10

[Amy Rich [:arr] [:arich]]

Hal, what was the verdict on this?  Are we paying to repair the r4s?

Comment 11

[Hal Wine [:hwine] use NI!]

Yes - we'll repair these 2. Not a policy yet, though.

Comment 12

[Amy Rich [:arr] [:arich]]

Can someone form desktop or dcops please confirm that these two machines have been sent for repair?  When will they be back in service again?

talos-r4-snow-030.build.scl1.mozilla.com
talos-r4-snow-050.build.scl1.mozilla.com

Comment 13

[Vinh Hua [:vinh]]

Amy,
I will have desktop order the replacement logic boards today.

Comment 14

[Vinh Hua [:vinh]]

Logic boards for the mac minis have been ordered today by Patrick Dang.

Comment 15

[Vinh Hua [:vinh]]

Snow-30 and snow-50 as been repaired and re-racked.  Since both logic boards were replaced, they now have new  nic MAC addresses.  

snow-30 = C4:2C:03:0A:D2:DC

snow-50 = C4:2C:03:0A:CB:C7

Comment 16

[Amy Rich [:arr] [:arich]]

Updated in inventory and added to deploystudio.  Do we need to reimage these, vihn?  If so, can you do that, please?

Comment 17

[Amy Rich [:arr] [:arich]]

They were up enough so that I could get into them, but they looked like they needed a reimage, so that's been done.  releng, snow-050 and snow-030 should be tested out.  I commented in 730545 since it looks like lion-058 got tested after the memory was replaced.

Comment 18

[Chris Cooper [:coop] (he/him)]

(In reply to Amy Rich [:arich] [:arr] from comment #17)
> They were up enough so that I could get into them, but they looked like they
> needed a reimage, so that's been done.  releng, snow-050 and snow-030 should
> be tested out.  I commented in 730545 since it looks like lion-058 got
> tested after the memory was replaced.

snow-030 and snow-050 are having dongle issues: bug 814260

Comment 19

[Vinh Hua [:vinh]]

I'm assuming we can close this bug out since all the bad mac minis have been addressed. There is a separate bug to address the dongle issues with snow-30 & 50.

Comment 20

[Amy Rich [:arr] [:arich]]

talos-r4-lion-058 still hasn't been addressed/fixed.

Comment 21

[Vinh Hua [:vinh]]

Ran a full diagnostics check on talos-r4-lion-058 using TechTool Pro and no issues were found. Host had passed hard disk, video, and memory checks.