782640 - Investigate why bld-centos6-hp-002 doesn't boot

Status: RESOLVED DUPLICATE of bug 779487

(Infrastructure & Operations :: DCOps, task)

Description

[Rail Aliiev [:rail]]

I tried to reboot bld-centos6-hp-002, but it's stuck at this screen:

http://people.mozilla.org/~raliiev/sattap/1d21b40d.png

Could you please investigate or just reimage the machine?

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Per email from arr, I think dmoore's group handles issues like this now.

Comment 2

[Hal Wine [:hwine] use NI!]

this is the 3rd hp in recent time with boot problems (see bug 779487) - tagging for deeper look

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Yup, if you hit enter there it says "Non system disk or disk error".  I tried doing a puppet reinstall, and it didn't help.

Comment 4

[Vinh Hua [:vinh]]

Similar to the other hosts on https://bugzilla.mozilla.org/show_bug.cgi?id=779487, after recreating the logical raid drives system appears to boot up fine. Closing ticket.  If any other issues arise, please let us know.

-Vinh Hua

Comment 5

[Hal Wine [:hwine] use NI!]

(In reply to Vinh Hua [:vinh] from comment #4)
> Similar to the other hosts on
> https://bugzilla.mozilla.org/show_bug.cgi?id=779487, after recreating the
> logical raid drives system appears to boot up fine. Closing ticket.  If any
> other issues arise, please let us know.

Vinh - with a total of 5 hosts (4 more in bug 779487) experiencing this in a short period of time, can you explain what went wrong, with an eye towards avoiding the issue on the other hp machines we have in use?

We haven't closed bug 779487 yet, so I'm unclear what is different about this host.

Comment 6

[Van Le [:van]]

I've ran across this issue before while working at Yahoo. It turned out to be a firmware issue with the raid controller and drives when there was a lot of load. The hardware we used, although not the same are related (HP raid controller, HP branded drives). We had to manually update the firmware on both drives and controller.

Comment 7

[Van Le [:van]]

:hwine, can we close this bug as a duplicate of Bug 779487? The SRE team is handling the root cause in that bug.

Thanks,
Van