612299 - please reimage linux-ix-slave08

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

linux-ix-slave08 has a broken /builds filesystem.  It has been comming back online because we aren't able to fully shut it down properly.

Please reimage this machine.  Not sure if it needs to move to scl, but now would be a great time if it does.

Comment 1

[Justin Dow [:jabba]]

I was hoping to move all of linux-ix-slave01 through 16 at the same time. But I can go ahead and re-image it here at castro for now.

Comment 2

[John Ford [:jhford] CET/CEST Berlin Time]

ok

Comment 3

[Justin Dow [:jabba]]

Ok, looks like the image server with the images and the ability to image these boxes is in SCL. How soon can we move these servers to scl? Given that we can't really proceed until we either move these servers to SCL, or spend a few hours getting a pxe environment working in mountain view, I'd prefer to hold off and just shoot for sometime this week or next week to move these servers.

Comment 4

[[e-mail] (:bhearsum)]

That sounds fine, aren't we waiting on shelves before we can move them, though? From the releng side, there's nothing blocking moving them.

Comment 5

[Justin Dow [:jabba]]

I think zandr is coordinating things at SCL. I'll put this in his queue for now. Once things are moved over, imaging this should be as easy as booting from the network and following a few prompts.

Comment 6

[Lukas Blakk [:lsblakk] use ?needinfo]

Is there an ETA on this?

Comment 7

[matthew zeier [:mrz]]

Can onsite (or remote - IPMI works) releng help reimage?

Comment 8

[Zandr Milewski [:zandr]]

ipmi works at 10.250.51.224

It wants to PXE. Do we not have the servers to reimage here?

Comment 9

[Justin Dow [:jabba]]

This server needs to get moved before pxe will work.

Comment 10

[Zandr Milewski [:zandr]]

Right, so it will be moved in the Great Downtime on 12/17 if not sooner.

Comment 11

[matthew zeier [:mrz]]

But that's not one of the 42 that has to move - it's one of the 50 that's going to stay.  Or are you shuffling things differently?

Comment 12

[Zandr Milewski [:zandr]]

mrz: Per email conversation with release:

>I've been under the impression that all of the "w32-ix-slaveNN" and "linux-
>ix-slaveNN" machines in Castro are moving to Santa Clara, and that "mv-moz2-
>linux-ix-slaveNN" and "mw32-ix-slaveNN" will be staying.

Comment 13

[Justin Dow [:jabba]]

that is how I understood it as well. that has been the plan all along and if not, we need to rework inventory, etc.

Comment 14

[Zandr Milewski [:zandr]]

Moving to scl1 tomorrow AM.

Comment 15

[Zandr Milewski [:zandr]]

Reimaged, and now looping trying to talk to puppet. I see a few machines in this state, and I'm not sure what to do about them.

Comment 16

[Mike Taylor [:bear]]

(In reply to comment #15)
> Reimaged, and now looping trying to talk to puppet. I see a few machines in
> this state, and I'm not sure what to do about them.

any you find looping for puppet you can pass them on to buildduty type person - we can clear them from puppet's list

Comment 17

[Mike Taylor [:bear]]

realized I moved this to releng when there was already a releng bug covering it - closing