1170212 - Decommission symbols{1,2}.dmz.phx

Status: RESOLVED FIXED

(Infrastructure & Operations :: Virtualization, task)

Description

[(not currently active) Ted Mielczarek]

Once we close all the deps of bug 1071724 and have everything uploading symbols directly to the Socorro symbol API we can decommission symbols{1,2}.dmz.phx.

(If this isn't the right component feel free to move it.)

Comment 1

[Greg Cox [:gcox]]

bug 1071724 comment 11 says "More work to do in dependent bugs, but in our current state does not block AWS move.  It does block turning off symbolpush".

Is this decom still blocked?
  (if so, can we get a new blockerbug attached for ease of go/nogo)?

Comment 2

[(not currently active) Ted Mielczarek]

Well, we turned off syncing from the NetApp to S3, so I guess that ship has sailed! We might still have some stragglers that need to migrate to the new setup (I was contacted by the Fedora distro maintainer, and I think I saw a bug about a B2G partner), but these machines are not actually doing anything useful now so we might as well get rid of them.

Comment 3

[Greg Cox [:gcox]]

OK then!
10.8.74.48 = symbols1.dmz.phx1
10.8.74.88 = symbols2.dmz.phx1
Has NFS, no netvault.

Pulled from nagios in change 107230, powered off.  Per standard practice we'll leave the VMs powered off but untouched.  If it causes issues we can power back on pretty quickly.  In a week or so we'll delete them and they won't be recoverable.. but you can ask for more time during that week.

Comment 4

[(not currently active) Ted Mielczarek]

Per an email thread, one of our B2G partners hadn't switched over to the new upload API. Can we power these back on until we get that resolved? (I know rhelmer turned off the auto-sync script, so we'll probably need to do a one-off sync if they upload via SSH.)

Comment 5

[Greg Cox [:gcox]]

Powered back on.  Did not revert nagios back to monitoring them, under the assumption this is temporary.

If there's bug for the partner to get fixed, please mark it as blocking this bug.

Comment 6

[Robert Helmer [:rhelmer]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #4)
> Per an email thread, one of our B2G partners hadn't switched over to the new
> upload API. Can we power these back on until we get that resolved? (I know
> rhelmer turned off the auto-sync script, so we'll probably need to do a
> one-off sync if they upload via SSH.)

I did indeed turn off syncing, just lmk if you need it back on or if you want me to do a one-off sync.

Comment 7

[Greg Cox [:gcox]]

VMs still needed?  If so, is bug 1130138 an appropriate blocker?

Comment 8

[(not currently active) Ted Mielczarek]

(In reply to Greg Cox [:gcox] from comment #7)
> VMs still needed?  If so, is bug 1130138 an appropriate blocker?

Yeah, that's the right blocker bug. I think we only have the one B2G partner that was using this upload endpoint. I'll help get them moved over so we can close this out for good.

Comment 9

[Lonnen :lonnen]

All blocker's out. This is ready to decomm

Comment 10

[Greg Cox [:gcox]]

ack, will pull them on Monday after I'm back from PTO.

Comment 11

[Greg Cox [:gcox]]

Powered off.  Entering hold for complaints until Friday.

Comment 12

[Greg Cox [:gcox]]

inventory, dns, rhn, puppetdashboard done.
zeus, Will file for webops due to the evacuation of phx1.

Removed node declaration from puppet (change 109638), continued not-removing module definitions.
Nothing in newrelic.
Will file netops changes.
VMs deleted from disk.
That removes the last users of the symbols NFS volume in phx1.  unmounted and offlined that, will destroy soon.