1239033 - Disable FTP ahead of decom'ing it

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Product Delivery, task)

Description

[Greg Cox [:gcox]]

Per emails with :travis, disable zeus pointing back to ftp / upload.
CC'ing him for an ack and timing.

Comment 1

[Eric Ziegenhorn :ericz]

The following VIPs look to be related to FTP and uploads.  Please ack which of these should be disabled.

pvtbuilds.pvt.build.mozilla.org
releases-zlb.vips.scl3.mozilla.com
upload-zlb.vips.scl3.mozilla.com
download-origin-zlb.vips.scl3.mozilla.com
ftp1-zlb.vips.scl3.mozilla.com
ftp-ssl-zlb.vips.scl3.mozilla.com
partnerbuilds-zlb.vips.scl3.mozilla.com
releases-zlb.vips.scl3.mozilla.com

and I assume ssl-ov-mozqa-zlb.vips.scl3.mozilla.com is not involved here.

Comment 2

[Eric Ziegenhorn :ericz]

To aid in identifying these VIPS, I've dug up these cnames in DNS:

upload-zlb.vips.scl3.mozilla.com:
  china-sync.mozilla.org
  pv-mirror01.mozilla.org
  pv-mirror02.mozilla.org
  stage.mozilla.org
  surf.mozilla.org

download-origin-zlb.vips.scl3.mozilla.com:
  download-origin-scl3.cdn.mozilla.net

Comment 3

[Corey Shields [:cshields]]

go ahead and disable these, per email from travis on 1/12

Comment 4

[Eric Ziegenhorn :ericz]

VIPs that have traffic recently:  ftp-ftp ftp.mozilla.org download.mozilla.org-http ftp.mozilla.org-ssl releases-http releases-https partnerbuilds-https pvtbuilds.pvt.build.mozilla.org upload-ssh
VIPs that do not have recent traffic:  ftp.mozqa.com

Comment 5

[Eric Ziegenhorn :ericz]

DNS for the ftp.mozilla.org, releases.mozilla.org and downloads.mozilla.org all points to AWS.

Comment 6

[Eric Ziegenhorn :ericz]

Needinfo'ing oremj for any insight and make sure all the right things / none of the wrong things get shut down.

Comment 7

[Jeremy Orem [:oremj]]

I'd like to point stage.mozilla.org at the AWS stack in case anyone is still using that name.

Nick, is anyone using the name surf.mozilla.org?

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

surf.mozilla.org can get definitely be removed, it's very old now. And I can't find any reason for stage.m.o to still be the old system, and it looks like webops made requests there a 301 to archive.

Comment 9

[u429623]

Gah! Because we hadn't made any change to motd or the like to prevent writing, I just uploaded a 16G file assuming the sync was still active. (I'll move that to S3 today.)

While we debate dns names, can we update the MOTD so folks don't write more stuff there?

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

+1 to that.

Comment 11

[u429623]

ftr, my new stuff is clear of upload1.dmz.scl3 /pub/ (aka 10.22.74.157:/ffxbld) I.e. comment 9 is not a blocker.

Comment 12

[Eric Ziegenhorn :ericz]

Disabled the following vservers for FTP and upload as per this bug and discussion on IRC:

ftp-ftp
ftp-http
ftp-https
upload-http
upload-ssh

FWIW, there are other things that use the same backend servers as per comments #1 and #2.

Comment 14

[Greg Cox [:gcox]]

Attachment: SVN diff for nagios changes

Peeled out the nagios checks for the above items, which were in alarm.

Comment 15

[Greg Cox [:gcox]]

Sorry, but I have to reopen this.

Anything still using the ftp / upload clusters either needs to be shut off at the VIP or to have someone say "we need the VIP to stay up but we're not using anything from the NFS mount."

Given that we've gotten permission in comment 3 I'm inclined to turn things off and let the chips fall.  Someone talk me off the ledge.

Comment 16

[Eric Ziegenhorn :ericz]

Ok, I will shut off more things using the FTP/upload clusters today and their corresponding Nagios alerts to see if anything/anyone complains.

Comment 17

[Eric Ziegenhorn :ericz]

Remaining VIPS pointing to ftp/upload pools shut down and Nagios taken care of:

download-origin
partnerbuilds
pvtbuilds
releases

Comment 18

[Eric Ziegenhorn :ericz]

For future reference when we decomm this stuff, notes have been added to all the Zeus VIPS I disabled and ftp.dynect.mozilla.net needs to be cleaned up as well.

Comment 19

[Eric Ziegenhorn :ericz]

The disabling was done 3 weeks ago and I'm not aware of any problems or anything being turned back on so calling this done so as not to block 1253390.