698888 - downtime request for scl1 intra-DC fixes

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

As discussed in the relops meeting, we'd like to plan at least one tree closure to work on the links in scl1.  The details of what is to be done are not fully worked out yet, but the general outline is:

 - add new interfaces on the SRX650's
 - switch some traffic over to them
 - switch the primary uplink to P2P

with testing and verification along the way.  It's likely we could do most of this without a tree closure, but if the trees are closed anyway, we'll do disruptive testing.

We may want a second tree closure one or more days later to do the P2P change, although we will test its failover during the first downtime.

For the moment, let's start by scheduling one downtime.  Parameters I know of (please correct if these are wrong):
 - 3h
 - not during a release

The pacific evening of November 10th was mooted as a possibility, although ravi is in NZ that week, so that may make the timing a little funny.  Discuss.  Negotiate.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

16:58 < dustin> thurs at 6pm pacific = fri at 3 pm auckland -- work for you?
16:59 < dustin> or thereabouts
16:59 < ravi> yep
16:59 < dustin> kk

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Coop ok'd the above time (6pm pacific) via email/IRC.

Comment 3

[Mike Taylor [:bear]]

downtime was completed with very minor blips, more detailed look at the logs needs to happen to know for sure.

calling this a win - dustin to file a tl;dr report along with a longer detailed one later

thanks Rel|Net Ops!

Comment 4

[Mike Taylor [:bear]]

ick, forgot to record the duration:

trees closed at 1800 PST and reopened at 2130 PST (overage due to bear being extra cautious)