1174947 - remove proxy arp from wireless configuration

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: NetOps: Other, task)

Description

[Van Le [:van]]

users have been experiencing issues where they don't receive an ipv4 address or if they do, they can't ping their gateway. ipv6 is persistent during this time so we believe proxy arp is at play. the configuration is only 1 line but may impact users so we'd like to perform this on a weekend and also before whistler so any lingering DHCP leases will expire.

date, time, duration of maintenance
6/20/2015, ~10am, 8 hours.

system(s) affected
wireless globally

end-user impact
users may needs to renew their dhcp lease

maintenance plan and timeline (link to a wiki or etherpad is fine)
1) arrive in MTV2, log into Mozilla Guest SSID with spare laptop
2) remove proxy arp configuration from Mozilla Guest SSID in NA cluster.
3) check to see if i lost network connectivity on spare laptop(make note). if i did, renew dhcp to see if i regain network connectivity. 
4) if everything works as expected, remove arp cache from the rest of the Mozilla SSIDs and test.

rollback plan / rollback point (at which point will you determine to roll back)
reenable proxy arp configuration

notification mechanisms
Whistle Pig

who will be point, who else will be involved
Van

Comment 1

[Van Le [:van]]

err... arp cache should read proxy arp in step 4. 

also if the NA cluster's change is successful, i will proceed to deploy the configuration change to all clusters as we've had reports of this issue in Berlin and Paris.

Comment 2

[Ashish Vijayaram [:ashish]]

https://whistlepig.mozilla.org/detail/361/ is the whistlpig notification. On completing this maintenance, please update or request the MOC oncall to update notification status to Resolved.

Comment 3

[Linda Ypulong [:unixfairy]]

Reviewed by CAB 6/17 - approved

Comment 4

[Van Le [:van]]

brought in 3 laptops - my osx to configure, 1 linux and 1 windows to test.

put linux and windows laptop on mozilla guest ssid but on different aps. removed proxy arp and noticed the windows laptop dropped 1-2 ping and the linux laptop up to 4. rinse and repeat for all the other ssids up to mozilla ssid. laptops kept their ips, but would drop 1-4 ping as the forwarding table was updated. 

removed proxy arp from mozilla ssid while on mozilla ssid and noticed my macbook kept all its ssh connections as well as stayed on vpn. pinged bob clary to test his autophones and he didn't report any issues. performed several roaming tests on the laptops and while i dropped a 1 or 2 ping packets here and there while reassociating, didn't drop my ssh connections or notice any issues. we'll have more data when people come back from whistler.

going to push configs to the other clusters.

Comment 5

[Van Le [:van]]

configs were pushed to other clusters.