On our Mac Minis  (located on the 2nd floor in Mountain View), we're seeing really slow throughput and connection speeds to various hosts. The weird thing is that nslookup, ping, traceoute, and dig all seem fine -- that is, doesn't seem to be DNS. http://screencast.com/t/gBvRb5wu captures some of that.  https://wiki.mozilla.org/QA/Execution/Web_Testing/Automation/Selenium_Grid#Architecture What can I try to troubleshoot?
I worked on this with :stephend on irc. we are working on resolving the root cause currently.
isolated the main issue. when we finished hooking in phx1 into our mpls network, we did not adequately account for the increased MTU costs of the transit traffic. after adjusting them on our border routers and removing the mss tweak in mtv1, verified traffic is flowing properly.
Verified FIXED; thanks again, Casey!