I posted this today to the Events log for all Seattle servers.
Quote:
We have been troubleshooting higher-than-normal latencies to Comcast customers in the Seattle area from our Seattle locations for some time now. Since the beginning, the issue has been on Comcast's side; our outbound traces stay in Seattle, while their inbound traces go down through California. The result is that pings are about 20ms higher than they should be. We have attempted to remedy this through numerous adjustments on our end -- installing a router, adding additional transit and peering, changing our BGP settings, etc, with no luck.
Recently, we were finally able to get ahold of a couple of backbone engineers at Comcast, who let us know what the real problem was: Comcast is arbitrarily sending our traffic through California as part of widespread changes they are making to their nationwide network. They were worried about overloading their connection to AT&T in Seattle, so they shifted a chunk of traffic down through their California network segment, and we happen to be in that chunk.
The engineers also told me that within 2-4 weeks, Comcast will be adding a Level3 peer in Seattle, at which point our latencies would return to normal. But, we want a solution now, so we've gone out of our way to find one.
Today, we were assigned a new IP block from one of our upstreams, Mzima. This new IP block does not exhibit the problem from Comcast customers because it is not assigned by InterNAP, as our other blocks are. We have modified our systems to allow us to shift any customers that need lower Comcast latencies over to the new block temporarily, while we wait for Comcast to perform its network upgrades.
If you want to be moved to one of the new Comcast-friendly IPs, please send us an email ASAP (our Live chat staff cannot perform these moves). We do not recommend that public servers make this change, as the new IP would only be temporarily, and public servers suffer greatly from IP changes (due to clients losing track of the server).
We are also continuing to communicate with Comcast and attempting to get them to modify their network's routing behavior. If we succeed, the routing to all of our IPs may return to normal for Comcast customers.
Login
Register
FAQ
Search
