The BF3 server requires an always-on connection to Blaze, which is the authentication and stats back-end for the game. It constantly sends data to and from Blaze -- when every player joins and leaves and whenever the round ends, for instance -- so it quickly notices if there is a problem with this connection. Whenever there is
a problem with Blaze, the game dumps out all of the players in the server.
Blaze is a cluster of machines located at a provider in the UK. Many situations can cause the connection to Blaze to fail, including:
- One of the machines within Blaze failing, or the entire cluster failing
- Blaze internally becoming out of sync
- The ISP that runs Blaze having connectivity or hardware problems
- Packet loss along the path between us and Blaze, which is thousands of miles long and involves many routers and other equipment, as well as multiple NSP interconnections (peering points)
- Blaze maintenance by EA (this is typically performed every Tuesday morning between 8am and 10am PST)
- Maintenance by an NSP (backbone provider) that we were not made aware of (this typically occurs in the late evening or early morning, local time to the equipment being worked on)
- Scheduled maintenance by InterNAP or NFO (if this occurs, you'll see a maintenance note posted to your control panel "Events log" page, or it will be listed as a part of regular morning maintenance)
- A DDoS attack against the individual server or a very large one that takes out the entire NFO location (if an overall location attack occurs, you'll see a DDoS note posted to your control panel "Events log" page)
Most often, we see individual servers lose their connection to Blaze because of internal Blaze issues and server-specific DDoS attacks (which are becoming more and more common).
We do our best to control every factor that we can possibly affect and immediately respond to each emergent situation that we see. We have overall and machine-specific BF3 and bandwidth usage graphs that we review frequently to look for problems, and we have automated monitoring systems that notify us when any part of our network goes down.