Anycast

Reading time: 11 minute Word count: 2265

Network Anycast IP Routing BGP DNS CDN

When the same service is deployed in many places, the usual approach is to give each node a different address and let DNS, a configuration center, or an upper-layer scheduler decide which one the user should connect to. That can work, but it has a clear cost: the access decision happens outside the network, so very often the name has already been resolved before the user discovers that they were sent somewhere suboptimal.

Anycast solves a different problem: “the same entry point should appear in many places.” It is not a secret extra forwarding layer, and it is not a dedicated Layer-7 load balancer. It lets multiple sites announce the same address, and then lets the existing routing system bring the client to the instance that is currently the nearest, easiest to reach, or most preferred by policy.

The core of Anycast is not “one address hanging off many machines.”
It is that the same address is reachable from multiple places at the same time,
and the question of which instance to reach first is left to routing topology and policy.

Why It Exists

If a large public-facing service only has one access point, it quickly runs into practical problems:

The user is too far from the entry point, so first-packet latency is high
Cross-carrier or cross-region path quality varies a lot
If one data center or one exit fails, all traffic is affected at once
Entry switching depends on upper-layer scheduling and cache refresh, so convergence is often not fast enough

One way to address this is to keep multiple different addresses and let the upper layer choose. That is not wrong, but it has its own limits:

The client first has to obtain a set of candidate addresses
DNS or application scheduling only sees part of the network, which is not necessarily the same as the real forwarding path
Once the number of entry addresses grows, configuration, certificates, probing, and fail-back logic all become more complex

In many engineering cases, what you actually want is not “give the user many choices,” but “give the user one stable entry, while allowing that entry to be reachable from many places.” That is the value of Anycast: it moves the access-point choice into the network layer instead of making the client or application decide every time.

What Problem It Solves, and In What Background

Anycast is not a heavy protocol. It is more like a usage pattern inside the IP addressing and routing system. It addresses a reality that became more visible as the Internet grew:

One logical service is often deployed in multiple places
User access quality depends more and more on network topology, not only on application configuration
Some base services need very fast local access and failover

That gives Anycast a very different character from unicast and multicast:

The address still looks like an ordinary IP address to the client
But behind that address there is not just one instance; there are multiple geographically distributed entry points
The client does not need to know all instance locations in advance
What decides “which place comes first” is not the application, but routing propagation and policy

So Anycast looks like “one address for many places,” but what it is really discussing is which layer should decide where the entry point lives.

The Main Model

To understand Anycast, keep four things in mind:

Multiple sites announce the same prefix or address
The client only sees one unified entry point
The routing system sends traffic to the currently more suitable announcer
When one announcer withdraws or its path becomes worse, traffic is converged to another one

The main flow can be summarized like this:

Site A announces Prefix P
Site B announces Prefix P
Site C announces Prefix P

Client sends traffic to Prefix P
  -> the network chooses a path based on routing and policy
  -> traffic reaches the currently best announcer first
  -> the site-local service accepts the request

Once this line is clear, a lot of later behavior around BGP, DNS, CDN, and failover becomes easier to read and much less likely to be mistaken for a black-box scheduler.

A Common End-to-End Path

A typical Anycast access flow looks like this:

sequenceDiagram participant C as Client participant ISP as ISP Network participant A as Site A participant B as Site B Note over A,B: Both sites announce the same Anycast prefix C->>ISP: Packet(dst=Anycast IP) ISP->>A: Forward according to current routing policy A->>C: Reply

If Site A withdraws the advertisement or its path is no longer preferred, the main path becomes:

sequenceDiagram participant C as Client participant ISP as ISP Network participant A as Site A participant B as Site B Note over A: Advertisement withdrawn or path no longer preferred C->>ISP: Packet(dst=Anycast IP) ISP->>B: After routing converges, traffic is sent to Site B B->>C: Reply

The important thing in both flows is not “there are two service copies,” but that entry selection has already been absorbed by routing:

The client does not explicitly choose a site
DNS does not necessarily need to return multiple distinct addresses
Convergence and cutover depend more on routing propagation than on application redirects

Why It Is Not Traditional Load Balancing

Anycast is often misunderstood as “load balancing at Internet scale.” That is only half true, and it can easily lead you in the wrong direction.

Its common ground with traditional load balancers is only this:

Eventually, different requests end up on different instances

But the basis for the decision is completely different.

Traditional load balancers are more like a scheduler behind one known entry point, commonly using:

Current connection count
Instance health
Layer-7 path, headers, cookies, or business tags

Anycast decides the entry point primarily through routing:

Which announcer looks closer to the client
Which path is more preferred by policy
Which site is still announcing that address right now

This means Anycast is not good at replacing fine-grained business scheduling. What it is good at is getting the user to a reasonably suitable edge entry first, and then letting site-local load balancing do the more detailed distribution.

So a more accurate way to think about it is:

Anycast decides “which site should I reach first”
Site-local load balancing decides “once I have reached this site, which machine should handle me”

Why It Is So Tightly Bound to Routing Policy

Anycast’s power does not come from the address itself. It comes from the fact that the same address is announced from multiple places, and the routing system then decides how to choose. At that point, it immediately becomes a routing problem, not just an application deployment problem.

That creates a few important realities:

“Nearest” is usually not geographic nearest; it is routing-nearest
Carrier policy, peering relations, and prefix propagation directly influence the final landing site
Different regions and different carriers may see completely different “best” sites

That is why Anycast often appears together with BGP. Without route announcements and convergence, the idea of one address appearing in many places simply does not hold.

So to judge whether Anycast is working, do not only check whether the service process is listening on the address. Also check:

Whether the prefix is really being announced correctly from multiple sites
Whether upstream networks are receiving and propagating those announcements as expected
Whether withdrawal is fast enough to affect path selection when a site goes down

Why It Often Appears with DNS, CDN, and Public Resolvers

Anycast is especially suitable for services that emphasize “reach a nearby entry first, then respond quickly locally.” Typical scenarios include:

Public DNS resolvers
CDN edge entry points
DDoS mitigation and security access layers
Distributed API entry points or edge proxies

These scenarios share one thing:

The user first needs a stable, low-latency, fast-failover entry point

Anycast is valuable here because it turns “which site should I reach first” into a network-default behavior. If the business later needs finer regional scheduling, content hit selection, or site-local distribution, you can add DNS, HTTP scheduling, or local load balancing on top.

That is also why CDNs and public DNS services like Anycast, but do not rely on Anycast alone:

Anycast solves entry-point front-loading and the initial landing site
DNS, cache, edge logic, and local scheduling solve the finer-grained parts

Why It Does Not Natively Guarantee Session Stability

Anycast is a great fit for services where requests are relatively independent, each round trip is short, and retry cost is manageable. It is not naturally friendly to long connections and strong session stickiness.

The reason is simple: once routing changes, later traffic from the same client may be sent to a different site. For the application, that creates several risks:

The connection was established at Site A, but later packets go to Site B
The session state was only local, so it cannot continue after the switch
Routing jitter can be amplified into instability for long-lived connections

That is why a common engineering approach is:

Use Anycast for DNS, UDP queries, edge probing, and short-request entry paths, where switching is easier
If you need to carry TCP, TLS, or HTTP long connections, add session retention, state synchronization, or fast rebuild mechanisms inside and across sites

So Anycast is not incapable of handling stateful protocols. Rather, it makes “the entry may move” a design assumption that must be handled explicitly.

Why the Idealized Spec Path and the Field Path Diverge

In the abstract model, Anycast is simple:

Many places announce the same address
The user naturally lands at the nearest one
When one place has a problem, routing automatically switches to another

Real networks are nowhere near that tidy. The common deviations usually come from these areas:

Some upstream networks do not propagate your route the way you expected
Different carrier policies make “nearest site” become another site entirely
One site is already unhealthy, but route withdrawal is too slow
The route has already moved away, but DNS, certificates, origin fetches, or local caches have not caught up

That means the protocol model and the field model must be read separately:

Theoretically nearest is not the same as lowest-latency in user experience
Routable is not the same as fully usable
A site still announcing the address does not mean it should still receive traffic

So the hard part of Anycast is usually not “put the same address in two places.” It is making route health, service health, and business-state health line up as closely as possible.

What You Should Actually Look at When Debugging

When you investigate Anycast issues, do not start by assuming it is a DNS or application bug. A more useful sequence is:

First check where the user actually landed

Confirm:

Where the Anycast address currently converges for this user
Whether different regions or different carriers land on different sites
Whether route switching lines up with the failure time

If you have not even confirmed which entry point the user reached, later application analysis will often drift.

Then check whether site health and route advertisement are still aligned

Many problems are not in the service itself, but in the fact that health is not reflected back into the routing plane:

The service is already broken, but the site keeps announcing
The service has recovered, but upstream preference has not converged back yet
Some prefixes were withdrawn but others were not, which splits user behavior

The key question here is not fields. It is whether “the site that should receive traffic” and “the site that is actually receiving traffic” are the same set.

Finally check whether sessions and state have side effects

If the business is stateful, then confirm:

Does the connection get rebuilt when the route changes?
Can the session continue after the site switch?
Do local cache, TLS, origin fetch, or authentication states depend on the original site?

Many problems that look like “Anycast is unstable” are actually caused by the application assuming the entry point never changes.

What You Should Really Think About Today

Do not treat Anycast as another load balancer. It primarily solves entry-site selection, not fine-grained business-instance scheduling
Do not interpret “nearest” as geographic nearest. It is usually route and policy nearest
Do not only check whether the address is listening. First check whether multi-site announcements, upstream propagation, and withdrawal convergence are really happening
Do not equate route reachability with business availability. If site health cannot be reflected in announcement state quickly enough, Anycast will faithfully send users to a bad site
Do not underestimate session problems. If long connections and strongly stateful services sit behind an Anycast entry, you must handle site switching explicitly
Do not use Anycast as a standalone answer. It usually needs to work together with local load balancing, DNS, CDN, or edge proxies