|
 |
|
|
|
|
|
Introduction to IP Addressing Types
Internet Protocol (IP) addresses are often overloaded to serve multiple purposes, such as a unique host identifier and locator. Most people
are familiar with an IP address that is used as a unique identifier such as one assigned to a local PC's Ethernet or dial-up PPP interface. IP addresses assigned as a unique identifier to a host interface are referred to as unicast addresses. A unicast address can be used either
as the source or destination address in an IP datagram. A sender inserts an assigned unicast address into the source address field of an
IP datagram for packets sent onto the network. When packets are destined for a single host, a sender inserts a unicast address into the
destination address field of an IP datagram, indicating who (and where) the packet should be delivered to. In some cases, hosts have multiple
unicast address assignments. Often times in those cases, it usually means that the host has multiple network attachments with a unicast address assigned to each. Although in some special circumstances, multiple unicast addresses can be assigned to a single interface. An example of a unicast address is 207.99.115.72. This is the unicast IP address that, at least as of this writing, is associated with the hostname www.kuro5hin.org.
Unicast addresses have historically been used to identify unique hosts or host interfaces. Knowing a unicast address usually meant knowing the actual, physical host the unicast address was associated with. This changes with anycast (and also with things like load balancers or IETF RFC 1918 addresses), but we'll come back to that part of the story shortly.
Broadcast addresses have been used to support discovery of services or hosts. Packets to an IP broadcast address are delivered to all hosts
on a particular physical data link network or IP subnet. Broadcast addresses should only be used in the destination address field of an IP header, because it would be illogical for a unique packet to be sourced by a group of hosts or interfaces. IP broadcast addresses take the form of either the limited broadcast address (255.255.255.255) or the
directed broadcast address (the network ID plus all 1's in the host ID portion of the intended subnet).
There is also the concept of a group address and for this the term multicast is used. A multicast address is associated with a group of interested receivers. A group can consist of any number of hosts, including all hosts on the network. Therefore, a broadcast address is a subset address type within the concept of multicast addressing. Like the broadcast address, multicast addresses can only be used in the destination address field of an IP header. IP multicast addresses are
allocated from the historic class D range of addresses (224.0.0.0/4 in CIDR notation, or an address between 224.0.0.0 and 239.255.255.255
inclusive, for those of you unfamiliar with IP address netblock allocations).
Anycast addresses can be used as either a source or destination address, but no longer uniquely identify a single host or service. However,
they do continue to act as a locator. Anycast addresses come from or are sent to the "nearest" host or service. The definition of "nearest"
a) depends on the network topology, b) protocols used to make forwarding decisions and c) the associated administrative policies within the internetwork. Sometimes anycast addresses are referred to as group addresses, but unlike multicast, packets are delivered only to one
member within the anycast group. Except for the 6to4 relay router specification, there is not an address range set aside for anycast use on the public IPv4 Internet today (see IETF RFC 3330). Individual organizations must allocate anycast addresses from an available unicast address netblock they have administrative control of.
By definition, IPv4 anycast addressing assigns a common unicast address to multiple interfaces, hosts or services on an internetwork. In other
words, duplicate IP addresses are purposefully configured into the IP internetwork. Assigning duplicate IP addresses can be a dangerous affair and in fact will often break (catastrophically in some cases) communications when used on the same physical data link network. Without special load balancing protocols or hardware, anycast IP
addresses should not be duplicated on any one data link network.
While some may argue that any use of duplicate IP addressing is fundamentally a bad idea, for our purposes in this article we will ignore the philosophical debates. So how then, does the network figure out where to send packets to an anycast address when that address can be multiple places at once?
Anycast Routing
Routers advertise netblocks (also known as prefixes) serving anycast address space just as they would advertise netblocks for unicast or
multicast prefixes. With anycast address space, routers advertise the address netblock from multiple origin points in the internetwork.
From a routing topology perspective, a common anycast address netblock looks as if it is multi-homed to all points of origin. In practice, an anycast netblock is often just a host route (/32 in CIDR notation) within an autonomous system (AS). To network routers and routing protocols, nothing changes about their operation and there are no new anycast specific routing concepts to learn.
To illustrate how anycast address netblocks are used in an internetwork, we will use an example. Imagine an internetwork with routers in three
cities, Chicago, Los Angeles and New York. Presume there are anycast service instances attached to the Los Angeles and New York routers. A client of the anycast service is attached to the Chicago router. Packets from the host in Chicago to the anycast address will be forwarded towards either New York or Los Angeles, arriving at a unique instance of the service at one city, but not both. The packets will be forwarded based on the routing topology as viewed from Chicago (and
other routers in between).
Presume packets from Chicago to the anycast address in the previous example normally travel to the New York instance. What happens if the
route to New York from Chicago changes such that Los Angeles looks closer from a topology perspective (as might happen when a link or router goes down)? As soon as the routing topology converges due to the change, packets would then transition to the Los Angeles path.
As an aside, one may be thinking, if the path changes in the middle of a conversation between the Chicago source and the anycast service in New
York or Los Angeles, won't communications fail? Without complex mechanisms to maintain synchronization, the other anycast service
instance will not know about the session that existed previously, right? Correct. This is why anycast is not well suited for communications that
involve "state" or long-lived flows of traffic. This eliminates TCP-based protocols and and even some UDP-based applications from being deployed with anycast addressing reliably. We will come back to specific applications where anycast is most applicable in the next section.
How anycast address netblocks populate the routing topology and how the routing "metrics" are calculated can vary from site to site. In the
simplest form, a static route can be set in a local router, pointing to an instance of an anycast service host. However, if the anycast host fails, the static route has to be manually removed so that packets to the anycast netblock can be forwarded to other instances on the network. The most popular way to inject and remove anycast address netblocks from the network routing tables is by running a routing process directly
on the anycast service host. This enables the anycast service host to automatically control whether it is available or not if downtime or
maintenance is to occur. It also allows the network to automatically age out routes, which may happen if an anycast host fails.
Packets travel to an anycast address based on the locality, from a routing perspective, between a client and anycast service host. This tends to keep network traffic within a well defined coverage area as long as anycast service hosts and the routing infrastructure remains stable. Through careful network design, traffic patterns and link loads can be greatly influenced by the deployment of anycast addressing. By leveraging the unicast routing system, redundancy, load balancing and attack resiliency can be designed into services utilizing anycast addressing. In the next section, we examine some of the applications that can take advantage of these properties.
Anycast Applications
While there is nothing to prevent someone from running any service using anycast addressing, even TCP-based ones, it may be sufficiently unwise
to do so. If routing instabilities or host failures occur, communications will fail as they are re-routed to another anycast service instance. The new anycast service instance will not recognize session in progress without complex state-sharing mechanisms. In addition, it would be foolish to begin transitioning services to run as anycast services without thoroughly understanding all of the nuances that come into managing applications and hosts with non-unique IP
addresses. In other words, anycast addressing is not for everyone and everything. It often doesn't make sense to go through the extra complexity in deploying a service with anycast addressing if it doesn't justify the benefit.
That said, anycast addressing is currently used successfully by a number of organizations and for a number of applications. We will focus our
discussion of application usage on DNS and multicast rendezvous points (RPs), while offering a glimpse into other applications that are most
likely candidates for anycast addressing deployments.
Domain Name System (DNS)
Perhaps the most widely publicized use of anycast addressing today is in
large DNS deployments, most notably by a handful of the Internet's
root DNS server operators. Nearly all exchanges between a DNS client
and a DNS server are a single UDP client query request and sole UDP
message server response. In this mode of operation, anycast addressing
is well suited. Its been measured that this simple exchange accounts for
over 99% of all legitimate DNS traffic between DNS clients and root DNS
servers. As a result of the enormous load required by DNS root servers
and the stateless nature of DNS communications, root servers are perfect
candidates for leveraging anycast addressing.
Those familiar with DNS protocol operations may know that DNS
messages can and sometimes do use TCP instead of UDP. Isn't this going to
be a problem if anycast routes or anycast services are unstable? Potentially
yes, but practically no. Root server operators have measured typical,
legitimate DNS traffic and found very little TCP-based communications.
In addition, since the routes and hosts are generally stable, risk due to
transition to other instances of an anycast address are very low. Even
if there was a transition, it would have to fall within a very short
window, since even TCP-based DNS queries are relatively short-lived.
The F root nameserver operated
by ISC was the first root nameserver to
deploy anycast addressing and they have been doing so since November, 2002.
Root nameservers such as F root have been on the receiving end of many powerful
and coordinated
denial of service attacks. It is widely recognized that anycasting of
some servers has been glaringly helpful in mitigating the impact of those
attacks. Primarily to deal with offered load, due to valid requests,
misconfigured or attack traffic, anycast addressing has proven to greatly
enhance DNS root server survivability.
Multicast Rendezvous Points (RPs)
One of the first and most widely used applications of anycast addressing
was for protocol independent multicast sparse-mode (PIM-SM) rendezvous
focal point, bringing together multicast senders and interested
receivers within a multicast domain.
IETF RFC 2362, the
PIM-SM specification originally stipulated a single RP for any
particular group. For large and geographically diverse networks, this
can imply a significant single point of failure, an unruly traffic load
aggregation point, large amounts of group state maintenance or all of
the above. Anycast addressing has been used to create multiple
instances of RPs for common groups within a PIM domain.
IETF RFC 3446 has
recently been written to update the original PIM-SM specification,
allowing for multiple RPs per group through the use of anycast
addressing.
In anycast RP environments, multicast group registration is delivered
to the nearest RP. However, because an RP must know about all sources
for any particular group it is an RP for, multiple anycast RPs must
share group registrations they each know about. This is currently
achieved through the use of multicast source discovery protocol (MSDP)
peering between the RPs. Anycast RP is now widely believed
to be the "current best practice" for PIM-SM deployments.
Other Anycast Applications
Dozens of other custom or standard applications can potentially be
deployed with anycast addressing techniques. Syslog servers, which collect
UDP based messages from remote hosts are a good example, because the
traditional Syslog server implementations never send messages in response to
received log messages. Traffic flow collectors, such as Cisco's Netflow
architecture or the freely implemented collectors like
cflowd
are also candidates for anycast addressing
deployments. Sinkhole networks, which are designed to route "bogon"
netblocks off production networks and are sometimes used to monitor bogus
address space usage for later analysis are also good candidates for anycast deployment.
The Simple Network Time Protocol (SNTP) would work
with anycast, but NTP could be a problem. SNTP uses a single request
and reply message exchange, but NTP generally requires at least two
packets from both the client and server to exchange time information
properly. IPv6 to IPv4 relay routers as defined in
IETF RFC 3068 have their own netblock, which acts as an anycast service for IPv6 networks to talk
to IPv4 networks. These 6to4 relay routers use anycast to help ease
network management of IPv4 to IPv6 protocol transition. SNMP trap managers
could also be configured for anycast addressing, because SNMP managers
do not generally respond to received traps. Although if used as a full
two-way SNMP manager, the SNMP manager should use a globally unique unicast
address for SNMP management.
It is possible to configure any application using anycast
addressing, but practically it may be unwise to do so. An unstable
network may interrupt long lived sessions. Coupled with the difficulty in troubleshooting broken connections, many applications will
often call for alternative solutions.
Anycast Service Management
When duplicate IP addresses are used, how does a network manager
monitor each instance of the anycast service? There are at least two
ways. First, the monitoring tools could be placed in proximity to
each instance of the anycast service, so that probe packets will arrive
at each anycast service instance. Second, a remote monitoring station
could be located remotely, but could tunnel management packets to a
unique address in close proximity to the anycast service to be managed.
In this way, when the probe packets reach the end of the tunnel, they
will be forwarded to the nearest anycast service. However, when the
response is sent back to the centralized management station, which by the
way does not need to be tunneled, how does the central management station know which
instance of the anycast service is responding? This is left as an
exercise for the reader.
Most anycast services are deployed along with a global, unique unicast
address for management and state-based communications. So for instance,
a DNS server may receive and reply to DNS queries using the anycast address, but it will receive and reply to zone transfers and remote management messages
on a separate, unique unicast address (possibly on the same interface as that
of the anycast address). Implementing a unique unicast address for management
purposes isn't strictly required, but it is
such a good idea that it should be. Usually the anycast address is
implemented as a virtual interface on the anycast service host, such as
in the form of a loopback address and the unique unicast management address is
associated with an actual physical data link interface.
Since an anycast service, such as DNS, is separate from the underlying
routing infrastructure that makes the service available through anycast
netblock routing updates, what happens
when the anycast service fails, but the route to its anycast address is
still in the routing topology? Typically there is no strong coupling
of application service to address route announcement, which means a failure in application service
does not induce a route withdrawal to its instance of the anycast address
from the network's routing tables. To get closer to a solution, the
host running the anycast service can run a distributed routing protocol
process locally and inject route updates to its local router for inclusion
in the local AS or global Internet routing tables. If the host crashes,
its anycast address route will eventually time-out, resulting in a route withdrawal
for that instance of the anycast address netblock. However, what if only
the application fails, but not the entire host? How does the local routing
process know to withdraw the route for the unavailable anycast service?
Today, the answer is "it probably doesn't". For most anycast applications
anyway, there is typically not going to be a strong coupling between the
application service and the local routing process. However, the host
can be carefully configured to monitor its running anycast service. If
the anycast service exits, crashes or becomes unresponsive, a local
process can send an alert to an administrator and even automatically
restart the anycast process. If the process exits and cannot be restarted,
a local process can be configured to shutdown the local routing
process or withdraw the anycast address netblock.
Conclusion
As used in IPv4 networks, anycast addressing is the strategic use of
duplicate unicast IP addressing to provide localization for services and
hosts in order to obtain robustness, redundancy and resiliency.
While anycast addressing is a useful technique, it is not suitable for all applications or environments.
Anycast addressing is not a panacea to scaling, redundancy or security
problems in the IPv4 Internet. It, like many other techniques, is just
one tool that if carefully considered and applied can be advantageous
in certain scenarios. For many problems, other solutions may be
preferable and any network designer or engineer should carefully
consider options and trade-offs.
References
|
|
|
![]](images/rt_edge3.gif){kind=small}
![[XML]](images/rssblue.gif){kind=small}
![[P]](images/print3.gif){kind=small}
