BorderGuard: Detecting Cold Potatoes from Peers
Nick Feamster Zhuoqing Morley Mao Jennifer Rexford
MIT Computer Science & AI Lab University of Michigan AT&T Labs Research
feamster@csail.mit.edu zmao@eecs.umich.edu jrex@research.att.com
Abstract
Internet Service Providers often establish contractual peering agree-
ments, where they agree to forward traf c to each other’s customers at
no cost. Consistent route advertisement at all peering points is a com-
mon provision in these agreements, because it gives an AS the exibil-
ity to select egress points for the traf c (e.g., performing hot potato
routing). Verifying consistent export is challenging because route
advertisements are exchanged at multiple peering points and may be
modi ed by routing policies. In this paper, we propose two algorithms
to detect inconsistent routes using routing and con guration data from
an AS’s border routers. The rst algorithm requires access to all eBGP
routes advertised by a peer. Because this data is often unavailable, we
propose another algorithm that detects inconsistencies using readily
available data. We have applied our algorithms to the routes adver-
tised by the peers of AT&T’s commercial IP backbone. Although a
peer may intentionally send inconsistent advertisements to prevent its
neighbor from performing hot-potato routing, we also discuss several
con guration scenarios where a peer may inadvertently advertise in-
consistent routes, despite having consistent export policies. Finally,
we explain how simple modi cations to the routers could make detec-
tion of inconsistent advertisements much easier than it is today.
Categories and Subject Descriptors
C.2.6 [Computer-Communication Networks]: Internetworking; C.4
[Performance of Systems]: Measurement Techniques
General Terms
Algorithms, Management, Measurement, Performance
Keywords
BGP, anomalies, peering, inconsistent advertisement
1. Introduction
Service providers in the core of the Internet connect to each other in
order to reach their respective customers. Before agreeing to peer,
two service providers sign a peering agreement that outlines the terms
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
IMC’04, October 25 27, 2004, Taormina, Sicily, Italy.
Copyright 2004 ACM 1›58113›821›0/04/0010 ...$5.00.
3210
AS A
AS B
3 52
68 419
17
a
b
Figure 1: Hot-potato routing between peers with four peering points:
Dashed lines highlight the intradomain path costs
of their relationship. These contracts typically require the Autonomous
Systems (ASes) to connect in multiple geographic locations [1]; in Fig-
ure 1, ASes A and B peer in four locations spread throughout their net-
works. In addition to providing redundancy, the multiple connections
are meant to give an AS the exibility to select a convenient egress
point for sending traf c to the other AS. Under the common practice of
hot-potato (or early-exit) routing, a router selects the closest egress
point in terms of the intradomain path costs, in order to reduce the net-
work resources required to carry the traf c. For example, in Figure 1,
router b in AS B can direct traf c through peering point 3 rather than
sending traf c a long distance across the network to one of the other
egress points. Similarly, router a in AS A’s network can direct traf-
c through peering point 0. In some cases, a network operator may
override hot-potato routing to balance the traf c load.
To give operators the exibility to select from multiple egress points,
peering contracts typically require the peer to provide consistent routes
at all interconnection points [1]. That is, an AS must make each desti-
nation reachable at every peering point via equally good routes. If a
destination connected to router a were reachable only through peering
point 0, traf c from b would have to travel over expensive long-haul
links in AS B and only a short distance in AS A. In this scenario,
AS A is violating its peering agreement by forcing AS B to do cold
potato routing. In addition, AS A must not try to make one peering
connection look less attractive to B than another (e.g., by making the
AS path appear longer), unless the two ASes have agreed in advance,
since this would force B to consume more resources to send traf c.
In this paper, we formulate the problem of checking the consistency
of routes advertised by a peer and present a technique for detecting
inconsistencies using routing and con guration data available in the
receiving AS. The most closely related work is an empirical study of
path in ation by Spring et al. [2], which analyzed traceroute data to
infer deviations from early exit routing without identifying the un-
derlying reason. In contrast, we determine whether an AS is forced to

1. Highest local preference
2. Lowest AS path length
3. Lowest origin type
4. Lowest MED (with same next-hop AS)
5. eBGP-learned over iBGP-learned
6. Lowest intradomain path cost to egress point
7. Lowest router ID of BGP speaker
Table 1: BGP decision process with peer-assigned attributes in bold
select a different egress point due to inconsistent route advertisements
from a peer rather than voluntarily choosing a different egress point to
satisfy its own traf c engineering goals.
An AS receives route advertisements from a peer via Border Gate-
way Protocol (BGP) sessions at the peering points. A BGP-speaking
router sends an advertisement to notify its neighbor of a new route
to the destination pre x and a withdrawal when the route is no longer
available. An advertisement includes attributes, such as the list of ASes
in the path, that affect the selection of the best route at each router. To
be consistent, multiple routes from the same peer for the same pre x
must agree in any aspects that affect the BGP decision process AS
path length, origin type, and multiple exit discriminator (MED) as
shown in bold in Table 1. Other steps in the decision process are con-
trolled by the receiving AS. For example, a router can apply an import
policy that assigns the local-preference attribute to favor one route over
another, and use the intradomain path cost to select the route with the
closest egress point. Although the operator can con gure an import
policy that resets the origin type and MED attributes to default values,
the receiving AS is especially vulnerable to inconsistencies in the AS
path lengths of the routes advertised by its peers.
Identifying inconsistencies should be as easy as comparing the BGP
routes learned from each peer for each pre x for differences in AS
path length, origin type, and MED, as discussed in Section 2. Un-
fortunately, acquiring a feed of all routes advertised by a neighboring
domain is dif cult in practice1. Instead, we consider how to detect
inconsistent routes from data readily available within the local AS
an internal BGP (iBGP) feed of the best route for each pre x from
each border router and the import policies con gured on each of these
routers. However, identifying inconsistencies from this data is chal-
lenging because our algorithm only has access to the best route for
each pre x, after the routes have been manipulated by the import poli-
cies. In Section 3, we determine how much these constraints limit
our ability to identify inconsistent route advertisements from peers and
present an algorithm that identi es inconsistencies that force the AS to
select a different egress router.
Section 4 presents the results of applying the algorithms to the routes
advertised by the peers of AT&T’s commercial IP backbone. We ap-
ply the rst algorithm to eBGP feeds provided by one large peer and
then apply the second algorithm to iBGP feeds from AT&T’s border
routers. Our analysis discovers many short-lived routing inconsisten-
cies that could be explained by transient routing updates during the
BGP convergence process, but we also nd inconsistencies that persist
for longer periods of time, suggesting either con guration mistakes or
malicious behavior. In Section 5, we present several examples that il-
lustrate how a peer might inadvertently advertise inconsistent routes
and suggest ways the sending AS could detect potential problems in
advance. Section 6 concludes the paper with a discussion of ways
1This would require either (1) extending today’s commercial routers to provide
a feed of all eBGP-learned routes, which, while definitely appealing, is not
likely to happen quickly, (2) deploying packet monitors on the many high-speed
links between peers to capture the BGP updates, which would be extremely
expensive, or (3) asking the peer AS to provide the eBGP data feed from its own
border routers, which runs the risk that the peer intentionally sends different
information to our detection algorithm than it does to the operational routers.
b b b1 2 3
Monitoring Point
iBGP
eBGP applied at border
Import policy
routers
I
AS BAS A
Figure 2: Monitoring inconsistent route advertisements in an AS with
three peering points.
router vendors and network operators can make the BorderGuard prob-
lem easier to solve in the future.
2. BorderGuard Using Direct eBGP Feeds
In this section, we formulate the BorderGuard problem and present
a solution that operates on a direct feed of the eBGP-learned routes
from each peer AS. Throughout the paper, our discussion focuses on
a single destination pre x, since routing decisions for each pre x are
independent.
A network has m peer ASes p = 1, 2, . . . ,m and has np eBGP
sessions with peer p. At any given time, the network has one (pos-
sibly null) route rp,u for the pre x from each peering point u =
1, 2, . . . , np. An advertisement message on session u replaces the old
value of rp,u with a new route; a withdrawal replaces the old value with
a null route. To compare the routes, we de ne a function λ(rp,u) that
ranks a route based on the rst ve steps of the BGP decision process
in Table 1 up to, but not including, the hot potato step that chooses
the route with the closest egress point2. A lower value of λ(r) implies
a less attractive route (e.g., a route with a longer AS path length); a null
route has the lowest possible value. We consider a peer as inconsistent
if λ(rp,u) 6= λ(rp,v) for some u, v ∈ [1, np].
Our algorithm applies this check to streams of eBGP data from a
given peer. Upon receiving an update message on session u, the al-
gorithm compares λ(rp,u) to the values λ(rp,v) for v ∈ [1, np] and
reports any mismatches. In the next section, we present a second al-
gorithm that operates on streams of the best BGP route from each
border router in the local AS.
3. BorderGuard Using Indirect iBGP Feeds
In this section, we describe the algorithm that detects inconsistent
route advertisements from peers, using only data that are directly avail-
able to that AS. We rst de ne this new problem and explain the chal-
lenges for inferring the characteristics of eBGP advertisements from
iBGP data and routing policy. We state the conditions that must be
true in order for this inference to be possible. We then present an algo-
rithm that accurately determines whether a peer advertises consistent
routes at all peering points as long as these conditions are satis ed.
3.1 Problem Formulation
An AS has k border routers, each of which may have zero or
more sessions to each of the AS’s peers. We also de ne a function
Routers(p) that returns the set of np routers in the AS that peer with
p. Each border router i applies an import policy, Ii, to the routes that
it receives via eBGP and selects a single best route bi for a destination.
2Since the local-preference attribute is local to an AS, an eBGP-learned route
does not have a local preference. Also, all eBGP-learned routes would receive
the same treatment in step 5 in the BGP decision process. As such, only the AS
path length, origin type, and MED affect the comparison between two eBGP-
learned routes. Steps 1 and 5 are important in Section 3, though, to compare
the “best” routes seen in different iBGP data feeds.