The delivery of IP traffic through the Internet depends on the complex interactions between thousands of autonomous systems (ASes) that exchange routing information using the Border Gateway Protocol (BGP). This paper investigates the topological structure of the Internet in terms of customer-provider and peer-peer relationships between ASes, as manifested in BGP routing policies. We describe a technique for inferring AS relationships by exploiting partial views of the AS graph available from different vantage points. Next we apply the technique to a collection of ten BGP routing tables to infer the relationships between neighboring ASes. Based on these results, we analyze the hierarchical structure of the Internet and propose a five-level classification of ASes. Our characterization differs from previous studies by focusing on the commercial relationships between ASes rather than simply the connectivity between the nodes.

Abstract—This paper examines the network interdomain routing information exchanged between backbone service providers at the major U.S. public Internet exchange points. Internet routing instability, or the rapid fluctuation of network reachability information, is an important problem currently facing the Internet engineering community. High levels of network instability can lead to packet loss, increased network latency and time to convergence. At the extreme, high levels of routing instability have led to the loss of internal connectivity in wide-area, national networks. In this paper, we describe several unexpected trends in routing instability, and examine a number of anomalies and pathologies observed in the exchange of inter-domain routing information. The analysis in this paper is based on data collected from BGP routing messages generated by border routers at five of the Internet core’s public exchange points during a nine month period. We show that the volume of these routing updates is several orders of magnitude more than expected and that the majority of this routing information is redundant, or pathological. Furthermore, our analysis reveals several unexpected trends and ill-behaved systematic properties in Internet routing. We finally posit a number of explanations for these anomalies and evaluate their potential impact on the Internet infrastructure. Index Terms—Communication system, communication system routing, computer network, Internet, routing, stability.

The Border Gateway Protocol (BGP) is the de facto interdomain routing protocol used to exchange reachability information between Autonomous Systems in the global Internet. BGP is a path-vector protocol that allows each Autonomous System to override distance-based metrics with policy-based metrics when choosing best routes. Varadhan et al. [18] have shown that it is possible for a group of Autonomous Systems to independently define BGP policies that together lead to BGP protocol oscillations that never converge on a stable routing. One approach to addressing this problem is based on static analysis of routing policies to determine if they are safe. We explore the worst-case complexity for convergenceoriented static analysis of BGP routing policies. We present an abstract model of BGP and use it to define several global sanity conditions on routing policies that are related to BGP convergence/divergence. For each condition we show that the complexity of statically checking it is either N...

Recent studies concerning the Internet connectivity at the AS level have attracted considerable attention. These studies have exclusively relied on the BGP data from Oregon route-views [1] to derive some unexpected and intriguing results. The Oregon route-views data sets reflect AS peering relationships, as reported by BGP, seen from a handful of vantage points in the global Internet. The possibility that these data sets from Oregon route-views may provide only a very sketchy picture of the complete inter-AS connections that exist in the actual Internet has received surprisingly little scrutiny. In this paper, we will use the term "AS peering relationship" to mean that there is "at least one direct router-level connection" between two existing ASs, and that these two ASs agree to exchange traffic by enabling BGP between them. By augmenting the Oregon route-views data sets with BGP summary information from a large number of Internet Looking Glass sites and with routing policy information from Internet Routing Registry (IRR) databases, we find that (1) a significant number of existing AS connections remain hidden from most BGP routing tables, (2) the AS connections to tier-1 ASs are in general more easily observed than those to non tier-1 ASs, and (3) there are at least about 25--50% more AS connections in the Internet than commonly-used BGP-derived AS maps reveal (but only about 2% more ASs). These findings point out the need for an increased awareness of and a more critical attitude toward the applicability and completeness of given data sets at hand when establishing the generality of any particular observations about the Internet.

In this paper, we describe an experimental study of Internet stability and the origins of failure in Internet protocol backbones. The stability of end-to-end Internet paths is dependent both on the underlying telecommunication switching system, as well as the higher level software and hardware components specific to the Internet's packet-switched forwarding and routing architecture. Although a number of earlier studies have examined failures in the public telecommunication system, little attention has been given to the characterization of Internet stability. Our paper analyzes Internet failures from three different perspectives. We first examine several recent major Internet failures and their probable origins. These empirical observations illustrate the complexity of the Internet and show that unlike commercial transaction systems, the interactions of the underlying components of the Internet are poorly understood. Next, our examination focuses on the stability of paths between In...

This paper examines the network routing messages exchanged between core Internet backbone routers. Internet routing instability, or the rapid fluctuation of network reachability information, is an important problem currently facing the Internet engineering community. High levels of network instability can lead to packet loss, increased network latency and time to convergence. At the extreme, high levels of routing instability have led to the loss of internal connectivity in wide-area, national networks. In an earlier study of inter-domain routing, we described widespread, significant pathological behaviors in the routing information exchanged between backbone service providers at the major U.S. public Internet exchange points. These pathologies included several orders of magnitude more routing updates in the Internet core than anticipated, large numbers of duplicate routing messages, and unexpected frequency components between routing instability events. The work described in this pape...

In this paper, we describe an experimental study of Internet topological stability and the origins of failure in Internet protocol backbones. The stability of end-toend Internet paths is dependent both on the underlying telecommunication switching system, as well as the higher level software and hardware components speci c to the Internet's packet-switched forwarding and routing architecture. Although a number of earlier studies have examined failures in the public telecommunication system, little attention has been given to the characterization of Internet stability. We provide analysis of the stability of major paths between Internet Service Providers based on the experimental instrumentation of key portions of the Internet infrastructure. We describe unexpectedly high levels of path uctuation and an aggregate low mean time between failures for individual Internet paths. We also provide a case study of the network failures observed in a large regional Internet backbone. We characterize the type, origin, frequency and duration of these failures. 1.

To maintain high availability in the face of changing network conditions, network operators must quickly detect, identify, and react to events that cause network disruptions. One way to accomplish this goal is to monitor routing dynamics, by analyzing routing update streams collected from routers. Existing monitoring approaches typically treat streams of routing updates from different routers as independent signals, and report only the “loud ” events (i.e., events that involve large volume of routing messages). In this paper, we examine BGP routing data from all routers in the Abilene backbone for six months and correlate them with a catalog of all known disruptions to its nodes and links. We find that many important events are not loud enough to be detected from a single stream. Instead, they become detectable only when multiple BGP update streams are simultaneously examined. This is because routing updates exhibit network-wide dependencies. This paper proposes using network-wide analysis of routing information to diagnose (i.e., detect and identify) network disruptions. To detect network disruptions, we apply a multivariate analysis technique on dynamic routing information, (i.e., update traffic from all the Abilene routers) and find that this technique can detect every reported disruption to nodes and links within the network with a low rate of false alarms. To identify the type of disruption, we jointly analyze both the network-wide static configuration and details in the dynamic routing updates; we find that our method can correctly explain the scenario that caused the disruption. Although much work remains to make network-wide analysis of routing data operationally practical, our results illustrate the importance and potential of such an approach. 1.

Modeling the Internet infrastructure is a challenging endeavor. Complex interactions between protocols, increasing traffic volumes and the irregular structure of the Internet lead to demanding requirements for the simulation developer. These requirements include implementation detail, memory efficiency and scalability, among others. We introduce a simulation model of the Border Gateway Protocol that we call BGP++, which is built on the popular ns-2 simulation environment. A novel development approach is presented that incorporates the public domain routing software GNU Zebra in the simulator. Most of the original software functionality is retained, while the transition to the simulation environment required a manageable amount of effort. Moreover, the discussed design inherits much of the maturity of the original software, since the later is only minimally modified. We analyze BGP++ features and highlight its potential to provide significant aid in BGP research and modeling. 1.

Failures at the BGP level can have significant impact on the overall Internet. Understanding the behavior of BGP is thus both an important practical challenge and an interesting research problem. To understand the true dynamics, and help interpret the multiple gigabytes of BGP log data, we have developed the “Link-Rank ” graphical toolset. Link-Rank weights the links between Autonomous Systems by the number of routing prefixes going through each link. Tracing these graphs over time results in a directed graph that shows the weight changes of the logical inter-AS links. From this graph one can easily visualize the complex BGP path changes and also combine views from multiple vantage points, to get a better picture of global routing dynamics. We illustrate the usefulness of Link-Rank by using it to examine BGP routing dynamics in three example cases. These examples show that Link-Rank is able to help BGP analysts estimate the scope of routing changes and to reveal important routing dynamics in the presence of superfluous BGP update messages.