This paper presents DONet, a Data-driven Overlay Network for live media streaming. The core operations in DONet are very simple: every node periodically exchanges data availability information with a set of partners, and retrieves unavailable data from one or more partners, or supplies available data to partners. We emphasize three salient features of this data-driven design: 1) easy to implement, as it does not have to construct and maintain a complex global structure; 2) efficient, as data forwarding is dynamically determined according to data availability while not restricted by specific directions; and 3) robust and resilient, as the partnerships enable adaptive and quick switching among multi-suppliers. We show through analysis that DONet is scalable with bounded delay. We also address a set of practical challenges for realizing DONet, and propose an efficient member- and partnership management algorithm, together with an intelligent scheduling algorithm that achieves real-time and continuous distribution of streaming contents.

Abstract—Overlay networks have emerged as a powerful and highly flexible method for delivering content. We study how to optimize throughput of large transfers across richly connected, adaptive overlay networks, focusing on the potential of collaborative transfers between peers to supplement ongoing downloads. First, we make the case for an erasure-resilient encoding of the content. Using the digital fountain encoding approach, end hosts can efficiently reconstruct the original content of size from a subset of any symbols drawn from a large universe of encoding symbols. Such an approach affords reliability and a substantial degree of application-level flexibility, as it seamlessly accommodates connection migration and parallel transfers while providing resilience to packet loss. However, since the sets of encoding symbols acquired by peers during downloads may overlap substantially, care must be taken to enable them to collaborate effectively. Our main contribution is a collection of useful algorithmic tools for efficient summarization and approximate reconciliation of sets of symbols between pairs of collaborating peers, all of which keep message complexity and computation to a minimum. Through simulations and experiments on a prototype implementation, we demonstrate the performance benefits of our informed content-delivery mechanisms and how they complement existing overlay network architectures. Index Terms—Bloom filter, content delivery, digital fountain, erasure code, min-wise sketch, overlay, peer-to-peer, reconciliation. I.

We present Slurpie: a peer-to-peer protocol for bulk data transfer. Slurpie is specifically designed to reduce client download times for large, popular files, and to reduce load on servers that serve these files. Slurpie employs a novel adaptive downloading strategy to increase client performance, and employs a randomized backoff strategy to precisely control load on the server. We describe a full implementation of the Slurpie protocol, and present results from both controlled localarea and wide-area testbeds. Our results show that Slurpie clients improve performance as the size of the network increases, and the server is completely insulated from large flash crowds entering the Slurpie network.

Abstract — Overlay networks are widely used to deploy functionality at edge nodes without changing network routers. Each node in an overlay network maintains connections with a number of peers, forming a graph upon which a distributed application or service is implemented. In an “Eclipse ” attack, a set of malicious, colluding overlay nodes arranges for a correct node to peer only with members of the coalition. If successful, the attacker can mediate most or all communication to and from the victim. Furthermore, by supplying biased neighbor information during normal overlay maintenance, a modest number of malicious nodes can eclipse a large number of correct victim nodes. This paper studies the impact of Eclipse attacks on structured overlays and shows the limitations of known defenses. We then present the design, implementation, and evaluation of a new defense, in which nodes anonymously audit each other’s connectivity. The key observation is that a node that mounts an Eclipse attack must have a higher than average node degree. We show that enforcing a node degree limit by auditing is an effective defense against Eclipse attacks. Furthermore, unlike most existing defenses, our defense leaves flexibility in the selection of neighboring nodes, thus permitting important overlay optimizations like proximity neighbor selection (PNS). I.

In order to maximize throughput in end-system multicast, it is necessary to have fine-grained control over the transmit load of each participating member. This both avoids bottlenecks where members are overloaded, and allows heterogeneous members to contribute as much transmit capacity as they are able or willing to. In this paper, we describe and simulate an unstructured endsystem multicast protocol called Chunkyspread that provides members with fine-grained control over their transmit load, scales well, has relatively low latencies, and can tolerate high membership churn. Chunkyspread is designed as a flexible framework that easily incorporates different constraints and optimizations. For instance, it is straightforward to add tit-for-tat or path disjointness as constraints to the system. This paper demonstrates the performance of Chunkyspread through extensive simulations, and provides partial validation of these simulations on Emulab. It also provides detailed comparisons with Splitstream, a structured heterogeneous end-system multicast protocol. The simulations show that Chunkyspread provides far better control over transmit load than Splitstream, while exhibiting comparable or better latency and responsiveness to churn.

Virtual machine distributed computing greatly simplifies the use of widespread computing resources by lowering the level of abstraction, benefiting both resource providers and users. Towards that end our Virtuoso middleware closely emulates the existing process of buying, configuring and using physical machines. Virtuoso's VNET component is a simple and efficient layer two virtual network tool that makes these virtual machines (VMs) appear to be physically connected to the home network of the user while simultaneously supporting arbitrary topologies and routing among them. Virtuoso's VTTIF component continually infers the communication behavior of the application running in a collection of VMs. The combination of overlays like VNET and inference frameworks like VTTIF has great potential to increase the performance, with no user or developer involvement, of existing, unmodified applications by adapting their virtual environments to the underlying computing infrastructure to best suit the applications. We show here how to use the continually inferred application topology and traffic to dynamically control three mechanisms of adaptation, VM migration, overlay topology, and forwarding to significantly increase the performance of two classes of applications, bulk synchronous parallel applications and transactional web ecommerce applications.

The growth of peer-to-peer applications on the Internet motivates interest in general purpose overlay networks. The construction of overlays connecting a large population of transient nodes poses several challenges. First, connections in the overlay should reflect the underlying network topology, in order to avoid overloading the network and to allow good application performance. Second, connectivity among active nodes of the overlay should be maintained, even in the presence of high failure rates or when a large proportion of nodes is not active. Finally, the cost of using the overlay should be spread evenly among peer nodes for fairness reasons as well as for the sake of application performance. To preserve scalability, we seek solutions to these issues that can be implemented in a fully decentralized manner and rely only on local knowledge from each node. In this

Overlay multicast constructs a multicast delivery tree among end hosts. Unlike traditional IP multicast, the nonleaf nodes in the tree are normal end hosts, which are potentially more susceptible to failures than routers and may leave the multicast group voluntarily. In these cases, all downstream nodes will be affected. Thus an important problem in overlay multicast is how to recover from node departures in order to minimize the disruption of service to those affected nodes. In this paper, we propose a proactive approach to restore overlay multicast trees. Rather than letting downstream nodes try to find a new parent after a node departure, each non-leaf node precalculates a parent-to-be for each of its children. When this nonleaf node is gone, all its children can find their respective new parents immediately. The salient feature of the approach is that each non-leaf node can compute a rescue plan for its children independently, and in most cases, rescue plans from multiple non-leaf nodes can work together for their children when they fail or leave at the same time. We develop a protocol for nodes to communicate with new parents so that the delivery tree can be quickly restored. Extensive simulations demonstrate that our proactive approach can recover from node departures 5 times faster than reactive methods in some cases, and 2 times faster on average.

This paper presents Araneola 1, a scalable reliable application-level multicast system for highly dynamic wide-area environments. Araneola supports multi-point to multi-point reliable communication in a fully distributed manner while incurring constant load (in terms of message and space complexity) on each node. For a tunable parameter k ≥ 3, Araneola constructs and dynamically maintains a basic overlay structure in which each node’s degree is either k or k +1, and roughly 90 % of the nodes have degree k. Empirical evaluation shows that Araneola’s basic overlay achieves three important mathematical properties of k-regular random graphs (i.e., random graphs in which each node has exactly k neighbors) with N nodes: (i) its diameter grows logarithmically with N; (ii) it is generally k-connected; and (iii) it remains highly connected following random removal of linear-size subsets of edges or nodes. The overlay is constructed and maintained at a low cost: each join, leave, or failure is handled locally, and entails the sending of only about 3k messages in total, independent of N. Moreover, this cost decreases as the churn rate increases. The low degree of Araneola’s basic overlay structure allows for allocating plenty of additional bandwidth for specific application needs. In this paper, we give an example for such a need — communicating with nearby nodes; we enhance the basic overlay with additional links chosen according to geographic

In this paper, we present an unstructured peer-to-peer network called GridMedia for live media streaming employing a push-pull approach. Each node in GridMedia randomly selects its neighbors in the overlay and uses push-pull method to fetch data from the neighbors. The pull mode in the unstructured overlay which is inherently robust can work well with the high churn rate in P2P environment while the push mode can efficiently reduce the accumulated latency observed at user nodes. A practical system based on this framework has been developed. And the performance evaluation of our system which is established on PlanetLab [8] demonstrates that the pull-push method in GridMedia achieves good qualities even in high group change rate. Furthermore, our system was adopted by CCTV to broadcast the Gala Evening for Spring Festival 2005 through the Internet and attracted more than 500,000 users all over the world at that night with the incredibly maximum concurrent users of 15,239.