Abstract—Overlay networks have emerged as a powerful and highly flexible method for delivering content. We study how to optimize throughput of large transfers across richly connected, adaptive overlay networks, focusing on the potential of collaborative transfers between peers to supplement ongoing downloads. First, we make the case for an erasure-resilient encoding of the content. Using the digital fountain encoding approach, end hosts can efficiently reconstruct the original content of size from a subset of any symbols drawn from a large universe of encoding symbols. Such an approach affords reliability and a substantial degree of application-level flexibility, as it seamlessly accommodates connection migration and parallel transfers while providing resilience to packet loss. However, since the sets of encoding symbols acquired by peers during downloads may overlap substantially, care must be taken to enable them to collaborate effectively. Our main contribution is a collection of useful algorithmic tools for efficient summarization and approximate reconciliation of sets of symbols between pairs of collaborating peers, all of which keep message complexity and computation to a minimum. Through simulations and experiments on a prototype implementation, we demonstrate the performance benefits of our informed content-delivery mechanisms and how they complement existing overlay network architectures. Index Terms—Bloom filter, content delivery, digital fountain, erasure code, min-wise sketch, overlay, peer-to-peer, reconciliation. I.

We present new, simple, efficient data structures for approximate reconciliation of set differences, a useful standalone primitive for peer-to-peer networks and a natural subroutine in methods for exact reconciliation. In the approximate reconciliation problem, peers A and B respectively have subsets of elements SA and SB of a large universe U. Peer A wishes to send a short message M to peer B with the goal that B should use M to determine as many elements in the set SB SA as possible. To avoid the expense of round trip communication times, we focus on the situation where a single message M is sent. We motivate the performance tradeoffs between message size, accuracy and computation time for this problem with a straightforward approach using Bloom filters. We then introduce approximation reconciliation trees, a more computationally efficient solution that combines techniques from Patricia tries, Merkle trees, and Bloom filters. We present an analysis of approximation reconciliation trees and provide experimental results comparing the various methods proposed for approximate reconciliation.

In this paper, we propose a Caching Neighborhood Protocol (CNP) that describes an infrastructure upon which proxy servers can build dynamic caching hierarchies. Compared to the conventional schemes based on static caching hierarchies where proxy servers can only communicate with each other following predefined path configurations, the proposed dynamic scheme permits flexible communications among proxy servers. Such a scheme can decrease the response times of requests by increasing the availability of documents without compromising the currency of rendered documents. CNP is based on the concept of caching neighborhood, in which originating servers extend their spans and availability through their caching representatives, incarnated as proxy servers. The coherence of documents provided by the originating server and the cached copies on its caching representatives is maintained via employing the X tolerance coherence requirement. Some qualitative reasonings and studies on the traces of re...

this report we will define a "Web application " to be any software component that contributes either directly or indirectly to the handling of HTTP requests and responses (e.g. browsers, servers, and proxies). The study of Web performance can be viewed as characterizing or improving any of these components. In this report, we will focus on traditional Web usage, where the client application is a Web browser, being controlled interactively by a person. This is in contrast to non-interactive Web clients, such as the Web crawlers used by search engines, or XML-based data exchange between business databases.

To enhance the performance of Web caching, we proposed a dynamic Web caching hierarchy scheme based upon the Caching Neighborhood Protocol (CNP) [1][2] to facilitate establishing flexible cooperation partnerships among the Web origin servers and proxy servers. One of the key issues to the success of using CNP for the Web caching is request forwarding, i.e., to which destination server a client-side proxy server should forward the request that resulted in a cache miss. Hence, we introduce the concept and the properties of pyramid set, which consists of the most frequently accessed Web origin servers as observed by a proxy server, to construct the request forwarding tables for the client-side proxy servers. By examining the proxy server traces, we proposed and investigated several heuristics for selecting a server among a group of candidates to forward the request to. A simple scheme as well as a family of hybrid schemes considering both the request access distributions of the previous d...

This paper explores the economics of scalable network services by posing two simple questions. First, what is the difference between scale economies and scalability? Second, why and how should we scale network services for competition and cooperation? By answering these questions in the context of network services such as multicast, QoS and web caching, we gain some insight into the tradeoffs involved in the design of scalable network services.

A Content Distribution Network, or CDN, is a system to improve the delivery of content to the end users (or clients) in the Internet, in which popular content may be cached or replicated at a number of servers, placed closer to some of the client populations. The design of a CDN consists of defining: (a) which content should be replicated at each server (server content), (b) the number of servers and where they should be placed in the network (server placement), (c) which server a client's requests should be sent to (server selection) and (d) how the server responses should be routed to the clients (routing). CDNs were originally designed for traditional web files (i.e., HTML, image files). However, given the increase in streaming media (i.e., video and audio) content in the Internet, the development of efficient CDN design methods that take into account the special characteristics of media objects is of great interest. These characteristics include the sustained high bandwidth requirements and the new and complex tradeoffs introduced by multicast delivery. The main goal of this thesis is to develop methods for designing streaming media CDNs with (near) minimum delivery cost, where the delivery cost includes both the server and the network bandwidth costs. We propose and evaluate a new simple minimum cost caching algorithm for

The main purpose of Bloom filters is to build a space-efficient data structure for set membership. Indeed, to maximize space efficiency, correctness is sacrifized: if a given key is not in the set, then a Bloom filter may give the wrong answer (this is called a false positive), but the probability of such a wrong answer can be made small. A typical application of Bloom filters is web caching. An ISP may keep several levels of carefully located caches to speed up the loading of commonly viewed web pages, in particular for large data objects, such as images and videos. If a client requests a particular URL, then the service needs to determine quickly if the requested page is in one of its caches. False positives, while undesirable, are acceptable: if it turns out that a page thought to be in a cache is not there, it will be loaded from its native URL, and the "penalty " is not much worse than not having the cache in the first place. So here is the formal set-up: we want to represent n-element sets S = {s1,..., sn} from a very large universe U, with |U | = u>> n. (Think of U as the set of URLs, n as the cache size, and S as the URLs of those web pages that are currently in the cache.) We want to support insertions and membership queries ("Given x 2 U, is x 2 S?") so that: 1. If the answer is No, then x 62 S. 2. If the answer is Yes, then x may or may not be in S, but the probability that x 62 S (false positive) is low. Both insertions and membership queries should be performed in constant time. (Bloom filters can also be made to support deletions, but we won't worry about those.)

Web caching is generally acknowledged as an important service for alleviating focused overloads when certain web servers' contents suddenly become popular. Cooperative caching systems are more effective than independent caches due to the larger collective backing store that cooperation creates. One such system currently being developed at UCLA, Adaptive Web Caching (AWC), uses an application-level forwarding table to locate the nearest copy of a requested URL's contents. This paper describes one specific design in AWC, a simple URL table compression algorithm allowing efficient content information sharing among neighboring caches. The compression algorithm is based on a hierarchical URL decomposition to aggregate URLs sharing common prefixes and an incremental hashing function to minimize collisions between prefixes. The algorithm's collision rate is derived analytically and verified by five sets of Web trace data. The results demonstrate that the collision rate is bounded and has little impact on page fetching latency. Finally, this compression method is compared to the Summary Cache method.