The Internet community has recently been focused on peer-to-peer systems like Napster, Gnutella, and Freenet. The grand vision --- a decentralized community of machines pooling their resources to benefit everyone --- is compelling for many reasons: scalability, robustness, lack of need for administration, and even anonymity and resistance to censorship.

Consider a hierarchical network in which each node periodically issues a request for an object drawn from a fixed set of unit-size objects. Suppose further that the following conditions are satisfied: the frequency with which each node accesses each object is known; each node has a cache of known capacity; any cache can be accessed by any node; any request is satisfied by the closest node with a copy of the desired object, at a cost proportional to the distance between the accessing node and the closest copy. In such an environment, it is desirable to fill the available cache space with copies of objects in such a way that the average access cost is minimized. We provide both exact and approximate polynomial-time algorithms for this hierarchical placement problem. Our exact algorithm is based on a reduction to min-cost flow, and does not appear to be practical for large problem sizes. Thus we are motivated to search for a faster approximation algorithm. Our main result is a simple constant-factor approximation algorithm for the hierarchical placement problem that admits an efficient distributed implementation.

1 Introduction This paper analyzes algorithms for automated placement of file replicas in the Farsite [3] system, using both theory and simulation. In the Farsite distributed file system, multiple replicas of files are stored on multiple machines, so that files can be accessed even if some of the machines are down or inaccessible. The purpose of the placement algorithm is to determine an assignment of file replicas to machines that maximally exploits the availability provided by machines. The file placement algorithm is given a fixed value, R, for the number of replicas of each file. For systems reasons, we are most interested in a value of R = 3 [9]. However, to ensure that our results are not excessively sensitive to the file replication factor, we also provide tight bounds for R = 2 and lower bounds for all R (tight at different values of R).

The Internet community has recently been focused on peer-to-peer systems like Napster, Gnutella, and Freenet, The grand vision — a decentralized community of machines pooling their resources to benefit everyone — is compelling for many reasons: scalability, robustness, lack of need for administration, and even anonymity and resistance to censorship. Esisting peer-to-peer (P2P) systems have focused on specific application domains (e.g. music files) or on providing file-system-like capabilities; these systems ignore the semantics of data, An important question for the database communityis how data management can be applied to P2P, and what we can learn from and contribute to the P2P area. We address these questions, identify a number of potential research ideas in the overlap between data management and P2P systems, present some preliminary fundamental results, and describe our initial work in constructing a P2P data management system.

Consider an arbitrary distributed network in which large numbers of objects are continuously being created, replicated, and destroyed. A basic problem arising in such an environment is that of organizing a distributed directory service for locating object copies. In this paper, we present a new data tracking scheme for locating nearby copies of objects in arbitrary distributed environments. Our tracking scheme supports ecient accesses to data objects while keeping the local memory overhead low. In particular, our tracking scheme achieves an expected polylog(n)- approximation in the cost of any access operation, for an arbitrary network. The memory overhead incurred by our scheme is O(polylog(n)) times the maximum number of objects stored at any node, with high probability. We also show that our tracking scheme adapts well to dynamic changes in the network. 1 College of Computer Science, Northeastern University, Boston, MA 02115, rraj@ccs.neu.edu. Supported by NSF CAREER award NSF CCR{9983901. 2 Department of Computer Science and Engineering, Arizona State University, Tempe, AZ 85287-5406, aricha@asu.edu. Supported in part by NSF CAREER Award CCR{9985284 and NSF Grant CCR{9900304. 3 Max-Planck-Institut f ur Informatik, Saarbr ucken, Germany. Supported in part by the IST Programme of the EU under contrac number IST-1999-14186 (ALCOM-FT) 4 Compaq Corporation, San Jose, CA, Gayathri.Vuppuluri@compaq.com. This work was done while the author was a graduate student at Arizona State University, supported in part by NSF CAREER Award CCR{9985284. 1

We examine the replica placement aspect of a distributed peer-to-peer le system that replicates and stores files on ordinary desktop computers. It has been shown that some desktop machines are available for a greater fraction of time than others, and it is crucial not to place all replicas of any file on machines with low availability. In this paper we study the efficacy of three hill-climbing algorithms for file replica placement. Based on large-scale measurements, we assume that the distribution of machine availabilities be uniform. Among other results we show that the MinMax algorithm is competitive, and that for growing replication factor the MinMax and MinRand algorithms have the same asymptotic worst-case efficacy.

Stable storage is an important requirement for many applications. It is usually implemented over traditional file systems via synchronous write operations to disk.

In order to execute high performance applications on a cluster, it is highly desirable to provide distributed services that globally manage physical resources distributed over the cluster nodes. However, as a distributed service may use resources located on dierent nodes, it becomes sensitive to changes in the cluster con guration due to node addition, reboot or failure. In this paper, we propose a generic service performing dynamic resource management in a cluster in order to provide distributed services with high availability. This service has been implemented in the Gobelins cluster operating system. The dynamic resource management service we propose makes node addition and reboot nearly transparent to all distributed services of Gobelins and, as a consequence, fully transparent to applications. In the event of a node failure, applications using resources located on the failed node need to be restarted from a previously saved checkpoint but the availability of the cluster operating system is guaranteed, provided that its distributed services implement recon guration features.

The performance scalability of the Hurricane File System (HFS) is studied under the context of the K42 Operating System. Both systems were designed for scalability on large-scale, shared-memory, non-uniform memory access multiprocessors. However, scalability of HFS was never studied extensively. Microbenchmarks for reading, writing, creating, obtaining file attributes, and name lookup were used to measure scalability. As well, a macrobenchmark in the form of a simulated Web server was used. The unoptimized version of HFS scaled poorly. Optimizations to the meta-data cache in the form of (1) finer grain locks, (2) larger hash tables, (3) modified hash functions, (4) padded hash list headers and cache entries, and (5) a modified block cache free list, resulted in significant scalability improvements.

this document down to learning socket-level network programming and using POSIX threads