optimistic, replication, survey Replication is a key enabling technology in distributed data sharing systems for improving both availability and performance. This paper surveys optimistic replication algorithms, which allow replica contents to diverge in the short term, in order to support concurrent work and to tolerate failures in low-quality communication links. The importance of such techniques is increasing as collaboration through wide-area and mobile networks is becoming more popular. Optimistic replication algorithms employ techniques vastly different from those for traditional pessimistic algorithms. Whereas a pessimistic algorithm relies on synchronous

We present a new replication algorithm that supports replication of a large number of objects on a diverse set of nodes. The algorithm allows replica sets to be changed dynamically on a per-object basis. It tolerates most types of failures, including multiple node failures, network partitions, and sudden node retirements. These advantages make the algorithm particularly attractive in large cluster-based data services that experience frequent failures and configuration changes. We prove the correctness of the algorithm and show that its performance is near-optimal.

Performance improvements to networked applications can have unintended consequences. In a study of the performance of the Network News Transport Protocol (NNTP), the initial results suggested it would be useful to disable TCP's Nagle algorithm for this application. Doing so significantly improved latencies. However, closer observation revealed that with the Nagle algorithm disabled, the application was transmitting an order of magnitude more packets. We found that proper application buffer management significantly improves performance, but that the Nagle algorithm still slightly increases mean latency. We suggest that modifying the Nagle algorithm would eliminate this cost. 1 Introduction The performance of a client/server application depends on the appropriate use of the transport protocol over which it runs. In today's Internet, most application protocols run over the Transport Control Protocol (TCP) [12] or the User Datagram Protocol (UDP) [11]. Usenet, one of the main Internet ap...

Optimistic replication algorithms allow replica contents to be stale but in a controlled way. By propagating updates in background and allowing any replica to be read most of the time, they become more available and more efficient than traditional replication algorithms using unreliable network media and inexpensive computers. The use of optimistic replication has recently grown explosively due to the proliferation of the Internet and mobile computing devices, but its systematic study has begun only recently. We survey optimistic replication algorithms in this report. In particular, we focus on the update propagation mechanisms invented for optimistic replication and study how these mechanisms affect the functional characteristics of the systems, including data consistency guarantees and performance.

This thesis introduces a novel approach to performance evaluation, called application-specific benchmarking, and presents techniques for designing and constructing meaningful benchmarks. A traditional benchmark usually includes a fixed set of programs that are run on different systems to produce a single figure of merit, which is then used to rank system performance. This approach often overlooks the relevance between the benchmark programs and the real applications they are supposed to represent. When the behaviors of the benchmark programs do not match those of the intended application, the benchmark scores are uninformative, and sometimes can be misleading. Furthermore, with the rapid pace of application development, it is impractical to create a new standard benchmark whenever a new “killer ” application emerges. The application-specific benchmarking approach incorporates characteristics of the application of interest into the benchmarking process, yielding performance metrics that reflect the expected behavior of a particular application across a range of different

Usenet is a popular distributed messaging and file sharing service: servers in Usenet flood articles over an overlay network to fully replicate articles across all servers. However, replication of Usenet’s full content requires that each server pay the cost of receiving (and storing) over 1 Tbyte/day. This paper presents the design and implementation of UsenetDHT, a Usenet system that allows a set of cooperating sites to keep a shared, distributed copy of Usenet articles. UsenetDHT consists of client-facing Usenet NNTP front-ends and a distributed hash table (DHT) that provides shared storage of articles across the wide area. This design allows participating sites to partition the storage burden, rather than replicating all Usenet articles at all sites. UsenetDHT requires a DHT that maintains durability despite transient and permanent failures, and provides high storage performance. These goals can be difficult to provide simultaneously: even in the absence of failures, verifying adequate replication levels of large numbers of objects can be resource intensive, and interfere with normal operations. This paper introduces Passing Tone, a new replica maintenance algorithm for DHash [7] that minimizes the impact of monitoring replication levels on memory and disk resources by operating with only pairwise communication. Passing Tone’s implementation provides performance by using data structures that avoid disk accesses and enable batch operations. Microbenchmarks over a local gigabit network demonstrate that the total system throughput scales linearly as servers are added, providing 5.7 Mbyte/s of write bandwidth and 7 Mbyte/s of read bandwidth per server. UsenetDHT is currently deployed on a 12-server network at 7 sites running Passing Tone over the wide-area: this network supports our research laboratory’s live 2.5 Mbyte/s Usenet feed and 30.6 Mbyte/s of synthetic read traffic. These results suggest a DHT-based design may be a viable way to redesign Usenet and globally reduce costs. 1

This dissertation proposes functionally homogeneous clustering (FHC), a new software architecture for building data-intensive Internet services that are manageable, available, fast, and inexpensive. FHC lets any node in the cluster manage any function and any piece of data, freeing humans from making specific decisions about the workload distribution. Its dynamic and self-regulative nature is the key to its scalability. This dissertation also presents three mechanisms that synergistically realize this architecture: automatic reconfiguration, high-throughput replication, and fine-grain load balancing. FHC offers an efficient and scalable automatic reconfiguration mechanism for redistributing functions and data after configuration change. It ensures that users can access all the data on live nodes after any number of failures. The replication mechanism stores important data on multiple disks with small overhead, while ensuring the consistency of their contents. The load balancing mechanism distributes incoming data evenly among nodes and masks the non-uniformity in the workloads and the cluster configuration. FHC scales without sacrificing its service quality by taking advantage of the semantics of data-intensive Internet services. For example, the name database used to locate on-disk data is storedonly in memory and is recomputed after failure by scanning disks. While such a design makes the contents and operations of the name database application-specific, it makes the system fast and robust. Our replication algorithm also takes advantage of application semantics and ensures only eventual data consistency. In return, this strategy makes the system extremely resilient against failures. We develop the Porcupine email server as proof of the concept of functionally homogeneous clustering. Porcupine distributes user management and email message storage dynamically to maximize system throughput and ensure continuous service to all users. It replicates the user profile and email messages to ensure their availability. We evaluate the manageability, availability, and performance of Porcupine on a 30-node PC cluster. Through the evaluation, we show that Porcupine’s performance indeed scales well and that it reacts to configuration changes gracefully and quickly. We also show that Porcupine’s load balancing service efficiently utilizes heterogeneous hardware resources and handles non-uniform workloads by automatically discovering idle resources in the cluster.

Optimistic replication algorithms allow replica contents to become stale but in a controlled way. In return, they become far more efficient and available than traditional replication algorithms that keep all the replicas consistent, especially when the network and computers are unreliable. The use of optimistic replication has grown explosively due to the proliferation of the use of the Internet and mobile computing devices, but its systematic study has begun only recently. This report surveys optimistic replication algorithms. In particular, it focuses on mechanisms for propagating updates among replicas and studies how they affect the functional characteristics of the systems, including data consistency guarantees and performance.

UsenetDHT is a system that reduces the storage and bandwidth resources required to run a Usenet server by spreading the burden of data storage across participants. UsenetDHT distributes data using a distributed hash table. The amount of data that must be stored on each node participating in UsenetDHT scales inversely with the number of participating nodes. Each node's bandwidth requirements are proportional to the fraction of articles read rather than to the total number posted.

Distributed hash tables (DHTs) have been proposed as a generic, robust storage infrastructure for simplifying the construction of large-scale, wide-area applications. For example, UsenetDHT is a new design for Usenet News developed in this thesis that uses a DHT to cooperatively deliver Usenet articles: the DHT allows a set of N hosts to share storage of Usenet articles, reducing their combined storage requirements by a factor of O(N). Usenet generates a continuous stream of writes that exceeds 1 Tbyte/day in volume, comprising over ten million writes. Supporting this and the associated read workload requires a DHT engineered for durability and efficiency. Recovering from network and machine failures efficiently poses a challenge for DHT replication maintenance algorithms that provide durability. To avoid losing the last replica, replica maintenance must create additional replicas when failures are detected. However,