When designing distributed web services, there are three properties that are commonly desired: consistency, availability, and partition tolerance. It is impossible to achieve all three. In this note, we prove this conjecture in the asynchronous network model, and then discuss solutions to this dilemma in the partially synchronous model.

Massive storage systems typically replicate and partition data over many potentially-faulty components to provide both reliability and scalability. Yet many commerciallydeployed systems, especially those designed for interactive use by customers, sacrifice stronger consistency properties in the desire for greater availability and higher throughput. This paper describes the design, implementation, and evaluation of CRAQ, a distributed object-storage system that challenges this inflexible tradeoff. Our basic approach, an improvement on Chain Replication, maintains strong consistency while greatly improving read throughput. By distributing load across all object replicas, CRAQ scales linearly with chain size without increasing consistency coordination. At the same time, it exposes noncommitted operations for weaker consistency guarantees when this suffices for some applications, which is especially useful under periods of high system churn. This paper explores additional design and implementation considerations for geo-replicated CRAQ storage across multiple datacenters to provide locality-optimized operations. We also discuss multi-object atomic updates and multicast optimizations for large-object updates. 1

Many distributed services are hosted at large, shared, geographically diverse data centers, and they use replication to achieve high availability despite the failure of an entire data center. Recent events show that non-crash faults occur in these services and may lead to long outages. While Byzantine Fault Tolerance (BFT) could be used to withstand these faults, current BFT protocols can become unavailable if a small fraction of their replicas are unreachable. This is because existing BFT protocols favor strong safety guarantees (consistency) over liveness (availability). This paper presents a novel BFT state machine replication protocol called Zeno, that trades consistency for higher availability. In particular, Zeno replaces linearizability with eventual consistency, where clients can temporarily miss each other’s updates but when the network is stable the states from the individual partitions are merged by having the replicas agree on a total order for the requests. We have built a prototype of Zeno and our evaluation using micro-benchmarks shows that Zeno provides better availability than traditional BFT protocols, and that its impact on performance is low, even when partitions occur or heal. 1

As distributed applications such as Grid and enterprise systems scale up and become increasingly complex, their authorization infrastructures—based predominantly on the request-response paradigm—are facing the challenges in terms of fragility and poor scalability. We propose an approach where each application server caches previously received authorizations at a secondary decision point (SDP) and shares them with other servers to mask authorization server failures and network delays. The main contribution of this paper is the design of the cooperative secondary authorization recycling (CSAR) system and its evaluation using simulation and prototype implementation. The results demonstrate that our approach improves the availability and the performance of authorization infrastructures. Specifically, by sharing secondary authorizations among SDPs, the cache hit rate—an indirect metric of availability—can reach 70 % even when only 10% of authorizations are cached. The performance is also improved, as the average time of authorizing a request to an application is reduced by up to 30%.

Spinnaker is an experimental datastore that is designed to run on a large cluster of commodity servers in a single datacenter. It features key-based range partitioning, 3-way replication, and a transactional get-put API with the option to choose either strong or timeline consistency on reads. This paper describes Spinnaker’s Paxos-based replication protocol. The use of Paxos ensures that a data partition in Spinnaker will be available for reads and writes as long a majority of its replicas are alive. Unlike traditional master-slave replication, thisistrueregardlessofthefailuresequencethat occurs. We show that Paxos replication can be competitive with alternatives that provide weaker consistency guarantees. Compared to an eventually consistent datastore, we show that Spinnaker can be as fast or even faster on reads and only 5 % to 10 % slower on writes. 1.

We present several general, broadly applicable mechanisms that enable computations to execute with reduced resources, typically at the cost of some loss in the accuracy of the result they produce. We identify several general computational patterns that interact well with these resource reduction mechanisms, present a concrete manifestation of these patterns in the form of simple model programs, perform simulationbased explorations of the quantitative consequences of applying these mechanisms to our model programs, and relate the model computations (and their interaction with the resource reduction mechanisms) to more complex benchmark applications drawn from a variety of fields.

The PODC Steering Committee is pleased to announce that PODC 2004 will be held in St. John's, Newfoundland. This will be the thirteenth PODC to be held in Canada but the first to be held there since 1995. Many thanks to Krishnamurthy Vidyasankar, Professor at the Memorial University of Newfoundland, who will serve as Local Arrangements Chair. The General Chair will

Geo-replicated, distributed data stores that support complex online applications, such as social networks, must provide an “alwayson” experience where operations always complete with low latency. Today’s systems often sacrifice strong consistency to achieve these goals, exposing inconsistencies to their clients and necessitating complex application logic. In this paper, we identify and define a consistency model—causal consistency with convergent conflict handling, or causal+—that is the strongest achieved under these constraints. We present the design and implementation of COPS, a key-value store that delivers this consistency model across the wide-area. A key contribution of COPS is its scalability, which can enforce causal dependencies between keys stored across an entire cluster, rather than a single server like previous systems. The central approach in COPS is tracking and explicitly checking whether causal dependencies between keys are satisfied in the local cluster before exposing writes. Further, in COPS-GT, we introduce get transactions in order to obtain a consistent view of multiple keys without locking or blocking. Our evaluation shows that COPS completes operations in less than a millisecond, provides throughput similar to previous systems when using one server per cluster, and scales well as we increase the number of servers in each cluster. It also shows that COPS-GT provides similar latency, throughput, and scaling to COPS for common workloads.

Distributed data stores have recently become increasingly popular. This can be largely attributed to their impressive ability to scale. However, to achieve this scalability, they sacrifice properties such as atomicity and consistency, important properties when constructing simple control flows and asserting correctness. CloudTPS is a transactional layer on top of distributed data stores, bringing back transactional consistency. Its request latency is related to how many servers are involved in servicing each transaction, and in this thesis we explore methods to reduce this number. First, we introduce a data placement mechanism to CloudTPS, providing a real environment to test and benchmark placement policies. Second, we implement a distributed approach to minimum k-cut partitioning including load-balancing and a tunable migration cost-model, both of which can

New data-consistency models make it possible for cloud computing developers to replicate soft state without encountering the limitations associated with the CAP theorem. The CAP theorem explores tradeoffs between consistency, availability, and partition tolerance, and concludes that a replicated service can have just two of these three properties. 1,2 To prove CAP, researchers construct a scenario in which a replicated service is forced to respond to conflicting requests during a wide-area network outage, as might occur if two different datacenters hosted replicas of some single service, and received updates at a time when the network link between them was down. The replicas respond without