Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving). Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the simple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable. 1

A method is presented for permitt.ing record updates by long-lived transactions without forbidding simultaneous access by other users to records modified. Earlier methods presented separately by Gawlick and Reuter are comparable but concentrate on “hot-spot ” situations, where even short transactions cannot lock frequently accessed fields without causing bottlenecks. The Escrow Method offered here is designed to support nonblocking record updates by transactions that are “long lived” and thus require long periods to complete. Recoverability of intermediate results prior to commit thus becomes a design goal, so that updates as of a given time can be guaranteed against memory or media failure while still retaining the prerogative to abort. This guarantee basically completes phase one of a two-phase commit, and several advantages result: (1) As with Gawlick’s and Reuter’s methods, high-concurrency items in the database will not act as a bottleneck; (2) transaction commit of different updates can be performed asynchronously, allowing natural distributed transactions; indeed, distributed transactions in the presence of delayed messages or occasional line disconnection become feasible in a way that we argue will tie up minimal resources for the purpose intended; and (3) it becomes natural to allow for human interaction in the middle of a transaction without loss of concurrent access or any special difficulty for the application programmer. The Escrow Method, like Gawlick’s Fast Path and Reuter’s Method, requires the database system to be an “expert ” about the type of transactional updates performed, most commonly updates involving incremental changes to aggregate quantities. However, the Escrow Method is extendable to other types of updates.

This paper deals with the transaction management aspects of the R * distributed database system. It concentrates primarily on the description of the R * commit protocols, Presumed Abort (PA) and Presumed Commit (PC). PA and PC are extensions of the well-known, two-phase (2P) commit protocol. PA is optimized for read-only transactions and a class of multisite update transactions, and PC is optimized for other classes of multisite update transactions. The optimizations result in reduced intersite message traffic and log writes, and, consequently, a better response time. The paper also discusses R*‘s approach toward distributed deadlock detection and resolution.

Cloud storage solutions promise high scalability and low cost. Existing solutions, however, differ in the degree of consistency they provide. Our experience using such systems indicates that there is a non-trivial trade-off between cost, consistency and availability. High consistency implies high cost per transaction and, in some situations, reduced availability. Low consistency is cheaper but it might result in higher operational cost because of, e.g., overselling of products in a Web shop. In this paper, we present a new transaction paradigm, that not only allows designers to define the consistency guarantees on the data instead at the transaction level, but also allows to automatically switch consistency guarantees at runtime. We present a number of techniques that let the system dynamically adapt the consistency level by monitoring the data and/or gathering temporal statistics of the data. We demonstrate the feasibility and potential of the ideas through extensive experiments on a first prototype implemented on Amazon’s S3 and running the TPC-W benchmark. Our experiments indicate that the adaptive strategies presented in the paper result in a significant reduction in response time and costs including the cost penalties of inconsistencies. 1.

When designing an extensible operating system, a developer must ensure that the operating system is protected from misbehaved extensions. Two kinds of protection are needed: first, extensions should not violate the operating system’s interface, and second, extensions should not be able to leave the operating system in an inconsistent state. The major research contributions of this thesis include: The design and evaluation of MiSFIT, a software fault isolation tool for the x86 architecture that ensures that extensions do not violate the operating system’s interface and incurs minimal overhead. The design and evaluation of VINO Lightweight Transactions, a low-overhead mechanism that allows the kernel to maintain its consistency in the face of ill-behaved extensions. Experiments that show the end-to-end overhead of MiSFIT and VLT protection is low, on the order of 1-2%, and the net performance gain possible from using application-specific extensions is significant, in some cases more than 20%. A cost-benefit framework for comparing extension technologies and an evaluation comparing

Remote backup copies of databases are often maintained to ensure availability of data even in the presence of exten-sive failures, for which local replication mechanisms may be inadequate. We present two versions of an epoch algo-rithm for maintaining a consistent remote backup copy of a database. The algorithms ensure scalability, which makes them suitable for very large databases. The correctness and the performance of the algorithms are discussed, and an additional application for distributed group commit is given. 1.

The notion of compensation is widely used as means of backward recovery in long-lived transactions as well as business processes supported by workflow management systems. In general, it is non-trivial to design compensating tasks for tasks in the context of a workflow. Actually, a task does not have to be compensatable. In this paper, we first look into the requirements that a compensating task has to satisfy. Then we introduce a new mechanism called confirmation. With the help of confirmation, we can modify some non-compensatable tasks so that they become compensatable. This greatly improves backward recovery for workflow applications in case of failures. To effectively incorporate confirmation and compensation into the workflow management environment, a three level bottom-up workflow design method is introduced. The implementation issues of this design are also discussed.

We report on the issues raised when designing a customizable locking mechanism for Persistent Java, a type-safe, object-oriented, orthogonally persistent system based on the language Java. Customizable locking mechanisms are supported by locking capabilities. A locking capability is a book-keeper of locks that automatically acquires locks with a customizable conflict detection mechanism. It implements the concepts of delegation of locks and ignorable conflicts, automatically keeps track of the dependencies created because of allowance of conflicts, supports querying of details about these dependencies, and supports the setting of user-defined notifications for conflicts that can't be ignored. Locking capabilities don't change the Java language specification, and allow use of any Java classes without changing a single line of their code to implement the body of transactions. We also present several novel techniques to implement efficiently this customizable locking mechanism in a persis...

Many applications, such as telecommunication, process control, and virtual reality, require real-time access to database. Main-memory DBMS, which becomes feasible with the increasing availability of large and relatively cheap memory, can provide better performance than disk-based systems for real-time applications. This paper presents an overall architecture of M 2 RT, a Main-Memory Real-Time DBMS, and an object-oriented design of its storage system called M 2 RTSS. M 2 RTSS provides classes that implement the core functionality of storage management, real-time transaction scheduling, and recovery. Implementation-specific information is encapsulated in these classes and extensions can be made by inheritance. With object-oriented features, M 2 RTSS can easily incorporate new development in application requirements and the result of ongoing research in realtime systems. Keywords: object-oriented design and implementation, extensibility, main-memory DBMS, real-time DBMS 1 Introdu...

B-trees have been ubiquitous in database management systems for several decades, and they are used in other storage systems as well. Their basic structure and basic operations are well and widely understood including search, insertion, and deletion. Concurrency control of operations in B-trees, however, is perceived as a difficult subject with many subtleties and special cases. The purpose of this survey is to clarify, simplify, and structure the topic of concurrency control in B-trees by dividing it into two sub-topics and exploring each of them in depth. 1