Hash-based scalable distributed data structures (SDDSs), like LH * and DDH, for networks of intcmonnected ampllters (multicomputers) were shown to open new perspectives for fik management. We prcpose a family of ordered SDDSs, called P, providing for ordered and dynamic files on mutticomputers, and thus for more etlicknt pmeessing of range queries and of ordered traversak of files The basic algorithm termed RP*K builds the file with the same key space partitioning as a Etree, but avoids indexes through the use of multi&. The a&nithms, RP*c and RP*s enbance throughput for faster network adding the indexes on clients, or OII clients and sexve-rs, while eithe-r decmsing or avoiding multicast. Rpo files are shown highly efficient with access performance exceeding traditional files by an order of magnitude or two, an & for non-range queries very close to LH*. 1.

This paper reexamines the rules of thumb for the design of data storage systems. Briefly, it looks at storage, processing, and networking costs, ratios, and trends with a particular focus on performance and price/performance. Amdahl's ratio laws for system design need only slight revision after 35 years---the major change being the increased use of RAM. An analysis also indicates storage should be used to cache both database and web data to save disk bandwidth, network bandwidth, and people's time. Surprisingly, the 5-minute rule for disk caching becomes a cache-everything rule for web caching.

We address the problem of finding parallel plans for SQL queries using the two-phase approach of join ordering followed by parallelization. We focus on the parallelization phase and develop algorithms for exploiting pipelined parallelism. We formulate parallelization as scheduling a weighted operator tree to minimize response time. Our model of response time captures the fundamental tradeoff between parallel execution and its communication overhead. We assess the quality of an optimization algorithm by its performance ratio which is the ratio of the response time of the generated schedule to that of the optimal. We develop fast algorithms that produce near-optimal schedules -- the performance ratio is extremely close to 1 on the average and has a worst case bound of about 2 for many cases. 1 Introduction We address the problem of parallel query optimization, which is to find optimal parallel plans for executing SQL queries. Following Hong and Stonebraker [HS91], we break the optimi...

It is often important for the correct processes in a distributed system to reach agreement, despite the presence of some faulty processes. Byzantine agreement (BA) is a paradigm problem that attempts to isolate the key features of reaching agreement. We focus here on the number of messages required to reach BA, with particular emphasis on the number of messages required in the failure-free runs, since these are the ones that occur most often in practice. The number of messages required is sensitive to the types of failures considered. In earlier work, Amdur et al. [1990] established tight upper and lower bounds on the worst- and average-case number of messages required in failurefree runs for crash failures. We provide tight upper and lower bounds for all remaining types of failures that have been considered in the literature on the BA problem: receiving omission, sending omission and general omission failures, as well as arbitrary failures with or without message authentication. We ...

We address the problem of finding parallel plans for SQL queries using the two-phase approach of join ordering and query rewrite (JOQR) followed by parallelization. We focus on the JOQR phase and develop optimization algorithms that account for communication as well as computation costs. Using a model based on representing the partitioning of data as a color, we devise an efficient algorithm for the problem of choosing the partitioning attributes in a query tree so as to minimize total cost. We extend our model and algorithm to incorporate the interaction of data partitioning with conventional optimization choices such as access methods and strategies for computing operators. Our algorithms apply to queries that include operators such as grouping, aggregation, intersection and set difference in addition to joins. 1 Introduction An important challenge in parallel database systems [DG92, Val93, BCC + 90, DGG + 86] is parallel query optimization. This is the problem of finding opt...

Abstract—There is an ever-increasing demand for more complex transactions and higher throughputs in transaction processing systems leading to higher degrees of transaction concurrency and, hence, higher data contention. The conventional two-phase locking (2PL) Concurrency Control (CC) method may, therefore, restrict system throughput to levels inconsistent with the available processing capacity. This is especially a concern in shared-nothing or data-partitioned systems due to the extra latencies for internode communication and a reliable commit protocol. The optimistic CC (OCC) is a possible solution, but currently proposed methods have the disadvantage of repeated transaction restarts. We present a distributed OCC method followed by locking, such that locking is an integral part of distributed validation and two-phase commit. This method ensures at most one re-execution, if the validation for the optimistic phase fails. Deadlocks, which are possible with 2PL, are prevented by preclaiming locks for the second execution phase. This is done in the same order at all nodes. We outline implementation details and compare the performance of the new OCC method with distributed 2PL through a detailed simulation that incorporates queueing effects at the devices of the computer systems, buffer management, concurrency control, and commit processing. It is shown that for higher data contention levels, the hybrid OCC method allows a much higher maximum transaction throughput than distributed 2PL in systems with high processing capacities. In addition to the comparison of CC methods, the simulation study is used to study the effect of varying the number of computer systems with a fixed total processing capacity and the effect of locality of access in each case. We also describe several interesting variants of the proposed OCC method, including methods for handling access variance, i.e., when rerunning a transaction results in accesses to a different set of objects. Index Terms—Distributed database systems, transaction processing, optimistic concurrency control, access invariance, commit protocols, system performance modeling.

We introduce a class of novel multiprocessor scheduling problems that arise in the optimization of SQL queries for parallel machines. These consist of scheduling a tree of interdependent communicating operators while exploiting both inter-operator and intra-operator parallelism. We develop algorithms for the specific problem of scheduling a Pipelined Operator Tree in which all operators run in parallel using inter-operator parallelism. Weights associated with nodes and edges represent respectively the cost of operators and communication. Communication cost is incurred only if adjacent operators are assigned different processors. The optimization problem is to assign operators to processors so as to minimize the maximum processor load. We develop two approximation algorithms for this NP-hard problem. The faster algorithm has a performance ratio of 3.56 while the slower algorithm has a ratio of 2.87. 1 Introduction Exploiting parallel execution [DG92, Val93] to speed up database querie...

: We investigate the power of a broadcast mechanism in a distributed network. We do so by considering the problem of synchronizing clocks in an error-free network, under the assumption that there is no upper bound on message transmission time, but that broadcast messages are guaranteed to be received within an interval of size ffl, for some fixed constant ffl. This is intended to be an idealization of what happens in multiple access networks, such as the Ethernet. We then consider tradeoffs between the type and number of broadcasts, and the tightness of synchronization. Our results include (1) matching upper and lower bounds of (1 + 1 K )ffl on the precision of clock synchronization attainable for n 3 processes using K (n \Gamma 1)-casts, 3 K n, (2) matching upper and lower bounds of (1 + 1 n )ffl on the precision of clock synchronization attainable for n 3 processes using an arbitrary number of (n \Gamma 1)-casts, and (3) matching upper and lower bounds of (1 + n\Gamma2 n ...

The Distributed Consensus problem involves n processors each of which holds an initial binary value. At most t processors may be faulty and ignore any protocol (even behaving maliciously), yet it is required that non-faulty processors eventually agree on a common value that was initially held by one of them. The quality of a consensus protocol is measured using the following parameters: the number of processors n, the number of rounds of message exchange r and the total number of bits transmitted B. The known lower bounds are respectively 3t + 1, t + 1 and\Omega\Gamma nt). While no known protocol is optimal in all three aspects simultaneously, the protocol presented in this paper takes further steps in this direction: it achieves for the first time asymptotically optimal bit transfer (B = O(nt)), together with optimal number of processors n and nearly optimal r. Previously existing consensus protocols required B = \Omega\Gamma n 2 t), regardless of the other parameters' values. I...

Abstract- The performance of high-volume transaction pro-cessing systems for business applications is determined by the degree of contention for hardware resources as well as for data. Hardware resource requirements may be met cost-effectively with a data-partitioned or shared-nothing architecture. However, the two-phase locking (2PL) concurrency control method may restrict the performance of a shared-nothing system more se-verely than that of a centralized system due to increased lock holding times. Deadlock detection and resolution are an added complicating factor in shared-nothing systems. In this paper, we describe distributed Wait-Depth Limited (WDL) concurrency control (CC), a locking-based distributed CC method that limits the wait-depth of blocked transactions to one, thus preventing the occurrence of deadlocks. Several implementations of distributed WDL which vary in the number of messages and the amount