Database management systems will continue to manage large data volumes. Thus, efficient algorithms for accessing and manipulating large sets and sequences will be required to provide acceptable performance. The advent of object-oriented and extensible database systems will not solve this problem. On the contrary, modern data models exacerbate it: In order to manipulate large sets of complex objects as efficiently as today’s database systems manipulate simple records, query processing algorithms and software will become more complex, and a solid understanding of algorithm and architectural issues is essential for the designer of database management software. This survey provides a foundation for the design and implementation of query execution facilities in new database management systems. It describes a wide array of practical query evaluation techniques for both relational and post-relational database systems, including iterative execution of complex query evaluation plans, the duality of sort- and hash-based set matching algorithms, types of parallel query execution and their implementation, and special operators for emerging database application domains.

This paper presents the design and an initial performance evaluation of the query optimizer generator designed for the EXODUS extensible database system. Algebraic transformation rules are translated into an executable query optimizer, which transforms query trees and selects methods for executing operations according to cost functions associated with the methods. The search strategy avoids exhaustive search and it modifies itself to take advantage of past experience. Computational results show that an optimizer generated for a relational system produces access plans almost as good as those produced by exhaustive search, with the search time cut to a small fraction.

With non-traditional application areas such as engineering design, image/voice data management, scientific/statistical applications, and artificial intelligence systems all clamoring for ways to store and efficiently process larger and larger volumes of data, it is clear that traditional database technology has been pushed to its lim-its. It also seems clear that no single database system will be capable of simultaneously meeting the functionality and performance requirements of such a diverse set of applications. In this paper we describe the initial design of EXODUS, an extensible database system that will facilitate the fast development of high-performance, application-specific database systems. EXODUS provides certain kernel facilities, including a versatile storage manager and a type manager. In addition, it provides an architectural framework for building application-specific database sys-tems, tools to partially automate the generation of such systems, and libraries of software components (e.g., access methods) that are likely to be useful for many application domains.

In this paper, we show how compression can be integrated into a relational database system. Specifically, we describe how the storage manager, the query execution engine, and the query optimizer of a database system can be extended to deal with compressed data. Our main result is that compression can significantly improve the response time of queries if very light-weight compression techniques are used. We will present such light-weight compression techniques and give the results of running the TPC-D benchmark on a so compressed database and a non-compressed database using the AODB database system, an experimental database system that was developed at the Universities of Mannheim and Passau. Our benchmark results demonstrate that compression indeed offers high performance gains (up to 55%) for IO-intensive queries and moderate gains for CPU-intensive queries. Compression can, however, also increase the running time of certain update operations. In all, we recommend to extend today's da...

The task of increasing the efficiency of database queries has recieved considerable attention. In this paper we describe the improvement of queries expressed as list comprehensions in a lazy functional language. The database literature identifies four algebraic and two implementation-based improvement strategies. For each strategy we show an equivalent improvement for queries expressed as list comprehensions. This means that welldeveloped database algorithms that improve queries using several of these strategies can be emulated to improve comprehension queries. We are also able to improve queries which require greater power than that provided by the relational algebra. Most of the improvements entail transforming a simple, inefficient query into a more complex, but more efficient form. We illustrate each improvement using examples drawn from the database literature. 1 Introduction The functional programming community is often accused of being too inward looking. Functional languages a...

Database programming requires knowledge of database semantics both to maintain database integrity and to explore more optimization opportunities. Automated programming of database transactions is desirable and feasible. In general, transactions use simple constructs and algorithms; specifications of database semantics are available; and transactions perform small incremental updates to database contents. Automated programming in such a restricted but well-understood and important domain is promising. We approach the synthesis of database transactions that preserve the validity of integrity constraints using deductive techniques. A transaction logic for a fairly expressive class of transactions is developed as the formalism within which the synthesis is conducted. Transactions are generated as the by-product of proving specifications in the logic. The Manna-Waldinger deductive-tableau system is extended with inference rules for the extraction of transactions from proofs, which require t...

A Functional Database Phil Trinder D.Phil. Thesis Wolfson College Michaelmas Term, 1989 This thesis explores the use of functional languages to implement, manipulate and query databases. Implementing databases. A functional language is used to construct a database manager that allows efficient and concurrent access to shared data. In contrast to the locking mechanism found in conventional databases, the functional database uses data dependency to provide exclusion. Results obtained from a prototype database demonstrate that data dependency permits an unusual degree of concurrency between operations on the data. The prototype database is used to exhibit some problems that seriously restrict concurrency and also to demonstrate the resolution of these problems using a new primitive. The design of a more realistic database is outlined. Some restrictions on the data structures that can be used in a functional database are also uncovered. Manipulating databases. Functions over the database a...

Testing of database applications is crucial for ensuring high software quality as undetected faults can result in unrecoverable data corruption. The problem of database application testing can be broadly partitioned into the problems of test cases generation, test data preparation and test outcomes verification. Among the three problems, the problem of test cases generation directly affects the effectiveness of testing. Conventionally, database application testing is based upon whether or not the application can perform a set of predefined functions. While it is useful to achieve a basic degree of quality by considering the application to be a black box in the testing process, white box testing is required for more thorough testing. However, the semantics of the Structural Query Language (SQL) statements embedded in database applications are rarely considered in conventional white box testing techniques. In this paper, we propose to complement white box techniques with the inclusion of the SQL semantics. Our approach is to transform the embedded SQL statements to procedures in some general-purpose programming language and thereby generate test cases using conventional white box testing techniques. Additional test cases that are not covered in traditional white box testing are generated to improve the effectiveness of database application testing. The steps of both SQL statements transformation and test cases generation will be explained and illustrated using an example adapted from a course registration system. We successfully identify additional faults involving the internal states of databases. 1

This paper presents a new query algebra based on fold iterations that facilitates database implementation. An algebraic normalization algorithm is introduced that reduces any program expressed in this algebra to a canonical form that generates no intermediate data structures and has no more nested iterations than the initial program. Given any inductive data type, our system can automatically synthesize the definition of the fold operator that traverses instances of this type, and, more importantly, it can produce the necessary transformations for optimizing expressions involving this fold operator. Database implementation in our framework is controlled by userdefined mappings from abstract types to physical structures. The optimizer uses this information to translate abstract programs and queries into concrete algorithms that conform to the type transformation. Database query optimization can be viewed as a search over the reduced space of all canonical forms which are equivalent to t...

A major challenge still facing the designers and implementors of database programming languages (DBPLs) is that of query optimisation. We investigate algebraic query optimisation for DBPLs in the context of a purely declarative functional language that supports sets as first-class objects. Since the language is computationally complete issues such as non-termination of expressions and construction of infinite data structures can be investigated, whilst its declarative nature allows the issue of side effects to be avoided and a rich set of equivalences to be developed. The language has a well-defined semantics which permits us to reason formally about the properties of expressions, such as their equivalence with other expressions and their termination. The support of a set bulk data type enables much prior work on the optimisation of relational languages to be utilised. In the paper we first give the syntax of an archetypal DBPL and briefly discuss its semantics. We then de...