Database management systems will continue to manage large data volumes. Thus, efficient algorithms for accessing and manipulating large sets and sequences will be required to provide acceptable performance. The advent of object-oriented and extensible database systems will not solve this problem. On the contrary, modern data models exacerbate it: In order to manipulate large sets of complex objects as efficiently as today’s database systems manipulate simple records, query processing algorithms and software will become more complex, and a solid understanding of algorithm and architectural issues is essential for the designer of database management software. This survey provides a foundation for the design and implementation of query execution facilities in new database management systems. It describes a wide array of practical query evaluation techniques for both relational and post-relational database systems, including iterative execution of complex query evaluation plans, the duality of sort- and hash-based set matching algorithms, types of parallel query execution and their implementation, and special operators for emerging database application domains.

Emerging database application domains demand not only new functionality but also high performance. To satisfy these two requirements, the Volcano project provides efficient, extensible tools for query and request processing, particularly for object-oriented and scientific database systems. One of these tools is a new optimizer generator. Data model, logical algebra, physical algebra, and optimi-zation rules are translated by the optimizer generalor into optimizer source code. Compared with our earlier EX-ODUS optimizer generator prototype, the search engine is more extensible and powerful; it provides effective support for non-trivial cost models and for physical properties such as sort order. At the same time, it is much more efficient as it combines dynamic programming, which until now had been used only for relational select-project-join optimization, with goal-directed search and branch-and-bound pruning. Compared with other rule-based optimi-zation systems, it provides complete data model indepen-dence and more natural extensibility. 1.

The explosion in complex multi-media content makes it crucial for database systems to support such data efficiently. We make the case that the next generation of object-relational database Data Type (E-ADT) technology, rather than on the “blackbox ” ADTs used in current systems. An E-ADT is an abstract data type that exposes the semantics of its methods. Query optimizations are performed using these semantics, resulting in efficient query processing. The added functionality does not compromise the modularity of data types and the extensibility of the type system. Fundamental architectural changes are required to build such a database system; these have been explored through the implementation of E-ADTs in Predutor. Initial performance results demonstrate an order of magnitude in performance improvements. 1

In this paper we address the problem of controlling the execution of a query optimizer. We describe a control for the optimization process that is based on planning. The controller described here is a goal-directed planner that intermingles planning with the execution of query transformations, and uses execution results to direct further planning of optimizer processing. We describe this control in the context of the Epoq extensible architecture. Epoq is an approach to extensible query optimization that integrates specialized rewrite strategies through its extensible control mechanism. This paper describes our planning-based approach to extensible control and illustrates it with a simple example. 1 Introduction Optimization of a query is inherently a process of searching the space of expressions equivalent to the query. Typically, a given optimizer can only visit some portion of this space, since the set of transformation rules is usually incomplete, the cost of optimization must be ...

Rule-based optimizers and optimizer generators use rules to specify query transformations. Rules act directly on query representations, which typically are based on query algebras. But most algebras complicate rule formulation, and rules over these algebras must often resort to calling to externally defined bodies of code. Code makes rules difficult to formulate, prove correct and reason about, and therefore compromises the effectiveness of rule-based systems. In this paper we present KOLA; a combinator-based algebra designed to simplify rule formulation. KOLA is not a user language, and KOLA's variable-free queries are difficult for humans to read. But KOLA is an effective internal algebra because its combinatorstyle makes queries manipulable and structurally revealing. As a result, rules over KOLA queries are easily expressed without the need for supplemental code. We illustrate this point, first by showing some transformations that despite their simplicity, require head and body rou...

In this thesis we address the problem of providing efficient processing of queries in the extensible environment induced by object-oriented databases. We define a framework for query processing in an object-oriented database and develop designs for major components of this framework. The framework encompasses an object-oriented data model, an algebra to query over that model, transformation rules for the algebra, an internal representation for queries expressed in the algebra, a cost model for analyzing query expressions, and an architecture for an extensible query optimizer. The major contributions of this thesis are an algebra and transformation rules, a representation, and an architecture for extensible query optimization. We show how these components fit into the framework and interact with each other. The EQUAL query algebra presented in this thesis is the first query algebra for object-oriented database systems to be completely consistent with data abstraction, and one of the few...

Query optimizers for persistent object systems should be extensible to react to user-supplied abstract types. Current architectures support only a single, non-extensible technique for controlling the optimization process. We propose an alternative to the current extensible architectures that will support multiple optimizer control strategies and the addition of new control strategies. The optimizer consists of a collection of optimization regions, each of which can transform queries according to a particular control strategy, set of transformations and cost model. A global optimizer control coordinates the movement of a query between these regions. This architecture provides extensibility in the optimizer's repertoire of control strategies through the addition of new regions. In this paper we describe our approach and demonstrate its utility by following the optimizer as it works on an example query. The optimizer will move the query between three distinct regions. The different region...

Rule-based optimizers are extensible because they consist of modifiable sets of rules. For modification to be straightforward, rules must be easily reasonedabout (i.e., understood and verified). At the same time, rules must be expressive and efficient (to fire) for rulebased optimizers to be practical. Production-style rules (as in [15]) are expressed with code and are hard to reason about. Pure rewrite rules (as in [1]) lack code, but cannot atomically express complex transformations (e.g., normalizations). Some systems allow rules to be grouped, but sacrifice efficiency by providing limited control over their firing. Therefore, none of these approaches succeeds in making rules expressive, efficient and understandable. We propose a language (COKO) for expressing an alternative form of input to a rule-based optimizer. A COKO transformation consists of a set of declarative (KOLA) rewrite rules and a (firing) algorithm that specifies their firing. It is straightforward to reason about C...

Abstract not found

Optimization for distributed and parallel database systems is an overly complex problem. This complexity arises due to the various kinds of parallelism and the coexistence of heterogeneous hardware modules. The query optimizers for those systems must exploit parallel algorithms for algebra operators, independent data parallelism and resource scheduling. In addition, unexpected deviations of the workload at runtime enforce new, dynamic execution strategies. This paper reports ongoing work in developing an agentbased query optimization and execution system to cope with this problem. It is based on the principles of distributed, symmetric processes which work in a cooperative, yet independent manner. This approach enables a dynamical adaptation to changes in the execution environment with local and simple actions. We present the architecture of the components and outline the principle of distributed query execution within this software system. Keywords: parallel, distributed DBMS; cooper...