Improved data management is crucial to the success of current scientific investigations of Global Change. New modes of research, especially the synergistic interactions between observations and model-based simulations, will require massive amounts of diverse data to be stored, organized, accessed, distributed, visualized, and analyzed. Achieving the goals of the U.S. Global Change Research Program will largely depend on more advanced data management systems that will allow scientists to manipulate large-scale data sets and climate system models. Refinements in computing --- specifically involving storage, networking, distributed file systems, extensible distributed data base management, and visualization --- can be applied to a range of Global Change applications through a series of specific investigation scenarios. Computer scientists and environmental researchers at several UC campuses will collaborate to address these challenges. This project complements both NASA's EOS project and ...

The World Wide Web (the Web) has made an enormous amount of data freely accessible over the Internet. However, finding the right information in the midst of this mountain of data has been likened to finding the proverbial needle in a haystack. Commonly used search engines (e.g., AltaVista) and directory services (e.g., Yahoo) have practical but limited success. The exponential growth of the Web is increasing the haystack rapidly. Instead of pull-based browsing, update monitoring is a promising area of research where the system brings the right information to the right user at the right time. In this paper we present the design and implementation of the Conquer continual query system, designed for update monitoring over the Web information sources. A Continual Query (CQ) is a standing query that monitors update of interest using distributed triggers and notifies the user of changes whenever an update of interest reaches specified thresholds or some time limit is reached. In co...

Modern approaches to integrity monitoring suggest to generate triggers from constraints as part of database design and to utilize constraint simplification techniques for trigger optimization. Such proposals, however, have been restricted to static conditions which constrain single states only. In this paper, we show how to derive triggers from dynamic integrity constraints which describe properties of state transitions or state sequences and which can be specified by formulae in temporal logic. Such constraints can equivalently be transformed into transition graphs which describe such life cycles of database objects that are admissible with respect to the constraints: Nodes correspond to situations in life cycles, and edges give the (changing) conditions under which a change into another situation is allowed. If object situations are stored, integrity maintaining triggers can be generated from transition graphs for all situations and all critical database operations. Additionally, new...

CLOCK is a classical cache replacement policy dating back to 1968 that was proposed as a low-complexity approximation to LRU. On every cache hit, the policy LRU needs to move the accessed item to the most recently used position, at which point, to ensure consistency and correctness, it serializes cache hits behind a single global lock. CLOCK eliminates this lock contention, and, hence, can support high concurrency and high throughput environments such as virtual memory (for example, Multics, UNIX, BSD, AIX) and databases (for example, DB2). Unfortunately, CLOCK is still plagued by disadvantages of LRU such as disregard for "frequency", susceptibility to scans, and low performance. As our main contribution, we propose a simple and elegant new algorithm, namely, CLOCK with Adaptive Replacement (CAR), that has several advantages over CLOCK: (i) it is scan-resistant; (ii) it is self-tuning and it adaptively and dynamically captures the "recency" and "frequency" features of a workload; (iii) it uses essentially the same primitives as CLOCK, and, hence, is low-complexity and amenable to a high-concurrency implementation; and (iv) it outperforms CLOCK across a wide-range of cache sizes and workloads. The algorithm CAR is inspired by the Adaptive Replacement Cache (ARC) algorithm, and inherits virtually all advantages of ARC including its high performance, but does not serialize cache hits behind a single global lock. As our second contribution, we introduce another novel algorithm, namely, CAR with Temporal filtering (CART), that has all the advantages of CAR, but, in addition, uses a certain temporal filter to distill pages with long-term utility from those with only short-term utility.

We present a priority system which is particularly suited for production rules coupled to databases. In this system, there are default priorities between all rules and overriding user-defined priorities between particular rules. Rule processing using this system is repeatable: for a given set of rules and priorities, the rules are considered for execution in the same order if the same set of transactions is executed twice on the same initial database state. The rule order adheres to the default order as closely as possible: rules are considered in the same order as the default order unless user-defined precedence constraints force an inversion. We present data structures and efficient algorithms for implementing such a priority system. We show how the data structures can be incrementally maintained as user-defined priorities are altered. We also discuss how the proposed scheme can be extended to build a multi-level hierarchical priority system. 1 Introduction Incorporation of producti...

A lightweight database system (LWDB) is a high-performance, application-specific DBMS. It differs from a general-purpose (heavyweight) DBMS in that it omits one or more features and specializes the implementationof its features to maximizeperformance. Although heavyweight monolithic and extensible DBMSs might be able to emulate LWDB capabilities, they cannot match LWDB performance. In this paper, we describe P2, a generator of lightweight DBMSs, and explain how it was used to reengineer a hand-coded, highly-tuned LWDB used in a production system compiler (LEAPS). We present results that show P2-generated LWDBs reduced the development time and code size of LEAPS by a factor of three and that the generated LWDBs executed substantially faster than versions built by hand or that use an extensible heavyweight DBMS.

DiSTiL is a software generator that implements a declarative domain-specific language (DSL) for container data structures. DiSTiL is a representative of a new approach to domain-specific language implementation. Instead of being the usual one-of-a-kind standalone compiler, DiSTiL is an extension library for the Intentional Programming (IP) transformation system (currently under development by Microsoft Research). DiSTiL relies on several reusable, general-purpose infrastructure tools offered by IP that substantially simplify DSL implementation.

As improvements in processor performance continue to far outpace improvements in storage performance, I /O is increasingly the bottleneck in computer systems, especially in large database systems that manage huge amounts of data. The key to achieving good I /O performance is to thoroughly understand its characteristics. In this article we present a comprehensive analysis of the logical I/O reference behavior of the peak production database workloads from ten of the world’s largest corporations. In particular, we focus on how these workloads respond to different techniques for caching, prefetching, and write buffering. Our findings include several broadly applicable rules of thumb that describe how effective the various I /O optimization techniques are for the production workloads. For instance, our results indicate that the buffer pool miss ratio tends to be related to the ratio of buffer pool size to data size by an inverse square root rule. A similar fourth root rule relates the write miss ratio and the ratio of buffer pool size to data size. In addition, we characterize the reference characteristics of workloads similar to the Transaction Processing Performance Council (TPC) benchmarks C (TPC-C) and D (TPC-D), which are de facto standard performance measures for online transaction processing (OLTP) systems and decision support systems (DSS), respectively. Since benchmarks such as TPC-C and TPC-D can only be

ABSTRACT- In this paper, we address the problem of buffer management in a DBMS when the workload consists of transactions of different priority levels. We present Priority-Hints. a new buffer management algorithm that uses hints provided by the DBMS access methods. The performance of Priority-Hints is compared to that of priority buffer management schemes introduced earlier for a variety of workloads. Our simulation results indicate that Priority-Hints performs consistently better than simple LRU-based algorithms. Furthermore, our algorithm approaches (and in some cases surpasses) the performance of highly sophisticated algorithms that require much more information to be provided to the buffer manager.

Magic sets rewriting is a well-known optimization heuristic for complex decision-support queries. There can be many variants of this rewriting even for a single query, which differ greatly in execution performance. We propose cost-based techniques for selecting an efficient variant from the many choices. Our first contribution is a practical scheme that modelsmagic sets rewriting as a special join method that can be added to any cost-based query optimizer. We derive cost formulas that allow an optimizer to choose the best variant of the rewriting and to decide whether it is beneficial. The order of complexity of the optimization process is preserved by limiting the search space in a reasonable manner. We have implemented this technique in IBM's DB2 C/S V2 database system. Our performance measurements demonstrate that the costbasedmagic optimization technique performs well, and that without it, several poor decisions could be made. Our second contribution is a formal algebraic model of ...