For very large databases such as those used by banks and airlines, cost considerations may forbid shutting down the service for a long period of time and reorganizing off-line. Similarly, the size of the database may preclude constructing another copy with the desired organization on another disk collection. Such databases need incremental on-line reorganization. References to records occur in many places in the database. If the identifier used for the record changes due to reorganization, all of these references must be changed. This paper concentrates on the problems of updating references to enable on-line parallel incremental reorganization to be correct while reusing existing code and making minimal changes to underlying transaction processing software. 1 Introduction In many large database installations, down time of only a few minutes can cause a loss of millions of dollars. Yet currently, the only way to reorganize the database is to shut it down and spend hours or days recon...

Parallel database systems are increasingly being deployed to support the performance demands of end-users. While declustering data across multiple nodes facilitates parallelism, existing data placement may no longer be optimal due to skewed workloads and changing access patterns. To prevent performance degradation, the placement of data must be reorganized, and this must be done on-line to minimize disruption to the system. In this paper, we consider a dynamic self-tuning approach to reorganization in a shared nothing system. We introduce a new index-based method that faciliates fast and e#- cient migration of data. Our solution incorporates a globally height-balanced structure and load tracking at di#erent levels of granularity. We conducted an extensive performance study, and implemented the methods on the Fujitsu AP3000 machine. Both the simulation and empirical results demonstrate that our proposed method is indeed scalable and e#ective in correcting any deterioration in system throughput. 1.

Stringent performance requirements in DB applications have led to the use of parallelism for database processing. To allow the database system to take advantage of the performance of parallel shared-nothing systems, the physical DB design must be appropriate for the DB structure and the workload. We develop decision algorithms that will select a good physical DB design both when the DB is first loaded into the system (static decision) and while the DB is being used by the workload (dynamic decision). Our decision algorithms take the database structure, workload, and system characteristics as inputs. The static (or initial) physical DB design decision algorithm involves: • selecting a partitioning attribute for each relation that determines how the relation is fragmented across the nodes (allowing for high I/O bandwidth); • selecting indexes on the relation attributes to allow faster accesses compared to sequential file scans; • selecting the attributes by which to cluster a relation in order to take advantage of the prefetching and caching involved in I/O access; • grouping of relations to allow DB operations (joins) on relation pairs to be executed locally

In this paper, we present an efficient method to do online reorganization of sparsely-populated B + -trees. It reorganizes the leaves first, compacting in short operations groups of leaves with the same parent. After compacting, optionally, the new leaves may swap locations or be moved into empty pages so that they are in key order on the disk. After the leaves are reorganized, the method shrinks the tree by making a copy of the upper part of the tree while leaving the leaves in place. A new concurrency method is introduced so that only a minimum number of pages are locked during reorganization. During leaf reorganization, Forward Recovery is used to save all work already done while maintaining consistency after system crashes. A heuristic algorithm is developed to reduce the number of swaps needed during leaf reorganization, so that better concurrency and easier recovery can be achieved. A detailed description of switching from the old B + -tree to the new B + -tree is describe...

With today’s demands for continuous avail-ability of mission-critical databases, on-line reorganization is a necessity. In this paper we present a new on-Iine reorganization algo-rithm which defers secondary index updates and piggybacks them with user transactions. In addition to the significant reduction of the total I/O cost, the algorithm also assures that almost all the database is available all of the time and that the reorganization is interrupt-ible and restartable. We believe that the tech-nique presented in this paper could be used for improving normal database update perfor-mance as well. 1

In recent years, as more applications start using massive databases as their main source of information, more emphasis is placed on the performance of the database system. These require not only that the database system have good performance, but also that it be continually available. The research in this thesis makes strides in meeting these requirements: dynamically clustering data improves the database performance, and efficient on-line reorganization methods enable the database systems to be continually available. An new algorithm, Enc, for dynamically clustering hierarchical data is presented in this thesis. It uses a primary B + -tree as the main storage structure, all relations in the hierarchy are stored in the B + -tree. The hierarchical relationship is encoded into the keys of the B + -tree. The Enc algorithm maintains good clustering in the presence of insertions and deletions. Experimental results show that using the Enc algorithm, hierarchical queries can be process...

Introduction On-line reorganization is and will be a major problem for transaction systems of the 1990s and the 2000s. Mainframes are rapidly being replaced by workstation farms. Most large software systems must eventually be ported to these new cheaper hardware architectures. The new hardware architectures lend themselves to a client-server software architecture. Presentation services are off-loaded to client workstations. Even servers are no longer on mainframes, but instead on collections of workstations, each perhaps responsible for only a part of the database. Application software will have to be rewritten to conform to the new hardware and software architectures. In addition, some companies may want to take advantage of newer object-oriented software and object-relational systems and new indexing and presentation options. They must then reorganize their data to make the best use of these new software systems. But at the same time, large companies such as airlines and ba