We describe a replicating garbage collector for a persistent heap. The garbage collector cooperates with a transaction manager to provide safe and efficient transactional storage management. Clients read and write the heap in primary memory and can commit or abort their write operations. When write operations are committed they are preserved in stable storage and survive system failures. Clients can freely access the heap during garbage collection because the collector concurrently builds a compact replica of the heap. A log captures client write operations and is used to support both the transaction manager and the replicating garbage collector. Our implementation is the first to provide concurrent and compacting garbage collection of a persistent heap. Measurements show that concurrent replicating collection produces significantly shorter pause times than stop-and-copy collection. For small transactions, throughput is limited by the logging bandwidth of the underlying log manager. The results suggest that replicating garbage collection offers a flexible and efficient way to provide automatic storage management in transaction systems, object-oriented databases and persistent programming environments.

The automatic reclamation of storage for unreferenced objects is very important in object databases. Existing language system algorithms for automatic storage reclamation have been shown to be inappropriate. In this paper, we investigate methods to improve the performance of algorithms for automatic storage reclamation of object databases. These algorithms are based on a technique called partitioned garbage collection, in which a subset of the entire database is collected independently of the rest. Specifically, we investigate the policy that is used to select what partition in the database should be collected. The policies that we propose and investigate are based on the intuition that the values of overwritten pointers provide good hints about where to find garbage. Using trace-driven simulation, we show that one of our policies requires less I/O to collect more garbage than any existing implementable policy and performs close to a near-optimal policy over a wide range of database s...

In this paper we develop and evaluate a number of storage reclamation algorithms for client-server persistent object stores. Experience with a detailed simulation and a prototype implementation in the Exodus storage manager shows that one of our proposed algorithms, the Incremental Partitioned Collector, is complete, maintains transaction semantics, and can be run incrementally and concurrently with client applications. Furthermore, it can significantly improve subsequent system performance by reclustering data, rendering it attractive even for systems that choose not to support automatic storage reclamation. 1 Introduction Experience with object-oriented programming languages has demonstrated that explicit storage management by programmers is a difficult and error prone task --- anyone who has spent time trying to find a storage leak in a non-trivial C++ program can attest to that fact. Fortunately, in C++ programs the effect of a storage leak is limited to individual runs of progra...

The principal tenet of the persistence model is that it abstracts over all the physical properties of data such as how long it is stored, where it is stored, how it is stored, what form it is kept in and who is using it. Experience with programming systems which support orthogonal persistence has shown that the simpler semantics and reduced complexity can often lead to a significant reduction in software production costs. Persistent systems are relatively new and it is not yet clear which of the many models of concurrency and distribution best suit the persistence paradigm. Previous work in this area has tended to build one chosen model into the system which may then only be applicable to a particular set of problems. This thesis challenges the orthodoxy by designing a persistent framework in which all models of concurrency and distribution can be integrated in an add-on fashion. The provision of such a framework is complicated by a tension between the conceptual ideas of persistence...

This paper describes a new garbage collection scheme for large persistent object stores that makes efficient use of the disk and main memory. The heap is divided into partitions that are collected independently using information about inter-partition references. We present efficient techniques to maintain this information stably using auxiliary data structures in memory and the log. The result is a scheme that truly preserves the localized and scalable nature of partitioned collection. Remembering

We describe an eficient server-based algorithm for garbage collecting object-oriented databases in a client/server environment. The algorithm is incremen-tal and runs concurrently with client transactions. Un-like previous algorithms, it does not hold any locks on data and does not require callbacks to clients. It is fault tolerant, but performs very little logging. The algorithm has been designed to be integrated into existing OODB systems, and therefore it works with standard implemen-tation techniques such as two-phase locking and write-ahead-logging. In addition, it supports client-server per-formance optimizations such as client caching and flexible management of client buffers. We describe an implemen-tation of the algorithm in the EXODUS storage manager and present results from an initial performance study. 1

We describe an efficient server-based algorithm for garbage collecting object-oriented databases in a workstation/server environment. The algorithm is incremental and runs concurrently with client transactions, however, it does not hold any locks on data and does not require callbacks to clients. It is fault tolerant, but performs very little logging. The algorithm has been designed to be integrated into existing OODB systems, and therefore it works with standard implementation techniques such as two-phase locking and write-ahead-logging. In addition, it supports client-server performance optimizations such as client caching and flexible management of client buffers. We describe an implementation of the algorithm in the EXODUS storage manager and present results from an initial performance study of the implementation. These results demonstrate that the introduction of the garbage collector adds minimal overhead to client operations. 1 Introduction A primary strength of Object Oriented...

Persistent, user-defined objects present an attractive abstraction for working with non-volatile program state. However, the slow speed of persistent storage (i.e., disk) has restricted their design and limited their performance. Fast, byte-addressable, non-volatile technologies, such as phase change memory, will remove this constraint and allow programmers to build high-performance, persistent data structures in non-volatile storage that is almost as fast as DRAM. Creating these data structures requires a system that is lightweight enough to expose the performance of the underlying memories but also ensures safety in the presence of application and system failures by avoiding familiar bugs such as dangling pointers, multiple free()s, and locking errors. In addition, the system must prevent new types of hard-to-find pointer safety bugs that only arise with persistent objects. These bugs are especially dangerous since any corruption they cause will be permanent. We have implemented a lightweight, high-performance persistent object system called NV-heaps that provides transactional semantics while preventing these errors and providing a model for persistence that is easy to use and reason about. We implement search trees, hash tables, sparse graphs, and arrays using NV-heaps, BerkeleyDB, and Stasis. Our results show that NV-heap performance scales with thread count and that data structures implemented using NV-heaps out-perform BerkeleyDB and Stasis implementations by 32 × and 244×, respectively, by avoiding the operating system and minimizing other software overheads. We also quantify the cost of enforcing the safety guarantees that NV-heaps provide and measure the costs of NV-heap primitive operations.

Traditional database management systems perform updates-in-place and use logs and periodic checkpointing to efficiently achieve atomicity and durability. In this Thesis we shall present a different method, Shades, for achieving atomicity and durability using a copy-on-write policy instead of updates-in-place. We shall also present index structures and the implementation of Shines, a persistent functional programming language, built on top of Shades. Shades includes real-time generational garbage collection. Real-timeness is achieved by collecting only a small part, a generation, of the database at a time. Contrary to previously presented persistent garbage collection algorithms, Shades has no need to maintain metadata (remembered sets) of intra-generation pointers on disk since the metadata can be reconstructed during recovery. This considerably reduces the amount of disk writing. In conjunction with aggressive commit grouping, efficient index structures, a design specialized to a main memory environment, and a carefully crafted implementation of Shines, we have achieved surprisingly high performance, handsomely beating commercial database management systems.

Over the past ten years much research effort has been expended in attempting to build systems which support orthogonal persistence. Such systems allow all data to persist for an arbitrary length of time, possibly longer than the creating program, and support access and manipulation of data in a uniform manner, regardless of how long it persists. Persistent systems are usually based on a persistent store which provides storage for objects. Most existing persistent systems have been developed above conventional architectures and/or operating systems. In this paper we argue that conventional architectures provide an inappropriate base for persistent object systems and that we must look towards new architectures if we are to achieve acceptable performance. The examples given are based on the Monads architecture which provides explicit hardware support for persistence and objects. 1. INTRODUCTION Over the past ten years much research effort has been expended in attempting to build systems w...