This paper presents a new technique for disk storage management called a log-structured file system. A logstructured file system writes all modifications to disk sequentially in a log-like structure, thereby speeding up both file writing and crash recovery. The log is the only structure on disk; it contains indexing information so that files can be read back from the log efficiently. In order to maintain large free areas on disk for fast writing, we divide the log into segments and use a segment cleaner to compress the live information from heavily fragmented segments. We present a series of simulations that demonstrate the efficiency of a simple cleaning policy based on cost and benefit. We have implemented a prototype logstructured file system called Sprite LFS; it outperforms current Unix file systems by an order of magnitude for small-file writes while matching or exceeding Unix performance for reads and large writes. Even when the overhead for cleaning is included, Sprite LFS can use 70 % of the disk bandwidth for writing, whereas Unix file systems typically can use only 5-10%. 1.

Abstract-Memory resident database systems (MMDB’s) store their data in main physical memory and provide very high-speed access. Conventional database systems are optimized for the particular characteristics of disk storage mechanisms. Memory resident systems, on the other hand, use different optimizations to structure and organize data, as well as to make it reliable. This paper surveys the major memory residence optimizations and briefly discusses some of the memory resident systems that have been designed or implemented. Index Terms- Access methods, application programming in-terface, commit processing, concurrency control, data clustering, data representation, main memory database system (MMDB), query processing, recovery. Invited Paper I.

This paper presents Slice, a new storage system architecture for highspeed LANs incorporating network-attached block storage. Slice interposes a request switching filter -- called a /proxy -- along the network path between the client and the network storage system (e.g., in a network adapter or switch). The purpose of the/proxy is to route requests among a server ensemble that implements the file service. We present a prototype that uses this approach to virtualize the standard NFS file protocol to provide scalable, high-bandwidth file service to ordinary NFS clients. The paper presents and justifies the architecture, proposes and evaluates several request routing policies realizable within the architecture, and explores the effects of these policies on service structure

Recovery activities, like logging, checkpointing and restart, are used to restore a database to a consistent state after a system crash has occurred. Recovery related overhead is particularly troublesome in a mainmemory database where I/O activities are performed for the sole purpose of ensuring data durability. In this paper we present a recovery technique for main-memory databases, whose benefits are as follows. First, disk I/O is reduced by logging to disk only redo records during normal execution. The undo log is normally resident only in main memory, and is garbage collected after transaction commit. Second, our technique reduces lock contention on account of the checkpointer by allowing action consistent checkpointing --- to do so, the checkpointer writes to disk relevant parts of the undo log. Third, the recovery algorithm makes only a single pass over the log. Fourth, our technique does not require the availability of any special hardware such as non-volatile RAM. Thus our reco...

As part of the Starburst extensible database project developed at the IBM Almaden Research Center, we designed and implemented a memory-resident storage component that co-exists with Starburst's diskoriented storage component. The two storage components share the same common services, such as query optimization, transaction management, etc. However, the memory-resident storage component is faster than the disk-oriented storage component and hence needs faster run-time services. This paper examines two runtime services, the lock manager and the latch mechanism, and investigates possible cost-cutting measures. We propose the use of of a single latch for protecting a table, all of its indexes, and all of its related lock information, in order to reduce storage component latch costs. We then show that although a table-level latch is a large granule latch, it does not significantly restrict concurrency. We also examine traditional lock manager design and suggest a different design that is a...

In traditional database management systems, recovery activities, like checkpointing and restart, are performed in a quiescent state where no transactions are active. This approach impairs the performance of on-line transaction processing systems. Recovery related overhead is particularly troublesome in an environment where a large volatile memory is used. The appealing alternative is to perform recovery activities incrementally and in parallel with transaction execution. An incremental scheme for recovery in main memory database systems is presented in this paper. We propose a page-based incremental restart algorithm that enables the resumption of transaction processing as soon as the system is up. Pages are recovered individually and according to the demands of the post-crash transactions. In addition, an incremental method for propagating updates from main memory to the backup database on disk is also provided. Here the emphasis is on decoupling the I/O activities related to the prop...

We propose a technique for maintaining coherency of a transactional distributed shared memory, used by applications accessing a shared persistent store. Our goal is to improve support for fine-grained distributed data sharing in collaborative design applications, such as CAD systems and software development environments. In contrast, traditional research in distributed shared memory has focused on supporting parallel programs; in this paper, we show how distributed programs can benefit from this shared-memory abstraction as well. Our approach, called log-based coherency, integrates coherency support with a standard mechanism for ensuring recoverability of persistent data. In our system, transaction logs are the basis of both recoverability and coherency. We have prototyped log-based coherency as a set of extensions to RVM [Satyanarayanan et al. 94], a runtime package supporting recoverable virtual memory. Our prototype adds coherency support to RVM in a simple way that does not requir...

ion Dal's architecture, illustrated in Figure 2, is organized in multiple layers of abstraction to support the toolkit approach discussed earlier. At the highest level, users can interact with Dal's relational manager. Below that level is what we call the heap-file/indexing layer, which provides support for fixed-length and variable-length collections, as well as template-based Locks Logs System database Database file 1 Database file 2 Database file N Process 1 user code Dal library Virtual memory of process 1 Virtual memory of process 2 System physical memory Process 2 user code Dal library Log of updates to database Checkpoint images of database Figure 1. Architecture of the Dal system. Bell LabsTechnical Journal uWinter 1997 39 indexing abstractions. In general, at this level, a user does not need to interact with individual locks or latches. (A latchis a short-term lock implemented by a high-speed mutual exclusion mechanism.) Instead, the user specifi...

Abstract not found

A real-time database system (RTDBS) can be defined as a database system where transactions are associated with real-time constraints typically in the form of deadlines. The system must process transactions so as to both meet the deadlines and maintain the data consistency. Previous research effort in this field has been focused on scheduling transactions with soft or firm deadlines under the conventional transaction model and database system architecture which cannot support predictable real#time transaction processing. In this thesis, we provide a framework to realize predictable real-time transaction processing, satisfying both timing and consistency constraints of a real-time database system. First, we classify data objects and transactions found in typical real-time database applications, considering their distinct characteristics and requirements. Each type of real-time data objects has its own correctness criteria, different from the conventional one. Real#time transactions a...