File systems and I/O subsystems should be smart ; they can analyze how they are being used and tune themselves dynamically to improve their performance. File systems should select caching and disk placement strategies on a per-file basis, and they should use system-wide disk reorganization strategies. For example, systems should be able to reorganize the data on disk automatically during idle periods so that system performance is improved during future periods of peak load. This dissertation presents the design and analysis of iPcress, a prototype of a nextgeneration file system. iPcress is a smart, high-performance, reliable file system. It uses statistical information collected on a per-file basis to tune itself. iPcress has a framework in which various optimizations can be performed by the file system automatically. It is extensible; other optimization techniques can be incorporated easily, so that the system may evolve. In addition, iPcress can incorporate a variety of file access...

n the replay log that records update activity made by a client while disconnected from all servers. Third, operation logging is used in resolution logs on servers to allow transparent resolution of directory updates made to partitioned server replicas. 1. RVM RVM, an acronym for recoverable virtual memory, is a Unix library that supports local, non-nested transactions on data structures mapped into a process' virtual memory [7]. A unique aspect of RVM is that it allows independent control over the basic transactional properties of atomicity, permanence, and serializability. Atomicity and permanence of transactions are obtained using a NO-UNDO/REDO value log. The log can be a raw disk partition or a Unix file. If a Unix file is used, true transactional guarantees are achieved only if the fsync system call blocks until dirty file buffer cache data has been written to disk. There are two log operations: flush and truncate. As transactions are committed, new-value records of virtual mem

Computer forensic analysis, intrusion detection and disaster recovery are all dependent on the existence of trustworthy log files. Current storage systems for such log files are generally prone to modification attacks, especially by an intruder who wishes to wipe out the trail he leaves during a successful breakin. In light of recent advances in storage capacity and sharp drop in prices of storage devices, as well as the demand for trustworthy storage systems, it is timely to design and develop fast storage systems that practically have no limit in capacity and admit “secure append-only ” operations (namely data can only be appended to a storage device; once appended it can no longer be modified, and can be read out by authorized users only.) This paper discusses an approach to building secure append-only storage systems. It proposes a possible secure append-only storage architecture that could be used to detect and prevent deletion or modification by inside/outside attackers. A specific implementation of the architecture based on block device drivers and magnetic storage firmwares is also presented. 1

This paper presents a performance comparison of several file system allocation policies. The file systems are designed to provide high bandwidth between disks and main memory by taking advantage of parallelism in an underlying disk array, catering to large units of transfer, and minimizing the bandwidth dedicated to the transfer of meta data. All of the file systems described use a multiblock allocation strategy which allows both large and small files to be allocated efficiently. Simulation results show that these multiblock policies result in systems that are able to utilize a large percentage of the underlying disk bandwidth; more than 90% in sequential cases. As general purpose systems are called upon to support more data intensive applications such as databases and supercomputing, these policies offer an opportunity to provide superior performance to a larger class of users. 1. Introduction Most current file systems can be divided into two distinct categories: fixed block systems ...

The Amadeus[1] platform stores objects in groups called clusters. A cluster-based storage mechanism is appropriate for supporting the implementation of persistent programming languages since object access in such languages tend to be very localised. However, this is not the case for OODB's where object access patterns cannot be predicted. Thus the motivation for this thesis is to provide a storage system for Amadeus which is flexible enough to fulfill the conflicting storage requirements of persistent programming languages and of object-oriented database systems (OODB). In order to provide support for OODB-type applications running in the Amadeus environment, we need some means of mapping the data of a specific object within a cluster into virtual memory without incurring the overhead of mapping in other objects which also reside in the same cluster. Furthermore, in order to provide for the implementation of high-performance OODB applications, the cost of accessing a specified object o...

Abet ract An algorithm is presented for the consistent recovery of replicated data in a client-server system. The algorithm is based on logging and is similar to the optimistic techniques that are well known in the literature. However, unlike in existing optimistic techniques, explicit dependency information is. not maintained. Instead, dependency in-formation is estimated from the ordering of messages found in servers’ logs. These dependency estimates can, in general, be expensive to compute. It is therefore shown how inexpensive estimates can be ap-plied when a system is well structured. 1

Logged virtual memory (LVM) provides a log of writes to one or more specified regions of the virtual address space. Logging is useful for applications that require rollback and/or persistence such as parallel simulations and memory-mapped object-oriented databases. It can also be used for output, debuggingand distributed consistency maintenance. This paper describes logged virtual memory as an extension of the standard virtual memory system software and hardware, our prototype implementation, and some performance measurements from this prototype. Based on these measurements and the experience with our prototype, we argue that logged virtual memory canbe supported with modest extensions to standard virtual memory systems, provides significant benefit to applications and servers, and is faster than other log-generation techniques. 1 Introduction Logged virtual memory (LVM) is a virtual memory system extension that provides logs of write activity to specified virtual memory regions. Ea...

Advances in computer system technology in the areas of CPUs, disk subsystems, and volatile RAM memory are combining to create performance problems existing file systems are ill-equipped to solve. This paper identifies the problems of using the existing UNIX file systems on 1990's technology and presents an alternative file system design that can use disks an order-of-magnitude more efficiently for typical UNIX workloads. The design, named LFS for log-structured file system, treats the disk as a segmented append-only log. This allows LFS to write many small changes to disk in a single large I/O while still maintaining the fast file reads of existing file systems. In addition, the logstructured approach allows near instantaneous file system crash recovery without coupling CPU and disk performance with synchronous disk writes. This paper describes and justifies the major data structures and algorithms of the LFS design. We compare an implementation of LFS in the Sprite distributed operati...

: In most modern local area network environments, NFS is used to provide remote file storage on a particular server machine. A consequence of this distributed architecture is that the failure of the server results in paralysis or a loss of work for users. This paper presents the design of a low-cost fault tolerant NFS server which can be installed on most Unix networking environments. FT-NFS runs as a user process and does not necessitate any underlying specific operating system functionality. The originality of our approach relies on the use of a stable cache which provides data availability and resiliency to a single failure. The main benefits of the stable cache are first to allow disk write operations to be safely performed in the background and second to permit the gathering of small files in large containers. The latter technique permits disk I/Os to be improved by reducing their number and increasing their length. Under the nhfsstone benchmark, FT-NFS outperforms the in-kernel S...

Sawmill: A Logging File System for a High-Performance Kenneth William Shirriff Doctor of Philosophy in Computer Science University of California at Berkeley Professor John Ousterhout, Chair The widening disparity between processor speeds and disk performance is causing an increasing I/O performance gap. One method of increasing disk bandwidth is through arrays of multiple disks (RAIDs). In addition, to prevent the file server from limiting disk performance, new controller architectures connect the disks directly to the network so that data movement bypasses the file server. These developments raise two questions for file systems: how to get the best performance from a RAID, and how to use such a controller architecture.