Configuring redundant disk arrays is a black art. To configure an array properly, a system administrator must understand the details of both the array and the workload it will support. Incorrect understanding of either, or changes in the workload over time, can lead to poor performance. We present a solution to this problem: a two-level storage hierarchy implemented inside a single diskarray controller. In the upper level of this hierarchy, two copies of active data are stored to provide full redundancy and excellent performance. In the lower level, RAID 5 parity protection is used to provide excellent storage cost for inactive data, at somewhat lower performance. The technology we describe in this paper, known as HP AutoRAID, automatically and transparently manages migration of data blocks between these two levels as access patterns change. The result is a fully redundant storage system that is extremely easy to use, is suitable for a wide variety of workloads, is largely insensitive to dynamic workload changes, and performs much better than disk arrays with comparable numbers of spindles and much larger amounts of front-end RAM cache. Because the implementation of the HP AutoRAID technology is almost entirely in software, the additional hardware cost for these benefits is very small. We describe the HP AutoRAID technology in detail, provide performance data for an embodiment of it in a storage array, and summarize the results of simulation studies used to choose algorithms implemented in the array.

Dynamic memory allocation has been a fundamental part of most computer systems since roughly 1960, and memory allocation is widely considered to be either a solved problem or an insoluble one. In this survey, we describe a variety of memory allocator designs and point out issues relevant to their design and evaluation. We then chronologically survey most of the literature on allocators between 1961 and 1995. (Scores of papers are discussed, in varying detail, and over 150 references are given.) We argue that allocator designs have been unduly restricted by an emphasis on mechanism, rather than policy, while the latter is more important; higher-level strategic issues are still more important, but have not been given much attention. Most theoretical analyses and empirical allocator evaluations to date have relied on very strong assumptions of randomness and independence, but real program behavior exhibits important regularities that must be exploited if allocators are to perform well in practice.

Many people have observed that computer systems spend much of their time idle, and various schemes have been proposed to use this idle time productively. The commonest approach is to off-load activity from busy periods to less-busy ones in order to improve system responsiveness. In addition, speculative work can be performed in idle periods in the hopes that it will be needed later at times of higher utilization, or non-renewable resource like battery power can be conserved by disabling unused resources. We found opportunities to exploit idle time in our work on storage systems, and after a few attempts to tackle specific instances of it in ad hoc ways, began to investigate general mechanisms that could be applied to this problem. Our results include a taxonomy of idle-time detection algorithms, metrics for evaluating them, and an evaluation of a number of idleness predictors that we generated from our taxonomy. 1. Introduction Resource usage is often bursty: periods of high utilizat...

Abstract Freeblock scheduling is a new approach to utilizing more of a disk's potential media bandwidth. By filling rotational latency periods with useful media transfers, 20-50 % of a never-idle disk's bandwidth can often be provided to background applications with no effect on foreground response times. This paper describes freeblock scheduling and demonstrates its value with simulation studies of two concrete applications: segment cleaning and data mining. Free segment cleaning often allows an LFS file system to maintain its ideal write performance when cleaning overheads would otherwise reduce performance by up to a factor of three. Free data mining can achieve over 47 full disk scans per day on an active transaction processing system, with no effect on its disk performance.

We describe an evolutionary path that allows operating systems to be used in a more flexible and appropriate manner by higher-level services An inf okernel exposes key pieces of inf rmation about its algorithms and internal state; thus, its def ault policies become mechanisms, which can be controlledf rom user-level We have implemented two prototype inf okernels based on the Linux 2 4 and NetBSD 1 5 kernels, called inf Linux and inf BSD, respectively The inf okernels export key abstractions as well as basic inf ormation primitives Using inf oLinux, we have implemented f ur case studies showing that policies within Linux can be manipulated outsideof the kernel Specifically, we show that the def ault file cache replacement algorithm, file layout policy, disk scheduling algorithm, and TCP congestion control algorithm can each be turned into base mechanisms For each case study, we havef ound that inf okernel abstractions can be implemented with little code and that the overhead and accuracyof synthesizing policies at user-level is acceptable Categories a n Subject Descriptors: D.4.7 [Operatin g Systems]: Organ inE in and Desi2 Ge n ral Terms: Desi9 , Experi51 tati1 , Performance Keywords: Poli) , MechaniE) Informatir 1.

This report makes a case for more expressive interfaces between operating systems (OSes) and storage devices. In today's systems, the storage interface consists mainly of simple read and wri t e commands; as a result, OSes operate with little understanding of device-specific characteristics and devices operate with little understanding of system priorities. More expressive interfaces, together with extended versions of today's OS and firmware specializations, would allow the two to cooperate to achieve performance and functionality that neither can achieve alone.

clean construction of disk maintenance applications. They can use it to expose the disk activity to be done, and then process completed requests as they are reported. The system ensures that these applications make steady forward progress without competing for disk access with a system's primary applications. It opportunistically completes maintenance requests by using disk idle time and freeblock scheduling. In this paper, three disk maintenance applications (backup, write-back cache destaging, and disk layout reorganization) are adapted to the system support and evaluated on a FreeBSD implementation. All are shown to successfully execute in busy systems with minimal (e.g., <2%) impact on foreground disk performance. In fact, by modifying FreeBSD's cache to write dirty blocks for free, the average read cache miss response time is decreased by 15--30%. For non-volatile caches, the reduction is almost 50%.

An adaptive UNIX disk device driver is described. To reduce seek times, the driver copies frequentlyreferenced blocks from their original locations to reserved space near the center of the disk. Block reference frequencies need not be known in advance. Instead, they are estimated by monitoring the stream of arriving requests. Measurements show that the adaptive driver reduces seek times and response times substantially. 1 Introduction Disk performance can be a limiting factor in the performance of a computer system. The problem of slow secondary storage devices has come to be known as the I/O bottleneck. One of the major components of disk access times is seek time, the time required to move the drive's read/write heads to the required data. By reducing seek time, the performance of a disk can be improved. In a recent paper [Akyurek 93] we introduced an adaptive block rearrangement technique which reduces disk seek times. Under this technique, a small number of frequently refere...

Permission is granted for noncommercial reproduction of the work for educational or research purposes.

Recent years have seen a meteoric rise in microprocessor performance. Overall computer system performance, however, has not kept pace with that of its most glamorous component. One key bottleneck in system performance is file system I/O [17]. This paper investigates methods for improving file system performance by reducing or hiding disk latency. These methods monitor file accesses and infer relationships about the accessed files. Based on this continuously updated information, the system makes decisions about how to improve file system performance. Two optimization techniques are examined: reorganizing the disk to colocate related files and prefetching files in anticipation of future requests. An algorithm for selecting files to prefetch based on second-order correlations despite the presence of noise is presented. Cluster-based disk reorganization was unable to achieve any benefit over traditional block and cylinder shuffling [21, 24, 23, 1]. Our prefetching systems were able to predict files to prefetch with over 50% accuracy. 1