The underutilization of disk parallelism and file cache buffers by traditional file systems induces I/O stall time that degrades the performance of modern microprocessor-based systems. In this paper, we present aggressive mechanisms that tailor file system resource management to the needs of I/O-intensive applications. In particular, we show how to use application-disclosed access patterns (hints) to expose and exploit I/O parallelism and to allocate dynamically file buffers among three competing demands: prefetching hinted blocks, caching hinted blocks for reuse, and caching recently used data for unhinted accesses. Our approach estimates the impact of alternative buffer allocations on application execution time and applies a cost-benefit analysis to allocate buffers where they will have the greatest impact. We implemented informed prefetching and caching in DEC’s OSF/1 operating system and measured its performance on a 150 MHz Alpha equipped with 15 disks running a range of applications including text search, 3D scientific visualization, relational database queries, speech recognition, and computational chemistry. Informed prefetching reduces the execution time of the first four of these applications by 20 % to 87%. Informed caching reduces the execution time of the fifth application by up to 30%.

We analyzed the UNIX 4.2 BSD file system by recording userlevel activity in trace files and writing programs to analyze the traces. The tracer did not record individual read and write operations, yet still provided tight bounds on what information was accessed and when. The trace analysis shows that the average file system bandwidth needed per user is low (a few hundred bytes per second). Most of the files accessed are open only a short time and are accessed sequentially. Most new information is deleted or overwritten within a few minutes of its creation. We also wrote a simulator that uses the traces to predict the performance of caches for disk blocks. The moderate-sized caches used in UNIX reduce disk traffic for file blocks by about 50%, but larger caches (several megabytes) can eliminate 90% or more of all disk traffic. With those large caches, large block sizes (16 kbytes or more) result in the fewest disk accesses. Trace-Driven Analysis of 4.2 BSD File System January 2, 1993 1...

This paper describes the architecture of eNVy, a large non-volatile main memory storage system built primarily with Flash memory. eNVy presents its storage space as a linear, memory mapped array rather than as an emulated disk in order to provide an efficient and easy to use software interface. Flash memories...

Prefetching has been shown to be an effective technique for reducing user perceived latency in distributed systems. In this paper we show that even when prefetching adds no extra traffic to the network, it can have serious negative performance effects. Straightforward approaches to prefetching increase the burstiness of individual sources, leading to increased average queue sizes in network switches. However, we also show that applications can avoid the undesirable queueing e ects of prefetching. In fact, we show that applications employing prefetching can signi cantly improve network performance, to a level much better than that obtained without any prefetching at all. This is because prefetching offers increased opportunities for traffic shaping that are not available in the absence of prefetching. Using a simple transport rate control mechanism, a prefetching application can modify its behavior from a distinctly ON/OFF entity to one whose data transfer rate changes less abruptly, while still delivering all data in advance of the user's actual requests.

This paper reports our research results that improve second level buer cache performance. Several previous studies have shown that a good single level cache replacement algorithm such as LRU does not work well with second level buer caches. Second level buer caches have dierent access pattern from rst level buer caches because Accesses to second level buer caches are actually misses from rst level buer caches.

Informed prefetching and caching based on application disclosure of future I/O accesses (hints) can dramatically reduce the execution time of I/O-intensive applications. A recent study showed that, in the context of a single hinting application, prefetching and caching algorithms should adapt to the dynamic load on the disks to obtain the best performance. In this paper, we show how to incorporate adaptivity to disk load into the TIP2 system, which uses cost-benefit analysis to allocate global resources among multiple processes. We compare the resulting system, which we call TIPTOE (TIP with Temporal Overload Estimators) to Cao et al's LRU-SP allocation scheme, also modified to include adaptive prefetching. Using disk-accurate trace-driven simulation we show that, averaged over eleven experiments involving pairs of hinting applications, and with data striped over one to ten disks, TIPTOE delivers 7% lower execution time than LRU-SP. Where the computation and I/O demands of each experi...

File system I/O is increasingly becoming a performance bottleneck in large distributed computer systems. lais is due to the increased file I/O demands of new applications, the inability of any single storage structure to respond to these demands, and the slow decline of disk access times (latency and seek) relative to the rapid increase in CPU speeds, memory size, and network bandwidth.

v Abstract Caching has long been recognized as a powerful performance enhancement technique in many areas of computer design. Most modern computer systems include a hardware cache between the processor and main memory, and many operating systems include a software cache between the file system routines and the disk hardware. In a distributed file system, where the file systems of several client machines are separated from the server backing store by a communications network, it is desirable to have a cache of recently used file blocks at the client, to avoid some of the communications overhead. In this configuration, special care must be taken to maintain consistency between the client caches, as some disk blocks may be in use by more than one client. For this reason, most current distributed file systems do not provide a cache at the client machine. Those systems that do place restrictions on the types of file blocks that may be shared, or require extra communication to confirm that...

In current mainframe architectures, extended memory is used as a pageaddressable main memory extension to improve I/O performance. If made non-volatile, extended memory can significantly increase transaction processing performance. In this paper, we study how locally distributed transaction systems may use shared extended memory to support high transaction rates and short response times. We outline how a specific store called GEM (Global Extended Memory) can be utilized to improve I/O performance and inter-system communication and cooperation. It turns out that distributed transaction systems of the “shared disk ” type (data sharing systems) can benefit most from such a shared store. We describe simple yet efficient schemes using GEM for global concurrency control and constructing a global log file for such systems. 1

A distributed file systetn provides file service from one or rnore &red file servers to a community of clierit workstations o’ver Q network. Wlaile the client-server paradigm has many advantages, it also presents new challenges to system designers concerning perfor-rnance and reliability. As both client workstations and file servers become increasingly well-resourced, a nu~n-ber of system design decisions need to be re-examined. This research concerus the caching of disk blocks in a distributed client-server enviromnent. Some recent re-search has suggested that various strategies for cache rnauagernent may not be equally suited to the circurn-stances at both the client and the server. Since any caching strategy is based on assumptions concerning the characteristics of the denland, the performance of the strategy is only as good as the accuracy of this assurnp-tion. The perfornlance of a caching strategy at a file server is strongly influenced by the presence of client cnches since these caches alter the characteristics of the stream of requests that reaches the server. This paper presents the results of an investigation of the effect of client caching on the nature of the server workload as a step towards understanding the performnnce of caching strategies at the server. The results demonstrate that client caches alter worklond characteristics in a way that will have CI profound impact on server cnche per-forrnance, and suggest worthwhile directions for future development of server caching strategies. 1