File systems have (at least) two undesirable characteristics: both the addressing model and the consistency semantics differ from those of memory, leading to a change in programming model at the storage boundary. Main memory is a single flat space of pages with a simple durability (persistence) model: all or nothing. File content durability is a complex function of implementation, caching, and timing. Memory is globally consistent. File systems offer no global consistency model. Following a crash recovery, individual files may be lost or damaged, or may be collectively inconsistent even though they are individually sound.

We present evidence that attributes that are known to the file system when a file is created, such as its name, permission mode, and owner, are often strongly related to future properties of the file such as its ultimate size, lifespan, and access pattern. More importantly, we show that we can exploit these relationships to automatically generate predictive models for these properties, and that these predictions are sufficiently accurate to enable optimizations. 1

In this thesis, I show how network storage servers can infer useful information about the requests they are likely to see in the future by analyzing the history of requests they have observed in the past. I also show that this information can be used to improve future decisions about disk block allocation and read-ahead and thereby increase network storage server performance without any change to its clients or the applications running on its clients.

File access traces have been used to drive simulations of storage management algorithms such as file caching, for workload characterization and modeling, and to identify interesting access patterns for performance optimization. Surprisingly they are rarely used to test the correctness and evaluate the performance of an actual file system or server. The main reason is that up until now there did not exist a flexible and easy-to-use player for file access traces. This paper describes the design, implementation, and evaluation of an NFS trace play-back tool called TBBT (Trace-Based file system Benchmarking Tool) that can automatically derive the file system hierarchy from an NFS trace, initialize the file system image with controllable aging effects, and speed up or slow down the trace play-back speed using temporal or spatial scaling without violating dependencies among trace entries. Experiments using a large NFS trace set show that TBBT can indeed produce different throughput and latency measurements than synthetic benchmarks such as SPECsfs. Moreover, TBBT's trace player is actually more efficient than SPECsfs's workload generator despite the fact that the former requires more CPU computation and disk I/O accesses.

When kernel functionality is desired in userspace, the common approach is to reimplement it for userspace interfaces. We show that use of existing kernel file systems in userspace programs is possible without modifying the kernel file system code base. Two different operating modes are explored: 1) a transparent mode, in which the file system is mounted in the typical fashion by using the kernel code as a userspace server, and 2) a standalone mode, in which applications can use a kernel file system as a library. The first mode provides isolation from the trusted computing base and a secure way for mounting untrusted file systems on a monolithic kernel. The second mode is useful for file system utilities and applications, such as populating an image or viewing the contents without requiring host operating system kernel support. Additional uses for both modes include debugging, development and testing. The design and implementation of the Runnable Userspace Meta Program file system (rump fs) framework for NetBSD is presented. Using rump, ten diskbased file systems, a memory file system, a network file system and a userspace framework file system have been tested to be functional. File system performance for an estimated typical workload is found to be ±5 % of kernel performance. The prototype of a similar framework for Linux was also implemented and portability was verified: Linux file systems work on NetBSD and NetBSD file systems work on Linux. Finally, the implementation is shown to be maintainable by examining the 1.5 year period it has been a part of NetBSD. 1

The performance scalability of the Hurricane File System (HFS) is studied under the context of the K42 Operating System. Both systems were designed for scalability on large-scale, shared-memory, non-uniform memory access multiprocessors. However, scalability of HFS was never studied extensively. Microbenchmarks for reading, writing, creating, obtaining file attributes, and name lookup were used to measure scalability. As well, a macrobenchmark in the form of a simulated Web server was used. The unoptimized version of HFS scaled poorly. Optimizations to the meta-data cache in the form of (1) finer grain locks, (2) larger hash tables, (3) modified hash functions, (4) padded hash list headers and cache entries, and (5) a modified block cache free list, resulted in significant scalability improvements.

It is widely believed that the IDE disks found in PCs are inexpensive but slow, whereas the SCSI disks used in servers and workstations are faster, more reliable, and more manageable. The belief that current IDE disks have performance and reliability disadvantages has been called into question by several recent reports. Thus we consider the possibility of achieving tremendous cost advantages by using IDE disks as the foundation of a storage system. In this paper, we give an extensive performance comparison of IDE and SCSI disks. We measure their performance on a variety of micro benchmarks and macro benchmarks, and we explain these results with the help of kernel instrumentation and device activity traces collected by a SCSI analyzer. We consider the impact of several factors, including sequential vs. random workloads, file system enhancements such as journaling and Soft Updates, I/O scheduling in the kernel vs. in the disk drive (as enabled by tagged queuing), and the use of RAID technology to obtain I/O parallelism. In our testbed we find that the IDE disk is faster than the SCSI disk for sequential I/O, but the SCSI disk is faster for random I/O. We also observe that the random I/O performance deficit of the IDE disk is partly overcome by kernel I/O scheduling, and is further mitigated by scheduling in the drive (as enabled by tagged queuing), and by the use of journaling and Soft Updates. Taken as a whole, our results lead us to conclude that RAID systems based on IDE drives can be both faster and significantly less expensive than SCSI RAID systems. 1

Abstract — This paper introduces a new benchmark tool,

Abstract not found

Small file accesses are still limited by disk head movement on modern disk drives with the high disk bandwidth. Small file performance can be improved by grouping and clustering, each of which places multiple files in a directory and places blocks of the same file on disks contiguously. These schemes make it possible for file systems to use large data transfers in accessing small files, reducing disk accesses. However, as file systems become aged, disks become too fragmented to support the grouping and clustering of small files. This fragmentation makes it difficult for file systems to take advantage of large data transfers, increasing disk I/Os. To offer a solution to this problem, we describe a De-fragmented File System (DFS). By using data cached in memory, DFS relocates and clusters data blocks of small fragmented files in a dynamic manner. Besides, DFS clusters related small files in the same directory at contiguous disk locations. Measurements of DFS implementation show that the techniques alleviate file fragmentation significantly and, in particular, performance for small file reads exceeds that of a traditional file system by 78%. 1