We propose and evaluate the concept of a semantically-smart disk system (SDS). As opposed to a traditional "smart" disk, an SDS has detailed knowledge of how the file system above is using the disk system, including information about the on-disk data structures of the file system. An SDS exploits this knowledge to transparently improve performance or enhance functionality beneath a standard block read/write interface. To automatically acquire this knowledge, we introduce a tool (EOF) that can discover file-system structure for certain types of file systems, and then show how an SDS can exploit this knowledge on-line to understand file-system behavior. We quantify the space and time overheads that are common in an SDS, showing that they are not excessive. We then study the issues surrounding SDS construction by designing and implementing a number of prototypes as case studies; each case study exploits knowledge of some aspect of the file system to implement powerful functionality beneath the standard SCSI interface. Overall, we find that a surprising amount of functionality can be embedded within an SDS, hinting at a future where disk manufacturers can compete on enhanced functionality and not simply cost-per-byte and performance.

Low disk throughput is one of the main impediments to improving the performance of data-intensive servers. In this paper, we propose two management techniques for the disk controller cache that can significantly increase disk throughput. The first technique, called File-Oriented Read-ahead (FOR), adjusts the number of read-ahead blocks brought into the disk controller cache according to file system information. The second technique, called Host-guided Device Caching (HDC), gives the host control over part of the disk controller cache. As an example use of this mechanism, we keep the blocks that cause the most misses in the host buffer cache permanently cached in the disk controller. Our detailed simulations of real server workloads show that FOR and HDC can increase disk throughput by up to 34 % and 24%, respectively, in comparison to conventional disk controller cache management techniques. When combined, the techniques can increase throughput by up to 47%.

This paper presents MVSS, a storage system for active storage devices. MVSS oers a single framework for supporting various services at the device level. It provides a exible interface for associating services to a le through multiple views of the le. Similar to views of a database in a multi-view database system, views in MVSS are generated dynamically and are not stored on physical storage devices. MVSS represents each view of an underlying le through a separate entry in the le system namespace. MVSS separates the deployment of services from le system implementations and thus allows services to be migrated to the storage devices. The paper presents the design of MVSS and how dierent services can be supported in MVSS at the device level. To illustrate our approach, we implemented a prototype system on PCs running Linux. We present results from the prototype implementation to demonstrate the eectiveness of our approach. 1

The advent of networking technologies and high performance transport protocols facilitates the service of storage over networks. However, they pose challenges in integration and interaction among storage server application components and system components. In this paper, we put forward a component, called Unifier, to provide more efficient integration and better interaction among these components. Unifier has three notable features. (1) Unifier integrates cache management and communication buffer management. It offers a single copy data sharing among all components in a server application safely and concurrently. (2) It reduces memory registration and deregistration costs to enable applications to take full advantage of RDMA operations. (3) It provides means to achieve adaptation, application-specific optimization, and better cooperation among different components. This paper presents the design and implementation of Unifier. This component has been deployed and evaluated in a version of PVFS1 implementation over InfiniBand. Experimental results show performance improvements between 30 % and 70 % over other approaches. Better scalability is also achieved by the PVFS I/O servers.

Solid-state devices (SSDs) have the potential to replace traditional hard disk drives (HDDs) as the de facto storage medium. Unfortunately, there are several decades of spinning-media assumptions embedded in the software stack as an “unwritten contract ” [20]. In this paper, we revisit these system-level assumptions in light of SSDs and find that several of them are invalidated by SSDs, breaking the unwritten contract and resulting in poor performance and lifetime. The underlying cause is the incorrect division of labor between file systems and storage. Block management must be removed from the file system and delegated to the SSD to prevent further accumulation of storage-specific assumptions. We find that object-based storage is an appropriate way to achieve this. 1

In the present paper, we examine the problem of supporting application-specific computation within a network storage server. Our objectives are i) to introduce an easy to use yet powerful architecture for executing both customdeveloped and legacy applications close to the stored data, ii) to investigate the potential performance improvement in I/O-intensive processing, iii) to exploit the I/O-tra#c information available within the file server for more effective resource management. One main di#erence from previous active storage research is our emphasis on the expressive power and usability of the storage server interface. We describe an extensible active storage framework that we built in order to demonstrate the feasibility of the proposed system design. Our conclusions are substantiated through experimentation with a popular multi-layer map warehouse application.

Emerging video surveillance, environmental monitoring applications, and constantly evolving large scientific setups require large, high-performance, and reliable storage systems with guaranteed real-time data access. These systems are often implemented using redundant arrays of independent disks (RAID). In this paper we investigate the effectiveness of preemptive disk-scheduling algorithms to achieve better quality of service (QoS) in RAID systems. We present an architecture for QoS-aware RAID systems that use Semi-preemptible IO for servicing internal disk IOs. We show when and how to preempt IOs to improve the overall performance of the RAID system. We evaluate the benefits and estimate the overhead of our approach using a preemptible RAID simulator that we have implemented.

Attached please find our submission to the ACM Transactions on Storage Systems. Our paper is titled “End-to-End Abstractions for Application-Aware Storage. ” In this article, we provide an overview of the problem of “information-gap ” in the storage stack, and present two novel abstractions that effectively bridge this gap, thereby enabling a range of functionality that is almost impossible to achieve with existing systems and interfaces. Most of the material in this article forms part of Gopalan Sivathanu’s Ph.D. dissertation. Our first abstraction is Type-Aware Storage that aims communicating pointer information to the disk hardware. We have published this abstraction in OSDI 2006. This article includes a new unpublished case-study of type-aware storage,“Disk-Level Data Consistency. ” This case-study proposes and evaluates how complex higher-level consistency properties can be achieved at the disk hardware-level, in a file-system–agnostic manner. Our second abstraction is Context-Aware I/O, a flexible mechanism to communicate between applications and data, across the storage stack. We present the design, implementation, and evaluation of the above abstraction, and demonstrate its usefulness through two separate case-studies. This abstraction and its case-studies have not been published in any other venue. Overall, of this 60 page article, about 60 percent is new unpublished material. This work was completely done when all authors were affiliated with the File systems and

Since large-scale and data-intensive applications have been widely deployed, there is a growing demand for high-performance storage systems to support data-intensive applications. Compared with traditional storage systems, next-generation systems will embrace dedicated processor to reduce computational load of host machines and will have hybrid combinations of different storage devices. We present a new architecture of active storage system, which leverage the computational power of the dedicated processor, and show how it utilizes the multi-core processor and offloads the computation from the host machine. We then solve the challenge of applying the active storage node to cooperate with the other nodes in the cluster environment by design a pipeline-parallel processing pattern and report the effectiveness of the mechanism. In order to evaluate the design, an open-source bioinformatics application is extended based on the pipeline-parallel mechanism. We also explore the hybrid configuration of storage devices within the active storage. The advent of flash-memory-based solid state disk has become a critical role in revolutionizing the storage world. However, instead of simply replacing the traditional magnetic harddisk with thesolid state disk, researchers believe that finding a complementary approach to corporate both of