The performance and capacity of commodity computer systems have improved drastically in recent years. However, these systems still lack the support for real-time data access, which is required by an increasing number of emerging applications. In this paper we first present several important storagebound real-time applications and classify their Quality of Service (QoS) requirements. We then survey the representative work on disk management in the areas of IO scheduling, admission control, and data placement. Finally, we present our approach for providing disk QoS in commodity systems and present key empirical results from the micro-benchmark-based evaluation of our QoS-enhanced Linux kernel.

Block-wise access to data is a central theme in the design of efficient external memory (EM) algorithms. A second important issue, when more than one disk is present, is fully parallel disk I/O. In this paper we present a deterministic simulation technique which transforms parallel algorithms into (parallel) external memory algorithms. Specifically, we present a deterministic simulation technique which transforms Coarse Grained Multicomputer (CGM) algorithms into external memory algorithms for the Parallel Disk Model. Our technique optimizes block-wise data access and parallel disk I/O and, at the same time, utilizes multiple processors connected via a communication network or shared memory. We obtain new improved parallel external memory algorithms for a large number of problems including sorting, permutation, matrix transpose, several geometric and GIS problems including 3D convex hulls (2D Voronoi diagrams), and various graph problems. All of the (parallel) external memory algorithms obtained via simulation are analyzed with respect to the computation time, communication time and the number of I/O’s. Our results answer to the challenge posed by the ACM working group on storage I/O for largescale computing [8]. 1

Blockwise access to data is a central theme in the design of efficient external memory (EM) algorithms. A second important issue, when more than one disk is present, is fully parallel disk I/O. In this paper we present a simple, deterministic simulation technique which transforms certain Bulk Synchronous Parallel (BSP) algorithms into efficient parallel EM algorithms. It optimizes blockwise data access and parallel disk I/O and, at the same time, utilizes multiple processors connected via a communication network or shared memory. We obtain new improved parallel EM algorithms for a large number of problems including sorting, permutation, matrix transpose, several geometric and GIS problems including three-dimensional convex hulls (two-dimensional Voronoi diagrams), and various graph problems. We show that certain parallel algorithms known for the BSP model can be used to obtain EM algorithms that meet well known I/O complexity lower bounds for various problems, including sorting.

In this study we present a scheme called two-dimensional BubbleUp (2DB) for managing parallel disks in a multimedia server. Its goal is to reduce initial latency for interactive multimedia applications, while balancing disk loads to maintain high throughput. The 2DB scheme consists of a data placement and a request scheduling policy. The data placement policy replicates frequently accessed data and places them cyclically throughout the disks. The request scheduling policy attempts to maintain free "service slots" in the immediate future. These slots can then be used to quickly service newly arrived requests. Through examples and simulation, we show that our scheme significantly reduces initial latency and maintains throughput comparable to that of the traditional schemes. Keywords: multimedia, data replication, initial latency, disk array. 1 Introduction Media servers are designed to provide large numbers of presentations in the form of audios, movies or news clips. These servers need...

State-of-the-art networked storage servers are equipped with increasingly powerful computing capability and large DRAM memory as storage caches. However, their contribution to the performance improvement of networked storage system has become increasingly limited. This is because the client-side memory sizes are also increasing, which reduces capacity misses in the client buffer caches as well as access locality in the storage servers, thus weakening the caching effectiveness of server storage caches. Proactive caching in storage servers is highly desirable to reduce cold misses in clients. We propose an effective way to improve the utilization of storage server resources through prefetching in storage servers for clients. In particular, our design well utilizes two unique strengths of networked storage servers which are not leveraged in existing storage server prefetching schemes. First, powerful storage servers have idle CPU cycles, under-utilized disk bandwidth, and abundant memory space, providing many opportunities for aggressive disk data prefetching. Second, the servers have the knowledge about high-latency operations in storage devices, such as disk head positioning, which enables efficient disk data prefetching based on an accurate cost-benefit analysis of prefetch operations. We present STEP – a Sequentiality and Thrashing dEtection based Prefetching scheme, and its implementation with Linux Kernel 2.6.16. Our performance evaluation by replaying Storage Performance Council (SPC)’s OLTP traces shows that server performance improvements are up to 94% with an average of 25%. Improvements with frequently used Unix applications are up to 53 % with an average of 12%. Our experiments also show that STEP has little effect on workloads with random access patterns, such as SPC ’ Web-Search traces. 1

One is scientific computing with massive datasets, such as those found in seismic processing, climate modeling, and so forth [dC94]. The second is databases [DG92]. The I/O bottleneck continues to be a serious concern for scientific computing, particularly Grand Challenge problems, where it is now commonly recognized as an obstacle. Many scientific applications generate 1 GB of I/O per run [dC94], and applications performing an order of magnitude more are not uncommon: applications in computational physics and fluid dynamics are projected to require I/O on the order of 1 TB [dC94]. It seems clear that these total I/O requirements will keep increasing as scientists continue to study phenomena at larger space and time scales, and at finer space and time resolutions. Since the response time that humans can tolerate for obtaining computational results--- no matter how comprehensive and detailed--- is always bounded, the I/O rates required will continue to increase also. Thus while curre

Input/Output is a big obstacle to effective use of teraflopsscale computing systems. Motivated by earlier parallel I/O measurements on an Intel TFLOPS machine, we conduct studies to determine the sensitivity of parallel I/O performance on multi-programmed mesh-connected machines with respect to number of I/O nodes, number of compute nodes, network link bandwidth, I/O node bandwidth, spatial layout of jobs, and read or write demands of applications. Our extensive simulations and analytical modeling yield important insights into the limitations on parallel I/O performance due to network contention, and into the possible gains in parallel I/O performance that can be achieved by tuning the spatial layout of jobs. Applying these results, we devise a new processor allocation strategy that is sensitive to parallel I/O traffic and the resulting network contention. In performance evaluations driven by synthetic workloads and by a real workload trace captured at the San Diego Supercomputing Cen...

A multimedia storage system plays a vital role for the performance and scalability of multimedia servers. To handle the server load imposed by increased user access to on-demand multimedia streaming applications, new storage system solutions are needed. Internet use and the amount of data that users download and stream from Internet servers are rapidly increasing. Analysts predict that by 2005, the World Wide Web and applications such as news- and video-on-demand (VoD) will constitute approximately 50 percent of the total available data stored. 1 Although mid-priced PCs can handle the load that multimedia applications impose on the client system, the potentially high number of concurrent users downloading or streaming data from media-on-demand servers is a challenge for a multimedia server's storage system. Multimedia storage systems store and retrieve data from storage devices and manage related issues including data placement, scheduling, file management, continuous data delivery, memory buffering, and prefetching. For high-data-rate multimedia systems, storage systems have long been viewed as a primary bottleneck for two reasons. First, multimedia applications

Lately, on-demand streaming multimedia applications have become very popular. Contemporary personal computers can handle the load imposed by such multimedia applications on the client side, but the potentially high number of concurrent users accessing a server represents a generic problem. The multimedia storage system is responsible for storage and retrieval of multimedia data from storage devices, and plays a vital role for the performance and scalability of multimedia servers. It deals with issues related to data placement, scheduling, file management, continuous data delivery, buffer management, prefetching, etc., and with the particular demands of multimedia applications, such as real-time characteristics, large file sizes, high data rates, and several data sources. Performing these tasks and supporting these requirements appropriately are burdened by an increasing speed mismatch between processors and the most prolific and affordable storage devices, -- magnetic disks --, and by the introduction of new requirements in new multimedia scenarios.

Abstract Storage-area networks are a popular and efficient way of building large storage systems both in an enterprise environment and for multi-domain storage service providers. In both environments the network and the storage has to be configured to ensure that the data is maintained securely and can be delivered efficiently. In this paper we describe a model of mandatory security for multi-domain storage services that is flexible enough to reflect the data requirements, tractable for the administrator, and implementable as part of an automatic configuration system. We describe the model abstractly, its implementation as part of a prototype SAN configuration system written in OPL, and illustrate its operation on a set of sample configurations.