Configuring redundant disk arrays is a black art. To configure an array properly, a system administrator must understand the details of both the array and the workload it will support. Incorrect understanding of either, or changes in the workload over time, can lead to poor performance. We present a solution to this problem: a two-level storage hierarchy implemented inside a single diskarray controller. In the upper level of this hierarchy, two copies of active data are stored to provide full redundancy and excellent performance. In the lower level, RAID 5 parity protection is used to provide excellent storage cost for inactive data, at somewhat lower performance. The technology we describe in this paper, known as HP AutoRAID, automatically and transparently manages migration of data blocks between these two levels as access patterns change. The result is a fully redundant storage system that is extremely easy to use, is suitable for a wide variety of workloads, is largely insensitive to dynamic workload changes, and performs much better than disk arrays with comparable numbers of spindles and much larger amounts of front-end RAM cache. Because the implementation of the HP AutoRAID technology is almost entirely in software, the additional hardware cost for these benefits is very small. We describe the HP AutoRAID technology in detail, provide performance data for an embodiment of it in a storage array, and summarize the results of simulation studies used to choose algorithms implemented in the array.

In this thesis, we present a new methodology for resource sharing algorithms in distributed systems. We propose that a distributed computing system should be composed of a decentralized community of microeconomic agents. We show that this approach decreases complexity and can substantially improve performance. We compare the performance, generality and complexity of our algorithms with non-economic algorithms. To validate the usefulness of our approach, we present economies that solve three distinct resource management problems encountered in large, distributed systems. The first economy performs CPU load balancing and demonstrates how our approach limits complexity and effectively allocates resources when compared to non-economic algorithms. We show that the economy achieves better performance than a representative non-economic algorithm. The load balancing economy spa...

Abstract--An important design issue in distributed data processing systems is to determine optimal data distribution. The problem requires a tradeoff between time and cost. For instance, quick response time conflicts with low cost. The paper addresses the data distribution problem in this conflicting environment. A formulation of the problem as a non-linear program is developed. An algorithm employing a simple search procedure is presented, which gives an optimal data distribution. An example is solved to illustrate the method.

Q uantitative M odeling I n P arallel S ystems

In computer information systems, some programs are used more frequently than others producing skewed distributions of program usages. We investigate the claim that static views of program usage frequencies are insufficient when they are used for storage allocation decisions making it necessary to study the implications of the use of dynamic frequencies in storage allocation. The use of dynamic frequencies provides a natural extension to previously presented static cost model literature for hierarchical storage allocation. In our work, we present the value of incorporating dynamic usage frequencies into program usage cost models. Thus, an optimization-based cost modeling methodology using Simon's Model for dynamic hierarchical storage allocation is presented. To illustrate, a simple example of program storage allocation is presented in both static and dynamic form. Cost-saving comparisons are then discussed.

There is now considerable interest in industry and academia for “autonomic” or self-adaptive networks and distributed systems. Large-scale applications, such as simulation, typically run on large distributed systems in which it is impossible to guarantee at each instant the full reliability and availability of all processing nodes and of all communication links. Nevertheless such systems have to accomplish their mission and provide a high level of dependability and the best possible performance to critical applications. Self-adaptation will therefore be an intrinsic feature of large distributed systems and networks in order to move tasks and files in response to changes in subsystem dependability and system load, and to provide Quality-of-Service and dependable connections in the presence of fluctuating workloads and unknown system behavior. In this paper we review our work on the design of adaptive on-line task management algorithms for distributed systems, and the control of cognitive packet networks (CPN) to offer user specified QoS.