Efficient data storage, a major concern in the modern computer industry, is mostly provided today by traditional magnetic disks. However, the cost of a disk transfer (measured in processor cycles) continues to increase with time, making disk accesses increasingly expensive.

We consider the use of cooperative caching to manage the memories of cluster-based servers. Over the last several years, a number of researchers have proposed locality-conscious servers that implement content-aware request distribution to address this problem During this development, it has become conventional wisdom that cooperative caching cannot match the performance of these servers . Unfortunately, while locality-conscious servers provide very high performance, their request distribution algorithms are typically bound to specific applications. The advantage of building distributed servers on top of a block-based cooperative caching layer is the generality of such a layer; it can be used as a building block for diverse services, ranging from file systems to web servers. In this paper, we reexamine the question of whether a server built on top of a generic block-based cooperative caching algorithm can perform competitively with locality-conscious servers. Specifically, we compare the performance of a cooperative caching-based web server against L2S, a highly optimized locality-conscious server. Our results show that by modifying the replacement algorithm of traditional cooperative caching algorithms, we can achieve much of the performance provided by locality-conscious servers. Our modification increases network communication to reduce disk accesses, a reasonable trade-off considering the current trend of relative performance between LANs and disks.

If we examine the structure of the applications that run on parallel machines, we observe that their I/O needs increase tremendously every day. These applications work with very large data sets which, in most cases, do not fit in memory and have to be kept in the disk. The input and output data files are also very large and have to be accessed very fast. These large applications also want to be able to checkpoint themselves without wasting too much time. These facts constantly increase the expectations placed on parallel and distributed file systems. Thus, these file systems have to improve their performance to avoid becoming the bottleneck in parallel/distributed environments. On the other hand, while the performance of the new processors, interconnection networks and memory increases very rapidly, no such thing happens with the disk performance. This lack of improvement is due to the mechanical parts used to build the disks. These components are slow and limit both the latency and t...

. Dimemas is a simulator that allows the study of message passing applications on distributed memory machines. Currently, we are using Dimemas to analyze the effects of different processor scheduling policies when several parallel applications share machine resources as processors, interconnection network,... The effect of sequential independent processes on the parallel applications is also being studied with the simulator. This situation is fairly frequent on clusters of workstations running a mixture of parallel and sequential workloads. We also study the influence of communication parameters (network bandwidth and conflicts) in the system performance. The paper presents the structure of the simulator and the workload used. This workload is a mixture of jobs from the NAS parallel benchmarks. We finally compare the effects of the above mentioned factors on both global system throughput and individual response time on each of the different parallel applications. Key words. processor ...

Clustering provides a viable approach to building scalable Web systems with increased computing power and abundant storage space for data and contents. We present a pure-Java-based parallel Web server with load balancing features for a heterogeneous cluster environment. Our design introduces a special in-memory cache layer, called the global object space (GOS), which is an integration of specially set-aside physical memory of individual participating cluster nodes. The GOS provides a unified view of cluster-wide memory resources and allows transparent accesses to cached objects independent of where they are located physically. The core of the GOS is a mechanism that performs dynamic in-memory caching of frequently requested Web objects. Using a technique known as cooperative caching, a requested Web object can be fetched from a node’s local cache or a peer server’s local cache, with the disk serving only as the last resort. A prototype system based on the W3C’s Jigsaw server has been implemented on a 16-node PC cluster. Three cluster-aware cache replacement algorithms were tested and evaluated. The benchmark results show good speedups with a real-life access log, proving that cooperative caching can have significant positive impacts on the performance of cluster-based parallel Web servers.

Typical large-scale scientific applications periodically write checkpoint files to save the computational state throughout execution. Existing parallel file systems improve such write-only I/O patterns through the use of client-side file caching and write-behind strategies. In distributed environments where files are rarely accessed by more than one client concurrently, file caching has achieved significant success; however, in parallel applications where multiple clients manipulate a shared file, cache coherence control can serialize I/O. We have designed a thread based caching layer for the MPI I/O library, which adds a portable caching system closer to user applications so more information about the application’s I/O patterns is available for better coherence control. We demonstrate the impact of our caching solution on parallel write performance with a comprehensive evaluation that includes a set of widely used I/O benchmarks and production application I/O kernels. 1.

In this paper, we examine some of the important problems observed in the design of cooperative caches. Solutions to the coherence, load-balancing and fault-tolerance problems are presented. These solutions have been implemented as a part of PAFS, a parallel/distributed file system, and its performance has been compared to the one achieved by xFS. Using the comparison results, we have observed that the proposed ideas not only solve the main problems of cooperative caches, but also increase the overall system performance. Although the solutions presented in this paper were targeted to a parallel machine, reasonable good results have also been obtained for networks of workstations. 1 Introduction There is a general trend to use inter-node cooperation for improving performance of parallel and distributed file systems. Cooperative caching is a good example of this concept [9]. In a cooperative cache, all nodes work together to build a global cache. This cooperation increases the cache size...

Caching is a technique intensively used in memory and traditional file systems to improve system performance. However, the use of caching in parallel file systems has been limited to I/O nodes to avoid cache coherence problems. In this paper we present the cache policies implemented in ParFiSys, a parallel file system developed at the UPM. ParFiSys exploits the use of cache both at computing nodes and I/O nodes, with aggressive prefetching and delayed-write techniques. The cache coherence problem is solved using a dynamic scheme of cache coherence protocols with different sizes and shapes of granularity. Some performance results, obtained on an IBM SP2, are presented to demonstrate advantages offered by the cache management used in ParFiSys. Resumen El concepto de cache ha sido ampliamente utilizado en memoria y sistemas de ficheros tradicionales como un mecanismo para mejorar el rendimiento del sistema. Sin embargo, el uso de cache en sistemas de ficheros paralelos se ha limitado exc...

Although transactions have been a valuable abstraction of atomicity, persistency, and recoverability, they have not been widely used in programming environments today, mostly because of their high overheads that have been driven by the low performance of magnetic disks. A major challenge in transaction-based systems is to remove the magnetic disk from the critical path of transaction management. In this paper we present PERSEAS , a transaction library for main memory databases that decouples the performance of transactions from the magnetic disk speed. Our system is based on a layer of reliable main memory that provides fast and recoverable storage of data. We have implemented our system as a user-level library on top of the Windows NT operating system in a network of workstations connected with the SCI interconnection network. Our experimental results suggest that PERSEAS achieves performance that is orders of magnitude better than traditional recoverable main memory systems. 1 Intro...

Clusters of SMPs are attractive for executing shared memory parallel applications but reconciling high performance and ease of programming remains an open issue. A possible approach is to provide an efficient Single System Image (SSI) operating system giving the illusion of an SMP machine. But the global management of all resources to provide a true SSI remains an open issue. In this paper, we introduce the concept of container as a mechanism to unify global resource management at the lowest operating system level. Higher level operating system services such as virtual memory system and file cache can be easily implemented based on containers and transparently take benefit of the whole memory resource available in the cluster.