As Data Grids become more commonplace, large data sets are being replicated and distributed to multiple sites, leading to the problem of determining which replica can be accessed most efficiently. The answer to this question can depend on many factors, including physical characteristics of the resources and the load behavior on the CPUs, networks, and storage devices that are part of the end-to-end path linking possible sources and sinks.

Abstract—RPS is a publicly available toolkit that allows a practitioner to straightforwardly create flexible online and offline resource prediction systems in which resources are represented by independent, periodically sampled, scalar-valued measurement streams. The systems predict the future values of such streams from past values and are composed at runtime out of a large and extensible set of communicating components that are in turn constructed using RPS’s extensible sensor, prediction, wavelet, and communication libraries. This paper describes the design, implementation, and performance of RPS. We have used RPS extensively to evaluate predictive models and build online prediction systems for host load, Windows performance data, and network bandwidth. The computation and communication overheads involved in such systems are quite low. Index Terms—Distributed systems, performance of systems. æ 1

In heterogeneous and dynamic environments, efficient execution of parallel computations can require mappings of tasks to processors whose performance is both irregular (because of heterogeneity) and time-varying (because of dynamicity). While adaptive domain decomposition techniques have been used to address heterogeneous resource capabilities, temporal variations in those capabilities have seldom been considered. We propose a conservative scheduling policy that uses information about expected future variance in resource capabilities to produce more efficient data mapping decisions. We first present techniques, based on time series predictors that we developed in previous work, for predicting CPU load at some future time point, average CPU load for some future time interval, and variation of CPU load over some future time interval. We then present a family of stochastic scheduling algorithms that exploit such predictions of future availability and variability when making data mapping decisions. Finally, we describe experiments in which we apply our techniques to an astrophysics application. The results of these experiments demonstrate that conservative scheduling can produce execution times that are both significantly faster and less variable than other techniques. 1

We describe and evaluate the Running Time Advisor (RTA), a system that can predict the running time of a compute-bound task on a typical shared, unreserved commodity host. The prediction is computed from linear time series predictions of host load and takes the form of a confidence interval that neatly expresses the error associated with the measurement and prediction processes--- error that must be captured to make statistically valid decisions based on the predictions. Adaptive applications make such decisions in pursuit of consistent high performance, choosing, for example, the host where a task is most likely to meet its deadline. We begin by describing the system and summarizing the results of our previously published work on host load prediction. We then describe our algorithm for computing predictions of running time from host load predictions. Finally, we evaluate the system using over 100,000 randomized testcases run on 39 different hosts.

Remos provides resource information to distributed applications. Its design goals of scalability, flexibility, and portability are achieved through an architecture that allows components to be positioned across the network, each collecting informationabout its local network. To collect information from different types of networks and from hosts on those networks, Remos provides several collectors that use different technologies, such as SNMP or benchmarking. By matching the appropriate collector to each particular network environment and by providing an architecture for distributing the output of these collectors across all querying environments, Remos collects appropriately detailed information at each site and distributes this information where needed in a scalable manner. Prediction services are integrated at the user-level, allowing history-based data collected across the network to be used to generate the predictions needed by a particular user. Remos has been implemented and tested in a variety of networks and is in use in a number of different environments.

{vazhkuda,

Remos provides resource information to distributed applications. Its design goals of scalability, flexibility, and portability are achieved through an architecture that allows components to be positioned across the network, each collecting information about its local network. To collect information from differenttypes of networks, Remos provides several Collectors that use differenttechnologies, including SNMP and benchmarking. By matching the Collector to the particular network environmentandbyproviding an architecture for distributing the output of these collectors across all querying environments, Remos collects appropriately detailed information at each site and distributes this information where needed in a scalable manner. Remos has been implemented and tested in a variety of networks and is in use in a number of differentenvironments.

The real-time scheduling advisor (RTSA) is an entirely userlevel system that an application running on a typical shared, unreserved distributed computing environment can turn to for advice on how to schedule its compute-bound soft real-time tasks. Given a list of hosts, a description of the CPU demands of the task, the deadline, and a confidence level, the RTSA will recommend one of the hosts and predict, as a confidence interval, the running time of the task on that host. The RTSA is based on a scalable and extensible shared resource prediction system based on statistical time series analysis. In this paper, we first describe how the RTSA builds on this underlying system to provide its service, and then we evaluate its performance using a randomized methodology based on real background workloads, determining the effect of different factors. We also compare it with a random approach and a measurement-based approach.

We describe and evaluate WatchTower, a set of library routines that simplifies the collection of performance data for the monitoring of Windows NT/2000. WatchTower has an overhead similar to that of existing tools but is more easily embedded into other applications. More important, we show how data reduction techniques can be used to diminish the volume of performance data gathered to only that which is useful; while still capturing the overall behavior of the computer.

abstract Cost-effectively collecting distributed state information is a challenging problem. It perhaps has no single perfect answer since different distributed application environments pose different requirements from the information collection process. In any case, knowledge of the environment, in terms of traffic models, load models etc. play a key role in determining tradeoffs between accuracy (needed to ensure QoS requirements) and costeffectiveness. In this paper, we develop an adaptive information collection algorithm that utilizes network traffic knowledge characterized using a time series model. The algorithm utilizes an information collection architecture consisting of a directory service integrated into the middleware layer with monitoring modules distributed across the network. The cost-effectiveness of the proposed information collection algorithm proposed is verified in simulations over diverse network traffic patterns, i.e. Internet WAN (TCP), MPEG(multimedia) and web access traffic traces. Our results show that the proposed adaptive information collection algorithm compensates for inaccuracies in network traffic predictions in a cost-effective manner. 1