This paper aims at improving the performance of parallel applications running on nondedicated distributed platforms through a dynamic load balancer which is kept hidden to the programmer. The support periodically checks the status of the platform and, if necessary, redistributes portions of the data domain from overloaded to underloaded nodes. Various experimental results pointed out the dependence of the performance on the frequency of checking the load status of the platform. Since at implementation time the user has not enough information to choose the best frequency of activation, we propose a performance model that is able to autonomously select at runtime a checkload interval close to the optimum. This model works for Single Program Multiple Data (SPMD) regular computations. The analytical values have been validated through a comparison with experimental results obtained on a cluster of nondedicated workstations. The experiments in various scenarios demonstrate that the model is ...

Load balancing on a multi-processor system involves redistributing tasks among processors so that each processor has roughly the same amount of work to perform. The token-distribution problem is a static variant of the load balancing problem for the case in which the workloads in the system cannot be divided arbitrarily; that is, where each token represents an atomic element of work. A scalable method for distributing tokens over a parallel architecture is the so-called dimension-exchange approach. Our results include improved analysis of two existing dimension-exchange algorithms for token distribution on arbitrary graphs and on arbitrary trees, respectively. In particular, we establish a logarithmic upper bound on the discrepancy of the resulting distribution when the second algorithm is applied to an arbitrary initial distribution on a tree. We then present a new dimension-exchange algorithm for token distribution on trees, which assuming each node knows the number of nodes in the tree, determines a ‘perfectly balanced’ distribution. Furthermore, the rate of convergence is worst-case optimal for trees of bounded degree. Note that an algorithm for token-distribution on trees is applicable to arbitrary architectures, since the algorithm can be applied on a spanning tree of any given connected graph.

Optimizing the performance of dynamic load balancing toolkits and applications requires the adjustment of several runtime parameters; however, determining sufficiently good values for these parameters through repeated experimentation can be an expensive and prohibitive process. We describe an analytic modeling method which allows developers to study and optimize adaptive application performance in the presence of dynamic load balancing. To aid tractibility, we first derive a “bi-modal ” step function which simplifies and approximates task execution behavior. This allows for the creation of an analytic modeling function which captures the dynamic behavior of adaptive and asynchronous applications, enabling accurate predictions of runtime performance. We validate our technique using synthetic microbenchmarks and a parallel mesh generation application and demonstrate that this technique, when used in conjunction with the PREMA runtime toolkit, can offer users significant performance improvements over several well-known load balancing tools used in practice today.