Dynamic load balancing in multicomputers can improve the utilization of processors and the efficiency of parallel computations through migrating workload across processors at runtime. We present a survey and critique of dynamic load balancing strategies that are iterative: workload migration is carried out through transferring processes across nearest neighbor processors. Iterative strategies have become prominent in recent years because of the increasing popularity of point-to-point interconnection networks for multicomputers. Key words: dynamic load balancing, multicomputers, optimization, queueing theory, scheduling. INTRODUCTION Multicomputers are highly concurrent systems that are composed of many autonomous processors connected by a communication network 1;2 . To improve the utilization of the processors, parallel computations in multicomputers require that processes be distributed to processors in such a way that the computational load is evenly spread among the processors...

. We analyze a class of adaptive integration algorithms on MIMD distributed memory systems. The integration region subdivided in the course of the adaptive process is the N-dimensional cube or simplex. At the subdivision of a subregion, the error behaves according to a prescribed model. The model is supported by the asymptotic behavior of the error for integrands which are continuously differentiable of a given order over all subregions with the possible exception of one subregion containing a vertex singularity of homogeneous type. Extensions are possible to deal with multiple vertex singularities. The analysis can also be applied to problems in other areas, as long as the task selection is based on a priority function which behaves according to a suitable model. Using an efficient management of the subregions, we show that a O(p= log p) speedup can be achieved on a p-processor hypercubic network, such as shuffle exchange, butterfly and hypercube. Furthermore, a speedup of O( p p) c...

We analyze a class of adaptive algorithms for integration over N-dimensional hyperrectangular or simplical regions, on distributed systems. An adaptive algorithm attempts to achieve the requested accuracy by refining the subdivision of the integration region, thus allowing for a concentration of subdivisions near singularities. At the subdivision of a region, the error behaves according to a prescribed model, relating the error of the parent region to that of its children. The analysis can also be applied to problems in other areas, as long as the task selection is based on a priority function which behaves according to a suitable model. Using an efficient management of the subregions, we show that a O(p= log p) speedup can be achieved on a p-processor hypercubic network, such as shuffle exchange, butterfly and hypercube. Furthermore, a speedup of O( p p) can be achieved on a p p \Theta p p mesh network. We also show that our algorithms compare favorably with well-known dynamic...

This paper presents a hydro-dynamic framework to solving the dynamic load balancing problem on a network of heterogeneous computers. In this approach, each processor is viewed as a liquid cylinder where the cross-sectional area corresponds to the capacity of the processor, the communication links are modeled as liquid channels between the cylinders, the workload is represented as liquid, and the load balancing algorithm describes the flow of the liquid. It is proved that all algorithms under this framework converge geometrically to the state of equilibrium, in which the heights of the liquid columns are the same in all the cylinders. In this way, each processor obtains an amount of workload proportional to its capacity. The parameters that affect the convergence rate of the algorithms are also identified and discussed. 1 Introduction It is useful to explore remote computing power in local area networks (LANs) as processors get more and more powerful and the availability of high spee...

In this paper, we present a parallel/ distributed algorithm for multivariate numerical integration and examine its performance on nCUBE-2 and PVM. The test results address the effect of algorithm elements such as a heuristic load balancing technique. We show that good speedups can be achieved for a variety of integration problems, in particular for problems with integrand singularities. This work is a part of a project (ParInt) whose main goal is to package a number of practical multivariate integration algorithms on a variety of multi-processor systems and make them available to researchers and practitioners in various disciplines of science and engineering. Keywords: Multivariate integration, adaptive algorithm, region partitioning, singularities, load balancing 1 Introduction Multivariate integration is an important problem in various areas of science and engineering, for example in finite element applications, statistics, etc. Numerical computation of N-dimensional int...

We give a survey of the ParInt project for Parallel/ distributed Integration. We analyze the "generic" distributed adaptive integration algorithm and modify it to retain an efficient distribution of the global work load over the processors. With this goal we consider a distributed adaptive integration method based on a global heap priority queue, which remains load balanced and assures that the subregions are processed according to their (global) priority (size of error); as well as a local heap priority queue enhanced by receiverinitiated or neighbor load balancing. We report results of these methods. 1 Introduction The objective is to compute an approximation Q to the multivariate integral I = R D f(~x)d~x and an error bound E such that jI \Gamma Qj E ffl = Maxfffla ; ffl r jIjg; for given absolute and relative error tolerances ffl a and ffl r , respectively. The integration domain D may be a set of hyperrectangular or simplicoidal regions. We propose the ParInt integration p...