Scheduling large task graphs is an important issue in parallel computing. In this paper we tackle the two following problems : (1) how to schedule a task graph, when it is too large to t into memory? (2) How to build a generic program such that parameter values of a task graph can be given at run-time? Our answers feature the parameterized task graph (PTG), which is a symbolic representation of the task graph. We propose a dynamic scheduling algorithm which takes a PTG as an entry and allows to generate a generic program. We present a theoritical study which shows that our algorithm nds good schedules for coarse grain task graphs, has a very low memory cost and a low computational complexity. When the average number of operations of each task is large enough, we prove that the scheduling overhead is negligible with respect to the makespan. We also provide experimental results that demonstrate the feasibility of our approach using several compute-intensive kernels found in numerical s...

This paper studies the benefits of compiling dataparallel languages onto a multithreaded runtime environment providing dynamic thread migration facilities. Each abstract process is mapped onto a thread, so that dynamic load balancing can be achieved by migrating threads among the processing nodes. We describe and evaluate an implementation of this idea in the Adaptor HPF compiler. We show that no deep modification of the compiler are needed, and that the overhead of managing threads can be kept small. As an experimental validation, we report on an HPF implementation of the Gauss Partial Pivoting algorithm. We show that using an initial BLOCK data distribution with our dynamic load balancing scheme can reach the performance of the optimal CYCLIC distribution. Introduction Data-parallel languages are now recognized as major tools for high performance computing. Considerable effort has been put in designing sophisticated methods to compile them efficiently onto a variety of architectures...

We introduce Madeleine, a communication interface specifically designed to support distributed, multithreaded applications in both a portable and efficient way. Thanks to its new API, Madeleine can implement RPC operations without any extra copy with respect to the underlying protocol. Madeleine can thus achieve very good performance on a variety of protocols (MPI, TCP, BIP, VIA, etc.) and networks (Fast-Ethernet, Myrinet, SCI, etc.). Madeleine serves as a building block for the multithread programming environment PM 2. We provide detailed experimental results for each representative implementation and discuss a number of

Introduction Distributed multithreading is currently a very popular technique to support the execution of parallel applications on MIMD machines. This is mainly due to the recent emergence of distributed multithreaded environments [14], [17], [9] and to their availability on a wide range of architectures. The distributed multithreaded approach has already been successfuly applied to several application fields, such as Combinatorial Optimization [10], Data Parallel Compiling [29], Large Simulations [5] and Molecular Dynamics [2]. For portability reasons, multithreaded environments are often implemented on top of high-level standard communication interfaces such as PVM or MPI. A large subset of these environments are RPC-based because the functionalities provided rely implicitly or explicitly on a mechanism able to invoque remote services (RSR in Nexus [14], RPC in PM 2 [9], Fork/Join in DTS [6]). This document is organized as follows: we first present the rese

In this paper, we show the thread concept can bring much more than a simple "overlap of communications by computations". With an efficient programming model that allows dynamic thread migration, threads play the role of "mobile virtual processes" whose mapping may change at runtime. The programmer can focus on the parallel decomposition of his algorithm while the execution support will take care of keeping the work fairly distributed during execution. 1. Introduction : the ESPACE project The goal of the Espace project is to provide an environment to both facilitate the development of massively parallel irregular applications and allow their efficient execution on distributed architectures. To achieve the first point, we think it is very important that applications are written regardless of the underlying architecture topology, and are expressed in terms of parallel decomposition rather than in terms of cooperating processes. From the programmer

This paper focuses on the use of distributed memory multithreaded environments in data parallel programs and has two main goals. The first is to show that data parallel programs can support features like communication overlapping, load balancing without global data parallel object redistribution and the efficient use of clusters of uniprocessor and/or symmetric multiprocessors (SMPs). Our extended model introduces virtual processes. Virtual processes are implemented with mobile threads. The second goal is to determine the interactions between data parallel programs and a model of distributed memory multithreaded environments, with respect to intra-node communications and especially to thread migration. This paper also discuss this multithreaded environment with respect to the different models of threads and shows that the HPF and the C* data parallel compilation models easily integrate the proposed model.