Scheduling in the context of parallel systems is often thought of in terms of assigning tasks in a program to processors, so as to minimize the makespan. This formulation assumes that the processors are dedicated to the program in question. But when the parallel system is shared by a number of users, this is not necessarily the case. In the context of multiprogrammed parallel machines, scheduling refers to the execution of threads from competing programs. This is an operating system issue, involved with resource allocation, not a program development issue. Scheduling schemes for multiprogrammed parallel systems can be classified as one or two leveled. Single-level scheduling combines the allocation of processing power with the decision of which thread will use it. Two level scheduling decouples the two issues: first, processors are allocated to the job, and then the job's threads are scheduled using this pool of processors. The processors of a parallel system can be shared i...

Abstract not found

A majority of the MPP systems designed to date have been MIMD distributed memory systems. For almost all of these systems, message passing environments have provided the primary mechanism for programming multiprocessor applications. In this paper we provide an introduction to MPP systems in general. We then introduce current MPP message passing interfaces, by tracing their historical development over the last 10 years. In addition to their use within a single MPP architecture, we discuss the use of message passing systems to interconnect more loosely coupled processors in heterogeneous environments. Finally we review the development of "portability platforms" - message passing systems that have been devised solely to allow portability of message passing programs between different systems. * Research supported in part by NSF Grand Challenges Applications Group grant ASC-9217394 and by NASA HPCC Group Grant NAG5-2218. + To appear in Parallel Computing, April 1994. TABLE OF CONTENTS ...

Zipcode is a message-passing and process-management system that was designed for multicomputers and homogeneous networks of computers in order to support libraries and large-scale multicomputer software. The system has evolved significantly over the last five years, based on our experiences and identified needs. Features of Zipcode that were originally unique to it, were its simultaneous support of static process groups, communication contexts, and virtual topologies, forming the "mailer" data structure. Point-to-point and collective operations reference the underlying group, and use contexts to avoid mixing up messages. Recently, we have added "gather-send" and "receive-scatter" semantics, based on persistent Zipcode "invoices," both as a means to simplify message passing, and as a means to reveal more potential runtime optimizations. Key features in Zipcode appear in the forthcoming MPI standard. Keywords: Static Process Groups, Contexts, Virtual Topologies, Point-to-Point Communica...

Introduction Explicit parallel programming using the Message Passing Interface (MPI), a de facto standard created by the MPI Forum [15], is quickly becoming the strategy of choice for performance-portable parallel application programming on multicomputers and networks of workstations, so it is inevitably of interest to C++ programmers who use such systems. MPI programming is currently undertaken in C and/or Fortran-77, via the language bindings defined by the MPI Forum [15]. While the committee deferred the job of defining a C++ binding for MPI to MPI-2 [16], it is already possible to develop parallel programs in C++ using MPI, with the added help of one of several support libraries [2, 6, 13]. These systems all strive to enable immediate C++ programming based on MPI. The first such enabling system, MPI++, is the focus of this chapter. MPI++ was an early effort on our part to let us leverage MPI while programming in C++. Here this system is, to a large extent, our vehicle to i

The diverse message passing interfaces provided on parallel and distributed computing systems have caused difficulty in movement of application software from one system to another and have inhibited the commercial development of tools and libraries for these systems. The Message Passing Interface (MPI) Forum has developed a de facto interface standard which was finalised in Q1 of 1994. Major parallel system vendors and software developers were involved in the definition process, and the first implementations of MPI are already appearing. This article presents an overview of the MPI initiative and the standard interface, in particular those aspects which merge demonstrated research with common practice. 1 Introduction The message passing paradigm is the most generally applicable and efficient programming model for parallel machines with distributed memory and has been used widely in parallel and distributed computing systems for some years. The development of parallel computing has bee...

This dissertation is concerned with efficient compilation of...

Parallel software packages which may be of use in scientific and engineering applications of the type carried out on the parallel computing facilities at EPCC and Daresbury Laboratory are surveyed. For each package, a brief description is given along with other useful information such as availability, contact addresses and systems supported. keywords: parallel computing, software packages, scientific applications. This report is available from http://www.dl.ac.uk/TCSC/HPCI/ c fl1996, Daresbury Laboratory. We do not accept any responsibility for loss or damage arising from the use of information contained in any of our reports or in any communication about our tests or investigations. ii CONTENTS iii Contents 1 Introduction 1 1.1 Criteria for inclusion : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1.2 Package areas : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1.3 Individual entries : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :...

The existing methods for distributing data across the processors are not very useful for irregular problems. However, there are some new distribution techniques for this kind of problems, as MRD (Multiple Recursive Decomposition) and BRS (Block Row Scatter), which could be applicated by a data-parallel compiler. In this paper we present a performance comparison of regular (BLOCK and CYCLIC) versus pseudo-regular (BRS) data distribution for sparse matrices on the Cray T3D. We will show results for one of the most useful nonstationary iterative methods for solving symmetric positive definite systems, the Sparse Conjugate Gradient algorithm, using CRAFT with its data-distributions and using two parallelized by-hand codes written in C and Fortran with the BRS sparse distribution. 1 Overview In literature, there are several methods for programming parallel machines: automatic parallelizers (Polaris [6], Parafrase [16]), data-parallel compilers (VFCS [10], Adaptor [8]) and using message pas...

This paper describes a tool for the development of distributed applications. The target execution environment is a network of heterogenous general-purpose workstations. This tool (called HERMES) mainly handles the message transport layer and does not deal with aspects more directly related to parallel computing, such as the topology of communicating processes or load balancing among processors. A coarse-grained concurrence is supported and the resulting parallel architecture is based on the multiple-instructions multiple-data paradigm. This system is mainly characterized by its ability of handling special messages (alarm and urgent) that are guaranteed immediate processing by the receiving process. Messages are always asynchronous, in that they are immediately delivered to destination, without any temporary buffering by the communication system (i.e. asynchronously with respect to the program execution). The system is still under development but a preliminary version, restricted to pro...