Many parallel algorithms are naturally expressed at a fine level of granularity, often finer than a MIMD parallel system can exploit efficiently. Most builders of parallel systems have looked to either the programmer or a parallelizing compiler to increase the granularity of such algorithms. In this paper we explore a third approach to the granularity problem by analyzing two strategies for combining parallel tasks dynamically at run-time. We reject the simpler load-based inlining method, where tasks are combined based on dynamic load level, in favor of the safer and more robust lazy task creation method, where tasks are created only retroactively as processing resources become available. These strategies grew out of work on Mul-T [15], an efficient parallel implementation of Scheme, but could be used with other languages as well. We describe our Mul-T implementations of lazy task creation for two contrasting machines, and present performance statistics which show the method's effectiveness. Lazy task creation allows efficient execution of naturally expressed algorithms of a substantially finer grain than possible with previous parallel Lisp systems.

When distributed systems first appeared, they were programmed in traditional sequential languages, usually with the addition of a few library procedures for sending and receiving messages. As distributed applications became more commonplace and more sophisticated, this ad hoc approach became less satisfactory. Researchers all over the world began designing new programming languages specifically for implementing distributed applications. These languages and their history, their underlying principles, their design, and their use are the subject of this paper. We begin by giving our view of what a distributed system is, illustrating with examples to avoid confusion on this important and controversial point. We then describe the three main characteristics that distinguish distributed programming languages from traditional sequential languages, namely, how they deal with parallelism, communication, and partial failures. Finally, we discuss 15 representative distributed languages to give the flavor of each. These examples include languages based on message passing, rendezvous, remote procedure call, objects, and atomic transactions, as well as functional languages, logic languages, and distributed data structure languages. The paper concludes with a comprehensive bibliography listing over 200 papers on nearly 100 distributed programming languages.

this article, we describe an execution model for supporting programs that use pointer-based dynamic data structures. This model uses a simple mechanism for migrating a thread of control based on the layout of heap-allocated data and introduces parallelism using a technique based on futures and lazy task creation. We intend to exploit this execution model using compiler analyses and automatic parallelization techniques. We have implemented a prototype system, which we call Olden, that runs on the Intel iPSC/860 and the Thinking Machines CM-5. We discuss our implementation and report on experiments with five benchmarks.

Abstract not found

We describe a parallel programming system for developing machine independent programs for all MIMD machines. Many useful approaches to this problem are seen to require a common base of support, which can be encapsulated in a language that abstracts over resource management decisions and machine specific details. This language can be used for implementing other high level approaches as well as for efficient application programming. The requirements for such a language are defined, and the language supported by the Charm system is described, and illustrated with examples. Charm is one of the first languages to support message driven execution, and embodies unique abstractions such as branch office chares and specifically shared variables. In Part II of this paper, we talk about the runtime support system for Charm. The system thus provides ease of programming on MIMD platforms without sacrificing performance. This research was supported in part by the National Science Foundatio...

Abstract not found

This paper discusses an approach for supporting SPMD (single-program, multiple-data) execution of programs with pointer-based data structures on distributed-memory machines. Through a combination of language design and new compilation techniques, static and dynamic implicit parallelism present in sequential programs based upon pointer-based data structures is exploited. Language support is provided for constructing and manipulating local as well as distributed data structures. The compiler translates the program for execution in SPMD mode in which each processor executes that part of the program which operates on the elements of distributed data structures local to the processor. Therefore the parallelism implicit in a sequential program is exploited. A novel approach for implementing pointers is presented in this paper which is based upon the generation of names for the nodes in a data structure. The name based strategy enables the dynamic distribution of data structures amo...

In a concurrent functional language processes are functions that are executed concurrently. Using special annotations based on lazy copying arbitrary dependencies between these functions can be used to specify arbitrary networks of processes. The communication and synchronization between the processes is realized using the lazy evaluation principle without any additional communication primitves. Communication takes place when a process demands a value that is being calculated by another process. A type system is proposed that enables the programmer to specify higher order functions as process skeletons for frequently occurring process structures. Introduction Functional languages have as advantage that, when a result is obtained, it will always be the same independent of the chosen order of reduction. This property makes functional languages very suited for interleaved and parallel evaluation. As is the case with eager evaluation, one has to be careful since changing the evaluation or...

This paper surveys sequential and parallel implementation techniques for functional programming languages, as well as optimizations that can improve their performance. Sequential implementations have evolved from simple interpreters to sophisticated super-combinator-based compilers, while most parallel implementations have explored a broad range of techniques. We analyze the purpose and function of each implementation technique and discuss the current state-of-the-art in functional language implementation.

Abstract not found