Fine-grained programming is important to parallel computing because it covers up the un- avoidable latency occuring in large systems. Dataflow computing, where small groups of instructions can be sequentialized into threads, is an extreme of the fine-grained or multi- threading approaches. Originally, dataflow programs were run on specialized hardware. The research described in this document unites advances in compiling dataflow programs to run on non-specialized hardware with the architectural advances of the J-Machine, a general-purpose massively-parallel computer.

In a non-strict language, functions may return values before their arguments are available, and data structures may be defined before all their components are defined. Compiling such languages to conventional hardware is not straightforward; instructions do not have a fixed compile time ordering. Such an ordering is necessary to execute programs efficiently on current microprocessors. Partitioning is the process of compiling a non-strict program into threads (i.e., a sequence of instructions). This process involves detecting data dependencies at compile time and using these dependencies to "sequentialize" parts of the program. Previous work on partitioning did not propagate dependence information across recursive procedure boundaries. Using a representation known as Paths we are able to represent dependence information of recursive functions. Also, we incorporate them into a known partitioning algorithm. However, this algorithm fails to make use of all the information contained in pat...

Early advocates of dataflow computers predicted that the characteristics of the dataflow model of execution would provide plenty of computing power. However, a direct implementation of computers based on the dataflow model has been found to be a monumental challenge. This article presents a survey of issues and evolutionary developments in dataflow computing. The discussion includes the major problems encountered in past dataflow projects and the emergence novel ideas which have shed a new light in dataflow computing.

Software agents can be used to automate many of the tedious, time-consuming information processing tasks that humans currently have to complete manually. However, to do so, agent plans must be capable of representing the myriad of actions and control flows required to perform those tasks. In addition, since these tasks can require integrating multiple sources of remote information – typically, a slow, I/O-bound process – it is desirable to make execution as efficient as possible. To address both of these needs, we present a flexible software agent plan language and a highly parallel execution system that enable the efficient execution of expressive agent plans. The plan language allows complex tasks to be more easily expressed by providing a variety of operators for flexibly processing the data as well as supporting subplans (for modularity) and recursion (for indeterminate looping). The executor is based on a streaming dataflow model of execution to maximize the amount of operator and data parallelism possible at runtime. We have implemented both the language and executor in a system called THESEUS. Our results from testing THESEUS show that streaming dataflow execution can yield significant speedups over both traditional serial (von Neumann) as well as non-streaming dataflow-style execution that existing software and robot agent execution systems currently support. In addition, we show how plans written in the language we present can represent certain types of subtasks that cannot be accomplished using the languages supported by network query engines. Finally, we demonstrate that the increased expressivity of our plan language does not hamper performance; specifically, we show how data can be integrated from multiple remote sources just as efficiently using our architecture as is possible with a state-of-the-art streaming-dataflow network query engine.

: The MIT J-Machine [2], a massively-parallel computer, is an experiment in providing general-purpose mechanisms for communication, synchronization, and naming that will support a wide variety of parallel models of comptuation. We have developed two experimental dataflow programming systems for the J-Machine. For the first system, we adapted Papadopoulos' explicit token store [12] to implement static and then dynamic dataflow. Our second system made use of Iannucci's hybrid execution model [10] to combine several dataflow graph nodes into a single sequence, decreasing scheduling overhead. By combining the strengths of the two systems, it is possible to produce a system with competitive performance. We have demonstrated the feasibility of efficiently executing dataflow programs on a general-purpose parallel computer. Keywords: compilation, parallelization, dataflow, hybrid architectures, MIMD. The research described in this paper was supported in part by the Defense Advanced Research P...

: Id is an architecture-independent, general-purpose parallel programming language that has evolved and been in use for a number of years. Id does not have a sequential core; rather, it is implicitly parallel, with the programmer introducing sequencing explicitly only if necessary. Id is a mostly-functional language, in the family of non-strict functional languages with a HindleyMilner static type system. Unlike other non-strict functional languages, it uses lenient, not lazy evaluation, for reasons of parallelism, as well as to give meaning to non-functional constructs. The non-functional constructs come in two layers: I-structures (which preserve determinacy, but not referential transparency, and are closely related to logic variables), and M-structures, which are side-effects with implicit synchronization. The layers are distinguished by syntax and types, so that it is possible for an implementation to force a program to be within a desired layer (e.g., purely functional)....

Synchronization is a crucial operation in many parallel applications. Conventional synchronization mechanisms are failing to keep up with the increasing demand for efficient synchronization operations as systems grow larger and network latency increases. The contributions of this paper are threefold. First, we revisit some representative synchronization algorithms in light of recent architecture innovations and provide an example of how the simplifying assumptions made by typical analytical models of synchronization mechanisms can lead to significant performance estimate errors. Second, we present an architectural innovation called active memory that enables very fast atomic operations in a shared-memory multiprocessor. Third, we use execution-driven simulation to quantitatively compare the performance of a variety of synchronization mechanisms based on both existing hardware techniques and active memory operations. To the best of our knowledge, synchronization based on active memory outforms all existing spinlock and non-hardwired barrier implementations by a large margin.

. Proponents of data flow machines always emphasize the high potential for parallelization and the high performance on numerical applications reachable by that kind of architecture. On the other hand opponents critisize the additional overhead during run time and the difficulties of the memory management. In order to evaluate the cost effectiveness of dynamical data flow machines we compare a well known representative, the Monsoon designed at the MIT, with two vector processors, the Cray I and the Spark 2.0. This comparison is based on a theoretical model, which provides measures for the run time and the cost of the machines. As numerical workload we have chosen some kernels from the Livermore Loops benchmark. In this paper we briefly review the evaluation model, we describe the machines under consideration and finally we present our results of the comparison. 1 Introduction Vector processors are able to execute identical operations on large sets of data using pipelined functional u...

A processor with an explicit dataflow instruction-set architecture may be able to achieve performance comparable to a superscalar RISC processor, even on serial code. To achieve this, the dataflow processor must support speculative operation, especially speculative branches, and a pipeline with bypassing for serial code. This paper outlines a set of mechanisms to implement speculative operation with a bypassing pipeline, in a paper design called the Speculative Dataflow Processor (SDP). The SDP uses several novel ideas as compared to traditional dataflow processors. Branches are predicted and speculated using a new branch firing rule. Several branch statements are grouped together so that they use a single branch prediction. The scheduling and bypass logic is similar to, but simpler and faster than, the corresponding logic in a superscalar RISC processor.

The MIT J-Machine [3], a massively-parallel computer, is an experiment in providing general-purpose mechanisms for communication, synchronization, and naming that will support a wide variety of parallel models of computation. We have developed two experimental dataflow programming systems for the J-Machine. For the first system, we adapted Papadopoulos' explicit token store [10] to implement static and then dynamic dataflow. Our second system made use of Iannucci's hybrid execution model [8] to combine several dataflow graph nodes into a single sequence, decreasing scheduling overhead. The conclusion describes how the strengths of the two systems can be combined. We have demonstrated the feasibility of efficiently executing dataflow programs on a general-purpose parallel computer. Introduction The dataflow programming model is attractive because it exposes parallelism in computer programs, a crucial step in compiling for multiprocessing computers. Once a program has been converted i...