We survey parallel programming models and languages using 6 criteria [:] should be easy to program, have a software development methodology, be architecture-independent, be easy to understand, guranatee performance, and provide info about the cost of programs. ... We consider programming models in 6 categories, depending on the level of abstraction they provide.

Because of the complexity and variety of parallel architectures, an efficient machine-independent parallel programming model is needed to make parallel computing truly usable for scientific programmers. We believe that Fortran D, a version of Fortran enhanced with data decomposition specifications, can provide such a programming model. This paper presents the design of a prototype Fortran D compiler for the iPSC/860, a MIMD distributed-memory machine. Issues addressed include data decomposition analysis, guard introduction, communications generation and optimization, program transformations, and storage assignment. A test suite of scientific programs will be used to evaluate the effectiveness of both the compiler technology and programming model for the Fortran D compiler.

Abstract Building a cost calculus for a parallel program development environment is difficult because of the many degrees of freedom available in parallel implementations, and because of difficulties with compositionality. We present a strategy for building cost calculi for skeleton-based programming languages which can be used for derivational software development and which deals in a pragmatic way with the difficulties of composition. The approach is illustrated for the Bird-Meertens theory of lists, a parallel functional language with an associated equational transformation system. Keywords: functional programming, parallel programming, program transformation, cost calculus, equational theories, architecture independence, Bird-Meertens formalism.

Fully-parallel execution of a high-level data-parallel language based on nested sequences, higher order functions and generalized iterators can be realized in the vector model using a suitable representation of nested sequences and a small set of transformational rules to distribute iterators through the constructs of the language.

The most important features that a parallel programming language should provide are portability, modularity, and easy usability, as well as performance and efficiency. Current parallel languages are only characterized by some of these features. For instance, most of these languages allow programmers to efficiently exploit the massively parallel target machine. Unfortunately, the estimation of the performance of each application is usually made by the programmer, without the support of any tool. Moreover, the programs produced by using such languages are not portable and easily modifiable. Here, we present a methodology to easily write efficient, high performance and portable massively parallel programs. The methodology is based on the definition of a new explicitly parallel programming language, namely P 3 L, and of a set of compiling tools that perform automatic adaptation of the program features to the target architecture hardware. Target architectures taken into account here are g...

Google’s MapReduce programming model serves for processing large data sets in a massively parallel manner. We deliver the first rigorous description of the model including its advancement as Google’s domain-specific language Sawzall. To this end, we reverse-engineer the seminal papers on MapReduce and Sawzall, and we capture our findings as an executable specification. We also identify and resolve some obscurities in the informal presentation given in the seminal papers. We use typed functional programming (specifically Haskell) as a tool for design recovery and executable specification. Our development comprises three components: (i) the basic program skeleton that underlies MapReduce computations; (ii) the opportunities for parallelism in executing MapReduce computations; (iii) the fundamental characteristics of Sawzall’s aggregators as an advancement of the MapReduce approach. Our development does not formalize the more implementational aspects of an actual, distributed execution of MapReduce computations.

We review the use of the Bird-Meertens Formalism as a vehicle for the construction of programs with massive implicit parallelism. We show that a simple result from the theory, concerning the expression of list homomorphisms, can help us in our search for parallel algorithms and demonstrate its application to some simple problems including the maximum segment sum problem. Our main purpose is to show that an understanding of the homomorphism lemma can be helpful in producing programs for problems which are "not quite" list homomorphisms themselves. A more general goal is to illustrate the benefits which can arise from taking a little theory with a pinch of pragmatic salt. 1 Introduction The use of bulk operations on aggregate data sets as a means of generating programs with a high degree of implicit parallelism has a long history (e.g. see [4] for a recent presentation). Although traditionally associated with an imperative programming style and SIMD machines, the approach lends itself e...

Shape theory provides a framework for the study of data types in which shape and data can be manipulated separately. This paper is concerned with shape checking, i.e. the detection of shape errors, such as array bound errors, without handling the data stored within. It can be seen as a form of partial evaluation in which data computations are ignored. We construct a simply-typed lambda-calculus that supports a vector type constructor, whose iteration yields types of arrays. It is expressive enough to construct all of the usual linear algebra operations. All shape errors in a term t can be detected by evaluating its shape #t. Evaluation of #t will terminate if that of t does. Keywords shape analysis, partial evaluation, arrays, higher-order. 1 Introduction Shape theory explores the consequences of manipulating shape and data separately (Jay [14]). Shape refers to the data structure in which the data is stored. For example, the shape of a three-dimensional regular array is a tuple of...

The skeleton approach to programming parallel machines promises to offer a high-level of abstraction to the programmer, whilst providing the implementation with sufficient information to effectively manage the resources available. Each skeleton captures a common pattern of computation and has associated with it parallel implementations. Functional programming languages are a suitable framework for exploring this approach as skeletons can be elegantly represented as higher-order functions. Applications are naturally expressed as combinations of several skeletons. This thesis explores the problem of optimising combinations of skeletons, where each skeleton may have more than one underlying parallel implementation. A skeleton approach suitable for expressing programs as combinations of skeletons is presented. The primitive skeletons of this approach are operators of parallel abstract data types. Skeletons are combined together using a set of combining skeletons which abstract patterns of...

Abstract program schemes, such as scan or homomorphism, can capture a wide range of data parallel programs. While versatile, these schemes are of limited practical use on their own. A key problem is that the more natural sequential specifications may not have associative combine operators required by these schemes. As a result, they often fail to be immediately identified. To resolve this problem, we propose a method to systematically derive parallel programs from sequential definitions. This method is special in that it can automatically invent auxiliary functions needed by associative combine operators. Apart from a formalisation, we also provide new theorems, based on the notion of context preservation, to guarantee parallelization for a precise class of sequential programs. 1 Introduction It is well-recognised that a key problem of parallel computing remains the development of efficient and correct parallel software. This task is further complicated by the variety of parallel arc...