Google’s MapReduce programming model serves for processing large data sets in a massively parallel manner. We deliver the first rigorous description of the model including its advancement as Google’s domain-specific language Sawzall. To this end, we reverse-engineer the seminal papers on MapReduce and Sawzall, and we capture our findings as an executable specification. We also identify and resolve some obscurities in the informal presentation given in the seminal papers. We use typed functional programming (specifically Haskell) as a tool for design recovery and executable specification. Our development comprises three components: (i) the basic program skeleton that underlies MapReduce computations; (ii) the opportunities for parallelism in executing MapReduce computations; (iii) the fundamental characteristics of Sawzall’s aggregators as an advancement of the MapReduce approach. Our development does not formalize the more implementational aspects of an actual, distributed execution of MapReduce computations.

The problems involved in developing efficient parallel programs have proved harder than those in developing efficient sequential ones, both for programmers and for compilers. Although program calculation has been found to be a promising way to solve these problems in the sequential world, we believe that it needs much more effort to study its effective use in the parallel world. In this paper, we propose a calculational framework for the derivation of efficient parallel programs with two main innovations: - We propose a novel inductive synthesis lemma based on which an elementary but powerful parallelization theorem is developed. - We make the first attempt to construct a calculational algorithm for parallelization, deriving associative operators from data type definition and making full use of existing fusion and tupling calculations. Being more constructive, our method is not only helpful in the design of efficient parallel programs in general but also promising in the construc...

Abstract program schemes, such as scan or homomorphism, can capture a wide range of data parallel programs. While versatile, these schemes are of limited practical use on their own. A key problem is that the more natural sequential specifications may not have associative combine operators required by these schemes. As a result, they often fail to be immediately identified. To resolve this problem, we propose a method to systematically derive parallel programs from sequential definitions. This method is special in that it can automatically invent auxiliary functions needed by associative combine operators. Apart from a formalisation, we also provide new theorems, based on the notion of context preservation, to guarantee parallelization for a precise class of sequential programs. 1 Introduction It is well-recognised that a key problem of parallel computing remains the development of efficient and correct parallel software. This task is further complicated by the variety of parallel arc...

Parallel primitives (skeletons) intend to encourage programmers to build a parallel program from ready-made components for which efficient implementations are known to exist, making the parallelization process easier. However, programmers often suffer from the difficulty to choose a combination of proper parallel primitives so as to construct efficient parallel programs. To overcome this difficulty, we shall propose a new transformation, called diffusion, which can efficiently decompose a recursive definition into several functions such that each function can be described by some parallel primitive. This allows programmers to describe algorithms in a more natural recursive form. We demonstrate our idea with several interesting examples. Our diffusion transformation should be significant not only in development of new parallel algorithms, but also in construction of parallelizing compilers.

. Systematic parallelization of sequential programs remains a major challenge in parallel computing. Traditional approaches using program schemes tend to be narrower in scope, as the properties which enable parallelism are difficult to capture via ad-hoc schemes. In [CTH98], a systematic approach to parallelization based on the notion of preserving the context of recursive sub-terms has been proposed. This approach can be used to derive a class of divide-and-conquer algorithms. In this paper, we enhance the methodology by using invariants to guide the parallelization process. The enhancement enables the parallelization of a class of recursive functions with conditional and tupled constructs, which were not possible previously. We further show how such invariants can be discovered and verified systematically, and demonstrate the power of our methodology by deriving a parallel code for maximum segment product. To the best of our knowledge, this is the first systematic parall...

Parallel programming is becoming an important cornerstone of general computing. In addition, type systems have significant impact on program analysis. In this paper, we demonstrate an automated typebased system that soundly detects parallelizability of sequential functional programs. Our type inference system discovers the parallelizability property of a sequential program in a modular fashion, by exploring a ring structure among the program’s operators. It handles self-recursive functions with accumulating parameters, as well as a class of non-linear mutual-recursive functions. Programs whose types are inferred to be parallelizable can be automatically transformed to parallel code in a mutumorphic form – a succint model for parallel computation. Transforming into such a form is an important step towards constructing efficient data parallel programs.

this paper, we propose

) Abstract We propose an inductive method to synthesize parallel divide-and-conquer programs from sequential recursive functions. Traditionally, such parallelization methods are based on schematic rules which attempt to match each given sequential program to a prescribed set of program schemes that have parallel counterparts. Instead of relying on specialized program schemes, we propose a new approach to parallelization based on elementary transformation rules. Our approach requires an induction to recover parallellism from sequential programs. To achieve this, we apply a second-order generalisation step to selected instances of sequential equations, before an inductive derivation procedure. The new approach is systematic enough to be semi-automated, and shall be shown to be widely applicable using a range of examples. 1 Introduction Most programs are more easily written via sequential specifications. Functional programs are no exception. As an example, the ubiquituous list...

Parallel programming is becoming an important cornerstone of general computing. In addition, type systems have significant impact on program analysis. In this paper, we demonstrate an automated typebased system that soundly synthesizes parallel programs from sequential functional programs. Our type inference system discovers the parallelizability property of a sequential program in a modular fashion, by exploring a ring structure among the program's operators. It handles self-recursive functions with accumulating parameters, as well as a class of non-linear mutual-recursive functions. We automatically generates parallel code in a mutumorphic form -- a succint model for parallel computation. Transforming into

We propose an inductive method to synthesize parallel divide-and-conquer programs from sequential recursive functions. Traditionally, such parallelization methods are based on schematic rules which attempt to match each given sequential program to a prescribed set of program schemes that have parallel counterparts. Instead of relying on specialized program schemes, we propose a new approach to parallelization based on elementary transformation rules. Our approach requires an induction to recover parallellism from sequential programs. To achieve this, we apply a second-order generalisation step to selected instances of sequential equations, before an inductive derivation procedure. The new approach is systematic enough to be semi-automated, and shall be shown to be widely applicable using a range of examples.