Abstract Building a cost calculus for a parallel program development environment is difficult because of the many degrees of freedom available in parallel implementations, and because of difficulties with compositionality. We present a strategy for building cost calculi for skeleton-based programming languages which can be used for derivational software development and which deals in a pragmatic way with the difficulties of composition. The approach is illustrated for the Bird-Meertens theory of lists, a parallel functional language with an associated equational transformation system. Keywords: functional programming, parallel programming, program transformation, cost calculus, equational theories, architecture independence, Bird-Meertens formalism.

We review the use of the Bird-Meertens Formalism as a vehicle for the construction of programs with massive implicit parallelism. We show that a simple result from the theory, concerning the expression of list homomorphisms, can help us in our search for parallel algorithms and demonstrate its application to some simple problems including the maximum segment sum problem. Our main purpose is to show that an understanding of the homomorphism lemma can be helpful in producing programs for problems which are "not quite" list homomorphisms themselves. A more general goal is to illustrate the benefits which can arise from taking a little theory with a pinch of pragmatic salt. 1 Introduction The use of bulk operations on aggregate data sets as a means of generating programs with a high degree of implicit parallelism has a long history (e.g. see [4] for a recent presentation). Although traditionally associated with an imperative programming style and SIMD machines, the approach lends itself e...

We present a family of parallel algorithms for simple language recognition problems involving bracket matching. The algorithms are expressed in the Bird-Meertens Formalism, exploiting only list operations which are inherently massively parallel. Our intention is to illustrate the practical efficacy with which such algorithms can be derived and expressed given the support of a well understood theoretical foundation. One of the variants produced is of particular interest in that it exploits the same theoretical result twice to produce nested parallelism. 1 Introduction In [8], we investigated an informal methodology for the generation of parallel algorithms based upon exploitation of a fundamental result from the Bird-Meertens "theory of lists". Our main example was an algorithm for the maximum segment sum problem. In this report we provide further examples of the approach. For completeness, the remainder of this section and sections 2 and 3 repeat the introductory material from [8]. Re...

Introduction We study two popular programming schemas: scan (also known as prefix sums, parallel prefix, etc.) and reduction (also known as fold). Originally from the functional world [3], they are becoming increasingly popular as primitives of parallel programming. The reasons are that, first, such higher-order combinators are adequate and useful for a broad class of applications [4], second, they encourage well-structured, coarse-grained parallel programming and, third, their implementation in the MPI standard [14] makes the target programs portable across different parallel architectures with predictable performance. Our contributions are as follows: -- We formally prove two optimization rules: the first rule transforms a sequential composition of scan and reduction into a single reduction, the second rule transforms a composition of two scans into a single scan. -- We apply the first rule in the formal derivation of a parallel algorithm for the

We study the use of well-defined building blocks for SPMD programming of machines with distributed memory. Our general framework is based on homomorphisms, functions that capture the idea of dataparallelism and have a close correspondence with collective operations of the MPI standard, e.g., scan and reduction. We prove two composition rules: under certain conditions, a composition of a scan and a reduction can be transformed into one reduction, and a composition of two scans into one scan. As an example of decomposition, we transform a segmented reduction into a composition of partial reduction and all-gather. The performance gain and overhead of the proposed composition and decomposition rules are assessed analytically for the hypercube and compared with the estimates for some other parallel models.

List homomorphisms are functions which can be efficiently computed in parallel since they ideally suit the divide-and-conquer paradigm. However, some interesting functions, e.g., the maximum segment sum problem, are not list homomorphisms. In this paper, we propose a systematic way of embedding them into list homomorphisms so that parallel programs are derived. We show, with an example, how a simple, and "obviously" correct, but possibly inefficient solution to the problem can be successfully turned into a semantically equivalent almost homomorphism by means of two transformations: tupling and fusion.

The Bird-Meertens formalism (BMF) of higher-order functions over lists is a mathematical framework supporting formal derivation of algorithms from functional specifications. This paper reports results of a case study on the systematic use of BMF in the process of parallel program development. We develop a parallel program for polynomial multiplication, starting with a straight-forward mathematical specification and arriving at the target processor topology together with a program for each processor of it. The development process is based on formal transformations; design decisions concerning data partitioning, processor interconnections, etc. are governed by formal type analysis and performance estimation rather than made ad hoc. The parallel target implementation is parameterized for an arbitrary number of processors; for the particular number, the target program is both time and cost-optimal. We compare our results with systolic solutions to polynomial multiplication.

. It has been attracting much attention to make use of list homomorphisms in parallel programming because they ideally suit the divide-and-conquer parallel paradigm. However, they have been usually treated rather informally and ad-hoc in the development of efficient parallel programs. This paper reports a case study on systematic and formal development of a new parallel program for the 2-dimensional maximum segment problem. We show how a straightforward, and "obviously" correct, but quite inefficient solution to the problem can be successfully turned into a semantically equivalent "almost list homomorphism" based on two transformations, namely tupling and fusion, which are defined according to the specific recursive structures of list homomorphisms. 1 Introduction It has been attracting wide attention to make use of list homomorphisms in parallel programming. List homomorphisms are those functions on finite lists that promote through list concatenation --- that is, function h for whic...

Broadcast, Reduction and Scan are popular functional skeletons which are used in distributed algorithms to distribute and gather data. We derive new parallel implementations of combinations of Broadcast, Reduction and Scan via a tabular classification of linearly recursive functions. The trick in the derivation is to not simply combine the individual parallel implementations of Broadcast, Reduction and Scan, but to transform these combinations to skeletons with a better performance. These skeletons are also linearly recursive. Keywords: functional programming, linear recursion, parallelization, skeletons 1. Introduction Functional programming offers a very high-level approach to specifying executable problem solutions. For example, the scheme of linear recursion can be expressed concisely as a higher-order function. In the data-parallel world, higher-order functions are used which represent classes of parallel algorithms on data structures; these higher-order functions are also call...

ion and Performance in the Design of Parallel Programs Der Fakultat fur Mathematik und Informatik der Universitat Passau vorgelegte Zusammenfassung der Veroffentlichungen zur Erlangung der venia legendi von Dr. Sergei Gorlatch Passau, Juli 1997 Contents 1 Introduction 2 2 Outline of the SAT Approach 6 2.1 Performance View . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Abstraction View . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Design in SAT: Stages and Transformations . . . . . . . . . . 9 2.4 SAT and Homomorphisms . . . . . . . . . . . . . . . . . . . . 11 3 List Homomorphisms 12 4 Extraction and Adjustment 14 4.1 The CS-Method . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.2 Mechanizing the CS-Method . . . . . . . . . . . . . . . . . . . 17 4.3 Almost-Homomorphisms: the MSS Problem . . . . . . . . . . 19 5 Composition of Homomorphisms 21 5.1 Rules of Composition . . . . . . . . . . . . . . . . . . . . . . . 21 5.2 Derivation by Transformation...