## Automatic Inversion Generates Divide-and-Conquer Parallel Programs

Citations: | 7 - 5 self |

### BibTeX

@MISC{Morita_automaticinversion,

author = {Kazutaka Morita and Akimasa Morihata and Kiminori Matsuzaki and Zhenjiang Hu and Masato Takeichi},

title = {Automatic Inversion Generates Divide-and-Conquer Parallel Programs},

year = {}

}

### OpenURL

### Abstract

Divide-and-conquer algorithms are suitable for modern parallel machines, tending to have large amounts of inherent parallelism and working well with caches and deep memory hierarchies. Among others, list homomorphisms are a class of recursive functions on lists, which match very well with the divide-and-conquer paradigm. However, direct programming with list homomorphisms is a challenge for many programmers. In this paper, we propose and implement a novel system that can automatically derive costoptimal list homomorphisms from a pair of sequential programs, based on the third homomorphism theorem. Our idea is to reduce extraction of list homomorphisms to derivation of weak right inverses. We show that a weak right inverse always exists and can be automatically generated from a wide class of sequential programs. We demonstrate our system with several nontrivial examples, including the maximum prefix sum problem, the prefix sum computation, the maximum segment sum problem, and the line-ofsight problem. The experimental results show practical efficiency of our automatic parallelization algorithm and good speedups of the generated parallel programs.

### Citations

2022 | Mapreduce: Simplified data processing on large clusters
- DEAN, GHEMAWAT
- 2004
(Show Context)
Citation Context ...and working well with caches and deep memory hierarchies [28]. Among others, list homomorphisms are a class of recursive functions on lists, which match very well with the divide-and-conquer paradigm =-=[9, 11, 24, 27, 30]-=-. Function h is said to be a list homomorphism, if Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are n... |

498 |
The Science of Programming
- Gries
- 1981
(Show Context)
Citation Context ...ther approaches. 6.2 Inversion and Parallelization We have reduced parallelization process to derivation of a weak right inverse, and thus our approach is related to researches on automatic inversion =-=[12, 13, 17, 18, 20, 23]-=-. Different from the inverse that does not always exist for a given function, our weak right inverse always exists for any function. To guarantee the efficiency of the weak right inverse, as in Sectio... |

477 | The Omega test: a fast and practical integer programming algorithm for dependence analysis
- Pugh
- 1991
(Show Context)
Citation Context ...pressions in the optimization step are in thesform of Presburger arithmetic, and we used the Omega library that solves truth judgments in the Presburger arithmetic fast based on the omega test method =-=[26]-=-. The use of the Omega library enables our parallel code generator to work in practical time. The generation of architecture-independent executable parallel programs is another implementation issue. W... |

227 |
Introduction to Functional Programming using Haskell
- Bird
(Show Context)
Citation Context ...ves a necessary and sufficient condition for the existence of list homomorphisms. 2.1 Notations on Functions and Lists Our notations are basically based on the functional programming language Haskell =-=[5]-=-. Functional application is denoted by a space and an argument may be written without brackets. Thus f a means f(a). Functions are curried, i.e. functions take one argument and return a function or a ... |

221 | An introduction to the theory of lists
- Bird
- 1987
(Show Context)
Citation Context ...d capture of the structure of recursive computation. We classify most of the list manipulation computation into three classes, namely rightwards functions, leftwards functions, and list homomorphisms =-=[4]-=-. Definition 2.1 (Leftwards function). Function h is leftwards if it is defined in the following form with function f and operator ⊕. h [a] = f a h ([a] ++ x) = a ⊕ h x That is, a leftwards function i... |

102 | Prefix sums and their applications
- Blelloch
- 1993
(Show Context)
Citation Context ...ters. Our algorithm can deal with the latter style of sequential programs, whenever f ′ 1, f ′ 2, g ′ 1, and g ′ 2 are defined leftwards and rightwards. As an example, consider the prefix sum problem =-=[7, 14, 15]-=-: given a list, compute the sums of all the prefix sublists. It can be sequentially described by psum x = psum ′ x 0 psum ′ [a] c = [a + c] psum ′ ([a] ++ x) c = [a + c] ++ psum ′ x (a + c) which can ... |

96 |
Theorem proving in arithmetic without multiplication
- Cooper
- 1972
(Show Context)
Citation Context ...domain. Therefore, the expression corresponding to Ci is redundant and can be removed. The expression (1) is in the form of Presburger arithmetic [25], and we can compute it by quantifier elimination =-=[10]-=-. 4.3.4 Verification As mentioned in Section 4.3.1, the derived weak right inverse may be a partial function. That is, W i Ci = true may not hold, where Ci denotes the ith condition. However, it is su... |

55 |
Algorithmic skeletons: a structured approach to the management of parallel computation
- Cole
- 1988
(Show Context)
Citation Context ...esult does not depend on the place where the input list is split, because the operator ⊙ is associative.s2.3 Parallel Skeletons List homomorphisms play a central role in skeletal parallel programming =-=[8, 27]-=-, in which users are provided with a fixed set of parallel computation patterns (skeletons) to write parallel programs. The following four parallel skeletons, namely map, reduce, and two scans, are co... |

55 |
Parallelizing complex scans and reductions
- Fisher, Ghuloum
- 1994
(Show Context)
Citation Context ...d can be automatically generated for a wide class of sequential functions. As will be seen later, this new approach is applicable to many nontrivial examples, including the maximum prefix sum problem =-=[6, 14, 15]-=-, the prefix sum computation [6], the maximum segment sum problem [3], and the line-of-sight problem [6]. Our main contribution can be summarized as follows. • We design a new automatic parallelizatio... |

49 | Automatic parallelization of divide and conquer algorithms
- Rugina, Rinard
(Show Context)
Citation Context ...e a solution to the original problem. They match very well for modern parallel machines, tending to have large amounts of inherent parallelism and working well with caches and deep memory hierarchies =-=[28]-=-. Among others, list homomorphisms are a class of recursive functions on lists, which match very well with the divide-and-conquer paradigm [9, 11, 24, 27, 30]. Function h is said to be a list homomorp... |

43 |
Parallel Programming with List Homomorphisms. Parallel Processing
- Cole
- 1995
(Show Context)
Citation Context ...and working well with caches and deep memory hierarchies [28]. Among others, list homomorphisms are a class of recursive functions on lists, which match very well with the divide-and-conquer paradigm =-=[9, 11, 24, 27, 30]-=-. Function h is said to be a list homomorphism, if Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are n... |

33 | W.N.: Parallelization in calculational forms
- Hu, Takeichi, et al.
- 1998
(Show Context)
Citation Context ...of natural recursive functions on lists, they enjoy many nice algebraic properties, among which, the three well-known homomorphism lemmas form the basis of the formal development of parallel programs =-=[9,19,21,22]-=-. Secondly, and very importantly, they are useful to solve really practical problems. For example, many algorithms executed on Google’s clusters are on MapReduce [24], and most of them, such as distri... |

26 | The Third Homomorphism Theorem
- Gibbons
- 1995
(Show Context)
Citation Context ...rd Homomorphism Theorem When writing a program in terms of list homomorphisms or parallel skeletons, the most difficult step is to find the associative binary operator. The third homomorphism theorem =-=[16]-=- is the theorem that gives a necessary and sufficient condition for the existence of the associative binary operator. Theorem 2.1 (Third homomorphism theorem [16]). Function h can be described as a li... |

25 |
die vollstandigkeit eines gewissen Systems der Arithmetic ganzer Zahlen, in welchem die Addition als einzige Operation hervortritt
- Uber
- 1930
(Show Context)
Citation Context ... as a weak right inverse if the input value is in the domain. Therefore, the expression corresponding to Ci is redundant and can be removed. The expression (1) is in the form of Presburger arithmetic =-=[25]-=-, and we can compute it by quantifier elimination [10]. 4.3.4 Verification As mentioned in Section 4.3.1, the derived weak right inverse may be a partial function. That is, W i Ci = true may not hold,... |

24 |
Scans as primitive operations
- Blelloch
- 1989
(Show Context)
Citation Context ...phisms in parallel programming, a challenge remains for a programmer to use them to solve their problems, particularly when the problems are a bit complicated. Consider the maximum prefix sum problem =-=[6]-=-, which is to compute the maximum sum of all the prefix sublists. For instance, supposing mps is the function that solves the problem, we have mps [1, −1,2] = 0 ↑ 1 ↑ (1 + (−1)) ↑ (1 + (−1) + 2) = 2 w... |

22 |
M.: Formal derivation of efficient parallel programs by construction of list homomorphisms
- Hu, Iwasaki, et al.
- 1997
(Show Context)
Citation Context ...of natural recursive functions on lists, they enjoy many nice algebraic properties, among which, the three well-known homomorphism lemmas form the basis of the formal development of parallel programs =-=[9,19,21,22]-=-. Secondly, and very importantly, they are useful to solve really practical problems. For example, many algorithms executed on Google’s clusters are on MapReduce [24], and most of them, such as distri... |

21 |
Program inversion
- Dijkstra
- 1978
(Show Context)
Citation Context ...ther approaches. 6.2 Inversion and Parallelization We have reduced parallelization process to derivation of a weak right inverse, and thus our approach is related to researches on automatic inversion =-=[12, 13, 17, 18, 20, 23]-=-. Different from the inverse that does not always exist for a given function, our weak right inverse always exists for any function. To guarantee the efficiency of the weak right inverse, as in Sectio... |

18 | Systematic extraction and implementation of divide-and-conquer parallelism
- Gorlatch
- 1996
(Show Context)
Citation Context ...of natural recursive functions on lists, they enjoy many nice algebraic properties, among which, the three well-known homomorphism lemmas form the basis of the formal development of parallel programs =-=[9,19,21,22]-=-. Secondly, and very importantly, they are useful to solve really practical problems. For example, many algorithms executed on Google’s clusters are on MapReduce [24], and most of them, such as distri... |

17 | A library of constructive skeletons for sequential style of parallel programming
- Matsuzaki, Iwasaki, et al.
- 2006
(Show Context)
Citation Context ...and working well with caches and deep memory hierarchies [28]. Among others, list homomorphisms are a class of recursive functions on lists, which match very well with the divide-and-conquer paradigm =-=[9, 11, 24, 27, 30]-=-. Function h is said to be a list homomorphism, if Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are n... |

12 | A Heuristic Approach to Program Inversion
- Eppstein
- 1985
(Show Context)
Citation Context ...ther approaches. 6.2 Inversion and Parallelization We have reduced parallelization process to derivation of a weak right inverse, and thus our approach is related to researches on automatic inversion =-=[12, 13, 17, 18, 20, 23]-=-. Different from the inverse that does not always exist for a given function, our weak right inverse always exists for any function. To guarantee the efficiency of the weak right inverse, as in Sectio... |

10 | A program inverter for a functional language with equality and constructors
- Glück, Kawabe
- 2003
(Show Context)
Citation Context |

9 | Parallelizing functional programs by generalization
- GESER, GORLATCH
- 1997
(Show Context)
Citation Context ...momorphisms, it mentions nothing of how to construct them. In fact, it remains open whether there is a general and automatic way to extract an efficient list homomorphism from two sequential programs =-=[15]-=-. In this paper, we propose a novel approach to automatic derivation of cost-optimal list homomorphisms from a wide class of sequential programs. Our idea is to reduce automatic extraction of list hom... |

9 |
Derivation of deterministic inverse programs based on LR parsing
- Glück, Kawabe
- 2004
(Show Context)
Citation Context |

8 | Inversion of applicative programs
- Korf
- 1981
(Show Context)
Citation Context |

8 |
Parallel programming and parallel abstractions in Fortress
- Steele
(Show Context)
Citation Context |

7 | An analytical method for parallelization of recursive functions
- Ahn, Han
(Show Context)
Citation Context ...+ [b]) = 0 ↑ a ↑ (a + b) and conclude (see Theorem 3.3) that mps (x ++ y) = mps x ⊙ mps y. This derived homomorphism is actually incorrect, because mps [1, −2,2, 1] should be 2, but mps [1, −2] ⊙ mps =-=[2, 1]-=- gives 3. The problem in this derivation is in its wrong application of Theorem 3.3; it did not check whether the function can be written by both leftwards and rightwards functions, which is required ... |

5 |
Algorithm design techniques
- Bentley
(Show Context)
Citation Context ... will be seen later, this new approach is applicable to many nontrivial examples, including the maximum prefix sum problem [6, 14, 15], the prefix sum computation [6], the maximum segment sum problem =-=[3]-=-, and the line-of-sight problem [6]. Our main contribution can be summarized as follows. • We design a new automatic parallelization algorithm based on the third homomorphism theorem, by reformalizing... |

4 | Parallel solution of simple indexed recurrence equations
- Ben-Asher, Haver
- 2001
(Show Context)
Citation Context ...+ [b]) = 0 ↑ a ↑ (a + b) and conclude (see Theorem 3.3) that mps (x ++ y) = mps x ⊙ mps y. This derived homomorphism is actually incorrect, because mps [1, −2,2, 1] should be 2, but mps [1, −2] ⊙ mps =-=[2, 1]-=- gives 3. The problem in this derivation is in its wrong application of Theorem 3.3; it did not check whether the function can be written by both leftwards and rightwards functions, which is required ... |

4 |
Make it practical: A generic linear time algorithm for solving maximum weightsum problems
- Sasano, Hu, et al.
- 2000
(Show Context)
Citation Context ... system, we consider the maximum segment sum problem, which computes the maximum of the sums for all the segments (contiguous sublists) of a list. It is an instance of the maximum weight-sum problems =-=[29]-=- that capture many dynamic-programming problems. Our system can automatically parallelize all the problems on lists in [29]. As an example, we show automatic parallelization of the maximum segment sum... |

3 | Ptype system: A featherweight parallelizability detector
- Xu, Hu
- 2004
(Show Context)
Citation Context ...as gained great interest, and there have been many approaches, such as the third homomorphism theorem based method [15, 19], function composition based method [14], matrix multiplication based method =-=[31]-=- and recurrence equation based method [2]. Our approach is unique in its use of weak right inverse in derivation of parallel programs. One of the advantages of our approach is that our parallelization... |