## Efficient External Memory Algorithms by Simulating Coarse-Grained Parallel Algorithms (2003)

### Cached

### Download Links

- [134.117.206.249:8000]
- [www.dehne.carleton.ca]
- [www.dehne.carleton.ca]
- [www.scs.carleton.ca]
- [www.scs.carleton.ca]
- [www.uni-paderborn.de]
- DBLP

### Other Repositories/Bibliography

Citations: | 39 - 10 self |

### BibTeX

@MISC{Dehne03efficientexternal,

author = {Frank Dehne and Wolfgang Dittrich and David Hutchinson},

title = {Efficient External Memory Algorithms by Simulating Coarse-Grained Parallel Algorithms},

year = {2003}

}

### Years of Citing Articles

### OpenURL

### Abstract

External memory (EM) algorithms are designed for large-scale computational problems in which the size of the internal memory of the computer is only a small fraction of the problem size. Typical EM algorithms are specially crafted for the EM situation. In the past, several attempts have been made to relate the large body of work on parallel algorithms to EM, but with limited success. The combination of EM computing, on multiple disks, with multiprocessor parallelism has been posted as a challenge by the ACMWorking Group on Storage I/O for Large-Scale Computing.

### Citations

1493 | Probability inequalities for sums of bounded random variables - Hoeffding - 1963 |

1130 |
A Bridging Model for Parallel Computation
- Valiant
- 1990
(Show Context)
Citation Context ...llel computers instead of reinventing new algorithms for EM is intriguing. In this paper we exploit a natural correspondence between external memory algorithms and BSP-like parallel models such as BSP=-=[23]-=-, BSP*[7, 8, 6] and CGM[13, 14, 15]. We provide simulation techniques that map BSP-like algorithms to EM algorithms, and we further show how, using a randomized approach, an EM machine can take full a... |

537 |
The input/output complexity of sorting and related problems
- Aggarwal, Vitter
- 1988
(Show Context)
Citation Context ...al machine with p = bv=rc real processors each one with multiple disks for a large range of integers r. 1.2 Previous Work Sorting, permutation and related problems in EM have been extensively studied =-=[1, 12, 27, 26, 20]-=-. I/O-optimal approaches to many computational geometry problems [17] and graph problems [11] have also been described. Data structures [3, 5, 22] and a number of applications [4] have been examined i... |

234 | Algorithms for parallel memory I: Two level memories
- Vitter, Shriver
- 1994
(Show Context)
Citation Context ...ing factor) too high, and if parallel disks are not properly utilized, the runtime can be a factor of D too high. More discussion of the traditional EM system model and related issues can be found in =-=[27]-=-. We identify coarse-grained communication as a desirable property of parallel algorithms for our simulation. Coarse grained communication occurs when the average message size of the parallel algorith... |

172 | External-memory graph algorithms - Chiang, Goodrich, et al. - 1995 |

165 | Direct bulk-synchronous parallel algorithms
- Gerbessiotis, Valiant
- 1994
(Show Context)
Citation Context ...er strong criterion and it ignores internal computation cost completely. However, work efficiency is a crucial factor for the efficiency of algorithms. The criterion of c-optimality was introduced in =-=[16]-=- for the BSP model. It gives an incentive to devise algorithms for which communication time is asymptotically smaller than computation time and the computation time is only by a factor c+o(1) larger t... |

149 | The bu er tree: A new technique for optimal I/O-algorithms
- Arge
- 1995
(Show Context)
Citation Context ...problems in EM have been extensively studied [1, 12, 27, 26, 20]. I/O-optimal approaches to many computational geometry problems [17] and graph problems [11] have also been described. Data structures =-=[3, 5, 22]-=- and a number of applications [4] have been examined in this context. Some implementation work has also been done [24, 25, 10]. A recent survey is included in [5]. The classical EM model is described ... |

121 | External-memory computational geometry - Goodrich, Tsay, et al. - 1993 |

112 | The uniform memory hierarchy model of computation. Algorithmica
- Alpern, Carter, et al.
- 1994
(Show Context)
Citation Context ... EM model is described in [1]. More complex models have been proposed as well, incorporating a hierarchy of memory layers rather than the two-level memory model of [1]. One such model is described in =-=[2]-=-, and sorting for this model is studied in [26]. Such models are interesting because modern computers typically have several layers of memory which include main memory and caches as well as disks. We ... |

86 |
Scalable parallel geometric algorithms for coarse grained multicomputers
- Dehne, Fabri, et al.
- 1993
(Show Context)
Citation Context ...einventing new algorithms for EM is intriguing. In this paper we exploit a natural correspondence between external memory algorithms and BSP-like parallel models such as BSP[23], BSP*[7, 8, 6] and CGM=-=[13, 14, 15]-=-. We provide simulation techniques that map BSP-like algorithms to EM algorithms, and we further show how, using a randomized approach, an EM machine can take full advantage of parallel disk I/O and m... |

79 |
The P-range tree: a new data structure for range searching in secondard memory
- Subramanian, Ramaswamy
- 1995
(Show Context)
Citation Context ...problems in EM have been extensively studied [1, 12, 27, 26, 20]. I/O-optimal approaches to many computational geometry problems [17] and graph problems [11] have also been described. Data structures =-=[3, 5, 22]-=- and a number of applications [4] have been examined in this context. Some implementation work has also been done [24, 25, 10]. A recent survey is included in [5]. The classical EM model is described ... |

76 | External-rnemory algorithms for processing line segments in geographic information systems
- Arge, Vengroff, et al.
- 1995
(Show Context)
Citation Context ...died [1, 12, 27, 26, 20]. I/O-optimal approaches to many computational geometry problems [17] and graph problems [11] have also been described. Data structures [3, 5, 22] and a number of applications =-=[4]-=- have been examined in this context. Some implementation work has also been done [24, 25, 10]. A recent survey is included in [5]. The classical EM model is described in [1]. More complex models have ... |

74 | Scalable parallel computational geometry for coarse grained multicomputers - Dehne, Fabri, et al. - 1996 |

69 | I/O complexity of graph algorithms - Munagala - 1999 |

60 | Vitter (eds.). External Memory Algorithms - Abello, S - 1999 |

59 | Efficient parallel graph algorithms for coarse grained multicomputers and BSP - Caceres, Dehne, et al. - 1997 |

49 | A randomized parallel 3D convex hull algorithm for coarse grained multicomputers
- Dehne, Deng, et al.
- 1995
(Show Context)
Citation Context ...einventing new algorithms for EM is intriguing. In this paper we exploit a natural correspondence between external memory algorithms and BSP-like parallel models such as BSP[23], BSP*[7, 8, 6] and CGM=-=[13, 14, 15]-=-. We provide simulation techniques that map BSP-like algorithms to EM algorithms, and we further show how, using a randomized approach, an EM machine can take full advantage of parallel disk I/O and m... |

46 |
auf der Heide. Truly efficient parallel algorithms: 1-optimal multisearch for an estension of the BSP rnodel
- Dittrich, Meyer
- 1996
(Show Context)
Citation Context ...ters instead of reinventing new algorithms for EM is intriguing. In this paper we exploit a natural correspondence between external memory algorithms and BSP-like parallel models such as BSP[23], BSP*=-=[7, 8, 6]-=- and CGM[13, 14, 15]. We provide simulation techniques that map BSP-like algorithms to EM algorithms, and we further show how, using a randomized approach, an EM machine can take full advantage of par... |

42 | Vzrtual Memory for Data Parallel Computing
- Cormen
- 1992
(Show Context)
Citation Context ...al machine with p = bv=rc real processors each one with multiple disks for a large range of integers r. 1.2 Previous Work Sorting, permutation and related problems in EM have been extensively studied =-=[1, 12, 27, 26, 20]-=-. I/O-optimal approaches to many computational geometry problems [17] and graph problems [11] have also been described. Data structures [3, 5, 22] and a number of applications [4] have been examined i... |

38 | Efficient External-Memory Data Structures and Applications
- Arge
- 1996
(Show Context)
Citation Context ...problems in EM have been extensively studied [1, 12, 27, 26, 20]. I/O-optimal approaches to many computational geometry problems [17] and graph problems [11] have also been described. Data structures =-=[3, 5, 22]-=- and a number of applications [4] have been examined in this context. Some implementation work has also been done [24, 25, 10]. A recent survey is included in [5]. The classical EM model is described ... |

34 | I/O-efficient scientific computation using TPIE
- Vengroff, Vitter
- 1996
(Show Context)
Citation Context ...lems [17] and graph problems [11] have also been described. Data structures [3, 5, 22] and a number of applications [4] have been examined in this context. Some implementation work has also been done =-=[24, 25, 10]-=-. A recent survey is included in [5]. The classical EM model is described in [1]. More complex models have been proposed as well, incorporating a hierarchy of memory layers rather than the two-level m... |

32 | Scalable and architecture independent parallel geometric algorithms with high probability optimal time
- Dehne, Kenyon, et al.
- 1994
(Show Context)
Citation Context ...einventing new algorithms for EM is intriguing. In this paper we exploit a natural correspondence between external memory algorithms and BSP-like parallel models such as BSP[23], BSP*[7, 8, 6] and CGM=-=[13, 14, 15]-=-. We provide simulation techniques that map BSP-like algorithms to EM algorithms, and we further show how, using a randomized approach, an EM machine can take full advantage of parallel disk I/O and m... |

27 |
Greed Sort: optimal deterministic sorting on parallel disks
- Nodine, Vitter
- 1995
(Show Context)
Citation Context ...al machine with p = bv=rc real processors each one with multiple disks for a large range of integers r. 1.2 Previous Work Sorting, permutation and related problems in EM have been extensively studied =-=[1, 12, 27, 26, 20]-=-. I/O-optimal approaches to many computational geometry problems [17] and graph problems [11] have also been described. Data structures [3, 5, 22] and a number of applications [4] have been examined i... |

25 | Large-scale sorting in uniform memory hierarchies - Vitter, Nodine - 1993 |

23 | Paradigms for optimal sorting with multiple dish - Nodine, Vitter - 1993 |

23 | Practical parallel algorithms for personalized communication and integer sorting - Bader, Helman, et al. - 1996 |

21 |
External memory graph algorithms
- Chiang, Goodrich, et al.
- 1995
(Show Context)
Citation Context ...Previous Work Sorting, permutation and related problems in EM have been extensively studied [1, 12, 27, 26, 20]. I/O-optimal approaches to many computational geometry problems [17] and graph problems =-=[11]-=- have also been described. Data structures [3, 5, 22] and a number of applications [4] have been examined in this context. Some implementation work has also been done [24, 25, 10]. A recent survey is ... |

21 | Bsplike external-memory computation
- Sibeyn, Kaufmann
- 1997
(Show Context)
Citation Context ...n I/O-optimal way. However, most problems do not have a geometrically decreasing size. Examples include problems like sorting, matrix multiplication, convex hull and Voronoi diagram construction; see =-=[21]-=-. Concurrent to the work presented in this paper, Sibeyn and Kaufmann [21] have developed a technique for simulating 1-optimal BSP algorithms to produce efficient EM algorithms. Their work is presente... |

20 | Fully dynamic search trees for an extension of the bsp model - Baumker, Dittrich - 1996 |

18 | Dynamic and I/O-efficient algorithms for computational geometry and graph problems: theoretical and experimental results
- Chiang
- 1995
(Show Context)
Citation Context ...lems [17] and graph problems [11] have also been described. Data structures [3, 5, 22] and a number of applications [4] have been examined in this context. Some implementation work has also been done =-=[24, 25, 10]-=-. A recent survey is included in [5]. The classical EM model is described in [1]. More complex models have been proposed as well, incorporating a hierarchy of memory layers rather than the two-level m... |

18 | Supporting I/O-efficient scientific computation in TPIE - Vengroff, Vitter - 1995 |

17 |
TPIE User Manual and Reference
- Vengroff
- 1995
(Show Context)
Citation Context ...lems [17] and graph problems [11] have also been described. Data structures [3, 5, 22] and a number of applications [4] have been examined in this context. Some implementation work has also been done =-=[24, 25, 10]-=-. A recent survey is included in [5]. The classical EM model is described in [1]. More complex models have been proposed as well, incorporating a hierarchy of memory layers rather than the two-level m... |

15 | Topology B-trees and their applications - Callahan, Goodrich, et al. - 1995 |

14 | Communication efficient parallel sorting - Goodrich - 1996 |

13 | On the parallel-decomposability of geometric problems - Atallah, Tsay - 1992 |

9 | Parallel Algorithms for Image Processing: Practical Algorithms with Experiments
- BĂ¤umer, Dittrich
- 1996
(Show Context)
Citation Context ...ters instead of reinventing new algorithms for EM is intriguing. In this paper we exploit a natural correspondence between external memory algorithms and BSP-like parallel models such as BSP[23], BSP*=-=[7, 8, 6]-=- and CGM[13, 14, 15]. We provide simulation techniques that map BSP-like algorithms to EM algorithms, and we further show how, using a randomized approach, an EM machine can take full advantage of par... |

9 | The deterministic complexity of parallel multisearch
- Baumker, Dittrich, et al.
- 1996
(Show Context)
Citation Context ...ters instead of reinventing new algorithms for EM is intriguing. In this paper we exploit a natural correspondence between external memory algorithms and BSP-like parallel models such as BSP[23], BSP*=-=[7, 8, 6]-=- and CGM[13, 14, 15]. We provide simulation techniques that map BSP-like algorithms to EM algorithms, and we further show how, using a randomized approach, an EM machine can take full advantage of par... |

9 | Coarse grained parallel next element search - Chan, Dehne, et al. - 1997 |

8 | Report of the working group on storage I/O issues in large-scale computing - Gibson, Vitter, et al. - 1996 |

7 |
Large scale sorting in uniform memory hierarchies
- Vitter, Nodine
- 1993
(Show Context)
Citation Context |