## Out-of-core algorithms for scientific visualization and computer graphics (2002)

### Cached

### Download Links

Venue: | In Visualization’02 Course Notes |

Citations: | 47 - 11 self |

### BibTeX

@INPROCEEDINGS{Silva02out-of-corealgorithms,

author = {Cláudio Silva and Yi-jen Chiang and Wagner Corrêa and Jihad El-sana and Peter Lindstrom},

title = {Out-of-core algorithms for scientific visualization and computer graphics},

booktitle = {In Visualization’02 Course Notes},

year = {2002}

}

### Years of Citing Articles

### OpenURL

### Abstract

Recently, several external memory techniques have been developed for a wide variety of graphics and visualization problems, including surface simplification, volume rendering, isosurface generation, ray tracing, surface reconstruction, and so on. This work has had significant impact given that in recent years there has been a rapid increase in the raw size of datasets. Several technological trends are contributing to this, such as the development of high-resolution 3D scanners, and the need to visualize ASCI-size (Accelerated Strategic Computing Initiative) datasets. Another important push for this kind of technology is the growing speed gap between main memory and caches, which penalizes algorithms that do not optimize for coherence of access. Because of these reasons, much research in computer graphics focuses on developing out-of-core (and often cache-friendly) techniques. This paper surveys fundamental issues, current problems, and unresolved questions, and aims to provide graphics researchers and professionals with an effective knowledge of current techniques, as well as the foundation to develop novel techniques on their own. Keywords: Out-of-core algorithms, scientific visualization, computer graphics, interactive rendering, vol-ume rendering, surface simplification.

### Citations

2410 | R-trees: a dynamic index structure for spatial searching
- Guttman
- 1984
(Show Context)
Citation Context ...eaded architecture with a set of streamline objects, three scheduling queue (wait, ready, and finished), free memory pool and a table of loaded octants. Leutenegger and Ma [59] propose to use R-trees =-=[48]-=- to optimize searching operations on large unstructured datasets. They arge that octrees (such as those used by Ueng et al. [82]) are not suited for storing unstructured data because of imbalance in t... |

1228 |
Multidimensional binary search trees used for associative searching
- Bentley
- 1975
(Show Context)
Citation Context ...ng regular grids is a much simpler task, and can be easily carried out by a greatly simplified version of the meta-cell technique). The resulting partition is similar to the one induced by a k-d-tree =-=[10]-=-, but there is no need to compute the multiple levels. The meta-cell technique has been used in Chiang et al. [20] for out-of-core isosurface extraction, in Farias and Silva [40] for out-of-core volum... |

1195 | Progressive meshes
- Hoppe
- 1996
(Show Context)
Citation Context ...to partition space and segment the surface. Both of these methods have the advantage of producing not a single static approximation but a multiresolution representation of the mesh—a progressive mesh =-=[50]-=-—which supports adaptive refinement, e.g., for view-dependent rendering. While being significantly slower (by about two orders of magnitude) and requiring more (although possibly controllable) memory ... |

1026 | P.S.: Surface Simplification Using Quadric Error Metrics - Garland, Heckbert - 1997 |

602 | Ubiquitous b-tree
- Comer
- 1979
(Show Context)
Citation Context ...rnal merge sort [3], out-of-core pointer de-referencing [14,17,18], and the meta-cell technique [20]. In Sec. 2.3, we review some important data structures for on-line computations, namely the B-tree =-=[9,26]-=- and B-tree-like data structures, and show a general method of converting a main-memory, binary-tree structure into a B-treelike data structure. In particular, we review the BBIO tree [19, 20], which ... |

600 | Decimation of triangle meshes
- Schroeder, Zarge, et al.
- 1992
(Show Context)
Citation Context ... triangles are specified as indices into a non-redundant list of vertices. There are many different ways to simplify surfaces. Popular coarsening operations for triangle meshes include vertex removal =-=[73]-=-, edge collapse [53], half-edge collapse [58], triangle collapse [49], vertex pair contraction [44], and vertex clustering [71]. While these operations vary in complexity and generality, they all have... |

563 |
The input/output complexity of sorting and related problems
- Aggarwal, Vitter
- 1988
(Show Context)
Citation Context ...illumination techniques. 2 EXTERNAL MEMORY ALGORITHMS The field of external-memory algorithms started quite early in the computer algorithms community, essentially by the paper of Aggarwal and Vitter =-=[3]-=- in 1988, which proposed the external-memory computational model (see below) that has been extensively used today. (External sorting algorithms were developed even earlier—though not explicitly descri... |

456 | QSplat: A multiresolution point rendering system for large meshes
- Rusinkiewicz, Levoy
- 2000
(Show Context)
Citation Context ...a Model At preprocessing time, an out-of-core rendering system builds a representation for the model on disk. The most common representations are bounding volume hierarchies (such as bounding spheres =-=[72]-=-) and space partitioning hierarchies (such as k-D trees [43, 80], BSP trees [85], and octrees [4, 23, 28, 82, 83]). Some approaches assume that this preprocessing step is performed on a machine with e... |

428 | The digital Michelangelo project: 3D scanning of large statues - Levoy, Rusinkiewicz, et al. - 2000 |

425 | View-Dependent Refinement of Progressive Meshes
- Hoppe
- 1997
(Show Context)
Citation Context ...ous criteria such as distance from the user’s viewpoint and screen space projection. Other systems use a multiresolution hierarchy, which encodes all the levels of detail, and is constructed off-line =-=[31,37,50,51,64,89]-=-. Then at real-time, the mesh adapts to the appropriate level of detail in a continuous, coherent manner. Overlapping Concurrent Tasks Another technique to improve the frame rates at runtime is to per... |

408 | Introduction to Algorithms, 2nd ed - Cormen, Leiserson, et al. |

390 |
The Art of Computer Programming (vol 3): Sorting and Searching
- Knuth
- 1973
(Show Context)
Citation Context ...elow) that has been extensively used today. (External sorting algorithms were developed even earlier—though not explicitly described and analyzed under the model of [3]; see the classic book of Knuth =-=[57]-=- in 1973.) Early work on external-memory algorithms, including Aggarwal and Vitter [3] and other follow-up results, concentrated largely on problems such as sorting, matrix multiplication, and FFT. La... |

370 | Mesh optimization
- Hoppe, DeRose, et al.
- 1993
(Show Context)
Citation Context ...fied as indices into a non-redundant list of vertices. There are many different ways to simplify surfaces. Popular coarsening operations for triangle meshes include vertex removal [73], edge collapse =-=[53]-=-, half-edge collapse [58], triangle collapse [49], vertex pair contraction [44], and vertex clustering [71]. While these operations vary in complexity and generality, they all have one thing in common... |

335 | External memory algorithms and data structures: Dealing with MASSIVE data - Vitter |

321 |
Hierarchical Geometric Models for Visible Surface Algorithms
- Clark
- 1976
(Show Context)
Citation Context ...is available, view-frustum culling can be optimized by recursively traversing the hierarchy from the root down to the leaves, and culling away entire subtrees that are outside the user’s view frustum =-=[24]-=-. Occlusion Culling Another technique to minimize the geometry that needs to be loaded from disk and sent to the graphics hardware is to cull away geometry that is occluded by other geometry. This is ... |

319 |
Multi-resolution 3D approximation for rendering complex scenes. Geometric Modeling
- Rossignac, Borrel
- 1993
(Show Context)
Citation Context ...Popular coarsening operations for triangle meshes include vertex removal [73], edge collapse [53], half-edge collapse [58], triangle collapse [49], vertex pair contraction [44], and vertex clustering =-=[71]-=-. While these operations vary in complexity and generality, they all have one thing in common, in that they partition the set of vertices from the input mesh by groupings(a) Spatial clustering partiti... |

298 | Visibility preprocessing for interactive walkthroughs
- Teller, Séquin
- 1991
(Show Context)
Citation Context ...em builds a representation for the model on disk. The most common representations are bounding volume hierarchies (such as bounding spheres [72]) and space partitioning hierarchies (such as k-D trees =-=[43, 80]-=-, BSP trees [85], and octrees [4, 23, 28, 82, 83]). Some approaches assume that this preprocessing step is performed on a machine with enough memory to hold the entire model [43,83]. Others make sure ... |

298 | Octrees for Faster Isosurface Generation
- Wilhelms, Gelder
- 1992
(Show Context)
Citation Context ...78] propose the T-BON (Temporal Branch-On-Need Octree) technique for fast extraction of isosurfaces of time-varying datasets. The preprocessing phase of their technique builds a branch-on-need octree =-=[87]-=- for each time step and stores it to disk in two parts. One part contains the structure of the octree and does not depend at all on the specific time step. Time-step specific data is saved separately ... |

234 | Smooth view-dependent level-of-detail control and its application to terrain rendering
- HOPPE
- 1998
(Show Context)
Citation Context ...ioning the surface over a uniform grid, or adaptive, e.g., by cutting the surface along feature lines. We begin by discussing uniform segmentation techniques. 4.2.1 Uniform Surface Segmentation Hoppe =-=[52]-=- described one of the first out-of-core simplification techniques based on surface segmentation for the special case of height fields. His method performs a 2D spatial division of a regularly gridded ... |

219 | P.: Visual navigation of large environments using textured clusters
- Maciel, Shirley
(Show Context)
Citation Context ...ing can be based on from-point visibility algorithms. Replacing Geometry with Imagery Image-based rendering techniques such as texture-mapped impostors can be used to accelerate the rendering process =-=[5,30,65,76]-=-. These texture-mapped impostors are generated either in a preprocessing step or at runtime (but not every frame). These techniques are suitable for outdoor models. Textured depth meshes [4, 30, 76] c... |

207 | Interactive rendering with coherent ray tracing - Wald, Slusallek, et al. |

185 | Simplification envelopes
- Cohen, Varshney, et al.
- 1996
(Show Context)
Citation Context ...il approaches have been used in the various walkthrough systems. Some systems use several static levels of detail, usually 3-10, which are constructed off-line using various simplification techniques =-=[25, 36, 44, 70, 81]-=-. Then at real-time, an appropriate level of detail is selected based on various criteria such as distance from the user’s viewpoint and screen space projection. Other systems use a multiresolution hi... |

178 | External-memory graph algorithms
- Chiang, Goodrich, et al.
- 1995
(Show Context)
Citation Context ...y on problems such as sorting, matrix multiplication, and FFT. Later, Goodrich et al. [47] developed I/O-efficient algorithms for a collection of problems in computational geometry, and Chiang et al. =-=[17]-=- gave I/O-efficient techniques for a wide range of computational graph problems. These papers also proposed some fundamental paradigms for external-memory geometric and graph algorithms. Since then, d... |

143 | G.: Fast and Memory Efficient Polygonal Simplification
- Lindstrom, Turk
- 1998
(Show Context)
Citation Context ...s used to accumulate both primal (distance-to-face) and dual (distance-to-vertex) quadric information. Based on this quadric information, a principal component analysiss(a) Original (b) Edge collapse =-=[63]-=- (c) Uniform clustering [61] Figure 5: Base of Happy Buddha model, simplified from 1.1 million to 16 thousand triangles. Notice the jagged edges and notches in (c) caused by aliasing from using a coar... |

141 | Portals and Mirrors: Simple, Fast Evaluation of Potential Visible Sets
- Luebke, Georges
- 1995
(Show Context)
Citation Context ...ous criteria such as distance from the user’s viewpoint and screen space projection. Other systems use a multiresolution hierarchy, which encodes all the levels of detail, and is constructed off-line =-=[31,37,50,51,64,89]-=-. Then at real-time, the mesh adapts to the appropriate level of detail in a continuous, coherent manner. Overlapping Concurrent Tasks Another technique to improve the frame rates at runtime is to per... |

139 | Out-of-core simplification of large polygonal models
- Lindstrom
- 2000
(Show Context)
Citation Context ...artition space, and we will discuss out-of-core methods for uniform clustering first. 4.1.1 Uniform Spatial Clustering To extend the clustering algorithm in [71] to the out-of-core setting, Lindstrom =-=[61]-=- proposed using Garland and Heckbert’s quadric error metric [44] to measure error. Lindstrom’s method, called OoCS, works by scanning a triangle soup representation of the mesh, one triangle at a time... |

123 | Fast Isocontouring for Improved Interactivity
- Bajaj, Pascucci, et al.
- 1996
(Show Context)
Citation Context ...given in the paper. Bajaj et al. [8] also proposes a technique for out-of-core isosurface extraction. Their technique is an extension of their seed-based technique for efficient isosurface extraction =-=[7]-=-. The seed-based approach works by saving a small set of seed cells, which can be used as starting points for an arbitrary isosurface by simple contour propagation. In the out-of-core adaptation, the ... |

123 | External-Memory Computational Geometry
- Goodrich, Tsay, et al.
- 1993
(Show Context)
Citation Context ... on external-memory algorithms, including Aggarwal and Vitter [3] and other follow-up results, concentrated largely on problems such as sorting, matrix multiplication, and FFT. Later, Goodrich et al. =-=[47]-=- developed I/O-efficient algorithms for a collection of problems in computational geometry, and Chiang et al. [17] gave I/O-efficient techniques for a wide range of computational graph problems. These... |

116 | Indexing for data models with constraints and classes
- Kanellakis, Ramaswamy, et al.
- 1993
(Show Context)
Citation Context ...r short) developed and used in [20], for out-of-core isosurface extraction. This scan and distribute paradigm enables them to perform preprocessing to build these trees (as well as the metablock tree =-=[54]-=-) in an I/O-optimal way; see Chiang and Silva [19] for a complete review of these data structures and techniques. 2.2.2 Out-of-Core Pointer De-Referencing Typical input datasets in scientific visualiz... |

112 | Speeding Up Isosurface Extraction Using Interval Trees
- Cignoni, Marino, et al.
- 1997
(Show Context)
Citation Context ...e (main-memory) binary interval tree [33]. As will be seen in Sec. 3, the process of finding active cells in isosurface extraction can be reduced to the following problem of interval stabbing queries =-=[22]-=-: given a set of N intervals in 1D, build a data structure so that for a given query point q we can efficiently report all intervals containing q. Such interval stabbing queries can be optimally solve... |

105 | Rendering complex scenes with memorycoherent ray tracing
- Pharr, Kolb, et al.
- 1997
(Show Context)
Citation Context ...adiosity computation for a given patch only requires information about other parts of the model that can be seen from that patch, which often is only a small subset of the whole dataset. Pharr et al. =-=[68]-=- describe techniques for ray tracing very complex scenes. Their work is based on the caching and reordering computations. Their approach uses three different types of caches: ray, geometry, and textur... |

103 | Application-controlled demand paging for out-of-core visualization - Cox, Ellsworth - 1997 |

94 |
A fast volume rendering algorithm for time-varying fields using a time-space partitioning tree
- Shen, Chiang, et al.
- 1999
(Show Context)
Citation Context ... cell data) has been read. Then, a second pass is performed to actually compute the isosurface. Their technique also uses meta-cells (or data bricking) [20] to optimize the I/O transfers. Shen et al. =-=[75]-=- proposes a different encoding for time-varying datasets. Their TSP (Time-Space Partioning) tree encodes in a single data structure the changes from one time-step to another. Each node of the octree h... |

91 | Interactive out-of-core isosurface extraction
- Chiang, Silva, et al.
- 1998
(Show Context)
Citation Context ...in batched computations that are fundamental for out-of-core scientific visualization and graphics: external merge sort [3], out-of-core pointer de-referencing [14,17,18], and the meta-cell technique =-=[20]-=-. In Sec. 2.3, we review some important data structures for on-line computations, namely the B-tree [9,26] and B-tree-like data structures, and show a general method of converting a main-memory, binar... |

90 | Adaptive real-time level-of-detail-based rendering for polygonal models
- Xia, El-Sana, et al.
- 1997
(Show Context)
Citation Context ...ous criteria such as distance from the user’s viewpoint and screen space projection. Other systems use a multiresolution hierarchy, which encodes all the levels of detail, and is constructed off-line =-=[31,37,50,51,64,89]-=-. Then at real-time, the mesh adapts to the appropriate level of detail in a continuous, coherent manner. Overlapping Concurrent Tasks Another technique to improve the frame rates at runtime is to per... |

86 |
A data reduction scheme for triangulated surfaces
- Hamann
- 1994
(Show Context)
Citation Context ...ices. There are many different ways to simplify surfaces. Popular coarsening operations for triangle meshes include vertex removal [73], edge collapse [53], half-edge collapse [58], triangle collapse =-=[49]-=-, vertex pair contraction [44], and vertex clustering [71]. While these operations vary in complexity and generality, they all have one thing in common, in that they partition the set of vertices from... |

83 |
The Visible Human Project
- Ackerman
- 1998
(Show Context)
Citation Context ...le, all surface segmentation methods discussed above have input-sensitive disk requirements. Finally, certain types of data sets, such as isosurfaces from scientific simulations [66] and medical data =-=[2]-=-, can have a very complicated topological structure, resulting either from noise or intrinsic fine-scale features in the data. In such cases, simplification of the topology is not only desirable but i... |

78 | Interactive Distributed Ray Tracing of Highly Complex Models
- Wald, Slusallek, et al.
(Show Context)
Citation Context ...tation for the model on disk. The most common representations are bounding volume hierarchies (such as bounding spheres [72]) and space partitioning hierarchies (such as k-D trees [43, 80], BSP trees =-=[85]-=-, and octrees [4, 23, 28, 82, 83]). Some approaches assume that this preprocessing step is performed on a machine with enough memory to hold the entire model [43,83]. Others make sure that the preproc... |

75 | I/O optimal isosurface extraction
- Chiang, Silva
- 1997
(Show Context)
Citation Context ...In Sec. 2.2, we review three techniques in batched computations that are fundamental for out-of-core scientific visualization and graphics: external merge sort [3], out-of-core pointer de-referencing =-=[14,17,18]-=-, and the meta-cell technique [20]. In Sec. 2.3, we review some important data structures for on-line computations, namely the B-tree [9,26] and B-tree-like data structures, and show a general method ... |

72 | E.: Efficient Implementation of Multi-Triangulations
- FLORIANI, MAGILLO, et al.
- 1998
(Show Context)
Citation Context |

71 |
External memory management and simplification of huge meshes
- Cignoni, Montani, et al.
(Show Context)
Citation Context ...nt that patch boundaries be left intact, which necessitates additional passes to coarsen the patch seams. This requirement can be avoided using the OEMM mesh data structure proposed by Cignoni et al. =-=[23]-=-. As in [69], an octree subdivision of space is made. However, when processing the surface patch for a node in this tree, adjacent nodes are loaded as well, allowing edges to be collapsed across node ... |

71 | Efficient Impostor Manipulation for Real-Time Visualization of Urban Scenery
- Sillion, Drettakis, et al.
- 1997
(Show Context)
Citation Context ...ing can be based on from-point visibility algorithms. Replacing Geometry with Imagery Image-based rendering techniques such as texture-mapped impostors can be used to accelerate the rendering process =-=[5,30,65,76]-=-. These texture-mapped impostors are generated either in a preprocessing step or at runtime (but not every frame). These techniques are suitable for outdoor models. Textured depth meshes [4, 30, 76] c... |

67 | A.: Generalized view-dependent simplification
- El-Sana, Varshney
- 1999
(Show Context)
Citation Context ...ree of Chiang et al. [19, 20] to externalize the interval tree [33] for out-of-core isosurface extraction, and of the meta-block tree of ElSana and Chiang [34] to externalize the view-dependence tree =-=[37]-=- for external memory view-dependent simplification and rendering. We believe that this externalization method is general and powerful enough to be applicable to a wide range of other problems in out-o... |

64 | MMR: An interactive massive model rendering system using geometric and image-based acceleration
- Aliaga, Cohen, et al.
- 1999
(Show Context)
Citation Context ...el on disk. The most common representations are bounding volume hierarchies (such as bounding spheres [72]) and space partitioning hierarchies (such as k-D trees [43, 80], BSP trees [85], and octrees =-=[4, 23, 28, 82, 83]-=-). Some approaches assume that this preprocessing step is performed on a machine with enough memory to hold the entire model [43,83]. Others make sure that the preprocessing step can be performed on a... |

55 | Out-of-core streamline visualization on large unstructured meshes
- Ueng, Sikorski, et al.
- 1997
(Show Context)
Citation Context ...hermore, their experiments show that the same techniques are effective for remote visualization, since less data needs to be transmitted over the network and cached at the client machine. Ueng et al. =-=[82]-=- present a related approach. In their work Ueng et al. focussed on computing streamlines of large unstructured grids, and they use an approach somewhat similar to Cox and Ellsworth in that the idea is... |

54 |
A new approach to rectangle intersections
- Edelsbrunner
- 1983
(Show Context)
Citation Context ...converting a main-memory, binary-tree structure into a B-treelike data structure. In particular, we review the BBIO tree [19, 20], which is an external-memory version of the main-memory interval tree =-=[33]-=- and is essential for isosurface extraction, as a non-trivial example. 2.1 Computational Model In contrast to random-access main memory, disks have extremely long access times. In order to amortize th... |

54 | Efficient conservative visibility culling using the prioritized-layered projection algorithm
- Klosowski, Silva
(Show Context)
Citation Context ...nique to minimize the geometry that needs to be loaded from disk and sent to the graphics hardware is to cull away geometry that is occluded by other geometry. This is a hard problem to solve exactly =-=[56, 80, 85]-=-, but fast and accurate approximations exist [35, 55]. Level-of-Detail Management (or Contribution Culling) The amount of geometry that fits in main memory typically exceeds the interactive rendering ... |

53 | Navigating static environments using image-space simplification and morphing
- Darsa, Silva, et al.
- 1997
(Show Context)
Citation Context ...ing can be based on from-point visibility algorithms. Replacing Geometry with Imagery Image-based rendering techniques such as texture-mapped impostors can be used to accelerate the rendering process =-=[5,30,65,76]-=-. These texture-mapped impostors are generated either in a preprocessing step or at runtime (but not every frame). These techniques are suitable for outdoor models. Textured depth meshes [4, 30, 76] c... |

53 | A memory insensitive technique for large model simplication - Lindstrom, Silva - 2001 |

51 | Parallel Accelerated Isocontouring for Out-of-Core Visualization
- Bajaj, Pascucci, et al.
- 1999
(Show Context)
Citation Context ...thors have an I/O thread that reads the active cells from disk, and several isosurface computation threads. Experimental results on relatively small regular grids are given in the paper. Bajaj et al. =-=[8]-=- also proposes a technique for out-of-core isosurface extraction. Their technique is an extension of their seed-based technique for efficient isosurface extraction [7]. The seed-based approach works b... |