In this paper, we describe an efficient image-based approach to computing and shading visual hulls from silhouette image data. Our algorithm takes advantage of epipolar geometry and incremental computation to achieve a constant rendering cost per rendered pixel. It does not suffer from the computation complexity, limited resolution, or quantization artifacts of previous volumetric approaches. We demonstrate the use of this algorithm in a real-time virtualized reality application running off a small number of video streams.

In this paper, we survey the techniques for image-based rendering. Unlike traditional 3D computer graphics in which 3D geometry of the scene is known, image-based rendering techniques render novel views directly from input images. Previous image-based rendering techniques can be classified into three categories according to how much geometric information is used: rendering without geometry, rendering with implicit geometry (i.e., correspondence), and rendering with explicit geometry (either with approximate or accurate geometry). We discuss the characteristics of these categories and their representative methods. The continuum between images and geometry used in image-based rendering techniques suggests that image-based rendering with traditional 3D graphics can be united in a joint image and geometry space. Keywords: Image-based rendering, survey. 1

We present an extension to texture mapping that supports the representation of 3-D surface details and view motion parallax. The results are correct for viewpoints that are static or moving, far away or nearby. Our approach is very simple: a relief texture (texture extended with an orthogonal displacement per texel) is mapped onto a polygon using a two-step process: First, it is converted into an ordinary texture using a surprisingly simple 1-D forward transform. The resulting texture is then mapped onto the polygon using standard texture mapping. The 1-D warping functions work in texture coordinates to handle the parallax and visibility changes that result from the 3-D shape of the displacement surface. The subsequent texture-mapping operation handles the transformation from texture to screen coordinates. CR Categories and Subject Descriptors: I.3.3 [Computer Graphics]: Picture/Image Generation I.3.6 [Computer Graphics]: Methodologies and Techniques; I.3.7 [Computer Graphics]: Three-...

We present an image-based modeling and editing system that takes a single photo as input. We represent a scene as a layered collection of depth images, where each pixel encodes both color and depth. Starting from an input image, we employ a suite of user-assisted techniques, based on a painting metaphor, to assign depths and extract layers. We introduce two specific editing operations. The first, a "clone brushing tool," permits the distortion-free copying of parts of a picture, by using a parameterization optimization technique. The second, a "texture-illuminance decoupling filter," discounts the effect of illumination on uniformly textured areas, by decoupling large- and small-scale features via bilateral filtering. Our system enables editing from different viewpoints, extracting and grouping of image-based objects, and modifying the shape, color, and illumination of these objects.

The 3D Image warping algorithm by McMillan and Bishop uses regular single-layered depth images (which are called reference images) as the initial input. Tears or gaps may appear in the output when the previously occluded areas are exposed in the new viewing position. Many reference images that are taken from different positions can be used to attenuate the occlusion problems but the rendering cost increases with the number of reference images. Also, combining the multiple reference images and eliminating the redundant information is a non-trivial problem. Recently, the Layered Depth Image (LDI) was proposed by Shade et al. to merge multiple reference images under a single center of projection. It tackles the occlusion problems by keeping multiple depth pixels per pixel location, while still maintaining the simplicity of warping a single reference image. However, it does not consider the issue of sampling rate. We present the LDI tree, which combines a hierarchical space partition scheme with the concept of LDI. It preserves the sampling rates of the reference images by adaptively selecting an LDI in the LDI tree for each pixel. While rendering from the LDI tree, we only have to traverse the LDI tree to the levels that are comparable to the sampling rate of the output image. We also present the progressive refinement feature and a "gap filling" algorithm by pre-filtering the LDI tree. We show that the amount of memory required is of the same order as the 2D reference images. This also bounds the complexity of rendering time to be less than directly rendering from all reference images. 1.

We present two efficient image-based approaches for computation and display of high-quality soft shadows from area light sources. Our methods are related to shadow maps and provide the associated benefits. The computation time and memory requirements for adding soft shadows to an image depend on image size and the number of lights, not geometric scene complexity. We also show that because area light sources are localized in space, soft shadow computations are particularly well suited to image-based rendering techniques. Our first approach---layered attenuation maps--- achieves interactive rendering rates, but limits sampling flexibility, while our second method---coherence-based raytracing of depth images---is not interactive, but removes the limitations on sampling and yields high quality images at a fraction of the cost of conventional raytracers. Combining the two algorithms allows for rapid previewing followed by efficient high-quality rendering.

We present a compact, image-based representation for threedimensional objects with complex shapes that can be rendered with correct perspective from arbitrary viewpoints using a listpriority algorithm. Objects are represented by six layered depth images sharing a single center of projection. They can be scaled, and freely translated and rotated, being used as primitives to construct more complex scenes. We also present a new listpriority algorithm for rendering such scenes and a back face culling strategy for a class of image-based objects. We demonstrate these concepts by constructing image-based representations from both synthetic and real objects, and rendering them at interactive rates on a PC. Due to their minimum storage requirements and rendering simplicity, image-based objects can find potential uses in games, virtual museum applications, and web catalogs. CR Categories and Subject Descriptors: I.3.3 [Computer Graphics]: Picture/Image Generation -- Display Algorithms; I.3.7 [...

Abstract. Bounding boxes are commonly used in computer graphics and other fields to improve the performance of algorithms that should process only the intersecting objects. A bounding-box-based heuristic avoids unnecessary intersection processing by eliminating the pairs whose bounding boxes are disjoint. Empirical evidence suggests that the heuristic works well in many practical applications, although its worst-case performance can be bad for certain pathological inputs. What is a pathological input, however, is not well understood, and consequently there is no guarantee that the heuristic will always work well in a specific application. In this paper, we analyze the performance of bounding box heuristic in terms of two natural shape parameters, aspect ratio and scale factor. These parameters can be used to realistically measure the degree to which the objects are pathologically shaped. We derive tight worst-case bounds on the performance for bounding box heuristic. One of the significant contributions of our paper is that we only require that objects be well shaped on average. Somewhat surprisingly, the bounds are significantly different from the case when all objects are well shaped.

: One of the most important goals of interactive computer graphics is to allow a user to freely walk around a virtual recreation of a real environment that looks as real as the world around us. But hand-modeling such a virtual environment is inherently limited and acquiring the scene model using devices also presents challenges. Interactively rendering such a detailed model is beyond the limits of current graphics hardware, but image-based approaches can significantly improve the status quo. We present an end-to-end system for acquiring highly detailed scans of large real world spaces, consisting of forty to eighty million range and color samples, using a digital camera and laser rangefinder. We explain successful techniques to represent these large data sets as image-based models and present contributions to image-based rendering that allow these models to be rendered in real time on existing graphics hardware without sacrificing the high resolution at which the data sets...

The images generated by real-time 3D graphics systems exhibit enormous frame-to-frame coherence, which is not exploited by the conventional graphics pipeline. I exploit this coherence by decoupling image rendering from image display. My system renders every Nth frame in the conventional manner, and generates the in-between frames with an image warper. The image warper modifies a rendered image so that it is approximately correct for a new viewpoint and view direction. My image warper uses McMillan's 3D image warp. Unlike perspective image warps, the 3D image warp can correct for changes in viewpoint, even for objects at different depths. As a result, my system does not require the application programmer to segment the scene into different depth layers, as is required by systems that use a perspective image warp. I attack thr...