Abstract. Stereo matching is one of the most active research areas in computer vision. While a large number of algorithms for stereo correspondence have been developed, relatively little work has been done on characterizing their performance. In this paper, we present a taxonomy of dense, two-frame stereo methods. Our taxonomy is designed to assess the different components and design decisions made in individual stereo algorithms. Using this taxonomy, we compare existing stereo methods and present experiments evaluating the performance of many different variants. In order to establish a common software platform and a collection of data sets for easy evaluation, we have designed a stand-alone, flexible C++ implementation that enables the evaluation of individual components and that can easily be extended to include new algorithms. We have also produced several new multi-frame stereo data sets with ground truth and are making both the code and data sets available on the Web. Finally, we include a comparative evaluation of a large set of today’s best-performing stereo algorithms.

Abstract not found

Robotic systems are becoming smaller, lower power, and cheaper, enabling their application in areas not previously considered. This is true of vision systems as well. SRI's Small Vision Module (SVM) is a compact, inexpensive realtime device for computing dense stereo range images, which are a fundamental measurement supporting a wide range of computer vision applications. We describe hardware and software issues in the construction of the SVM, and survey implemented systems that use a similar area correlation algorithm on a variety of hardware. 1 Introduction Realtime stereo analysis, until recently, has been implemented in large custom hardware arrays (Kanade 1996, Matthies 1995). But computational power and algorithmic advances have made it possible to do such analysis on single processors. At the same time, increased density, speed and programmability of floating-point gate arrays (FPGAs) make custom hardware a viable alternative. In this paper, we discuss the implementation of ar...

> y)). Scale-independent scalar features of each flow, based on moments of the moving point weighted by |u|, |v|,or|(u, v)|, characterize the spatial distribution of the flow. We then analyze the periodic structure of these sequences of scalars. The scalar sequences for an image sequence have the same fundamental period but differ in phase, which is a phase feature for each signal. Some phase features are consistent for one person and show significant statistical variation among persons. We use the phase feature vectors to recognize individuals by the shape of their motion. As few as three features out of the full set of twelve lead to excellent discrimination. Keywords: action recognition, gait recognition, motion features, optic flow, motion energy, spatial frequency, analysis Recognizing People by Their Gait: The Shape of Moti

Our goal is to reconstruct both the shape and reflectance properties of surfaces from multiple images. We argue that an object-centered representation is most appropriate for this purpose because it naturally accommodates multiple sources of data, multiple images (including motion sequences of a rigid object), and self-occlusions. We then present a specific objectcentered reconstruction method and its implementation. The method begins with an initial estimate of surface shape provided, for example, by triangulating the result of conventional stereo. The surface shape and reflectance properties are then iteratively adjusted to minimize an objective function that combines information from multiple input images. The objective function is a weighted sum of stereo, shading, and smoothness components, where the weight varies over the surface. For example, the stereo component is weighted more strongly where the surface projects onto highly textured areas in the images, and less strongly othe...

This paper presents a stereo algorithm for obtaining disparity maps with occlusion explicitly detected. To produce smooth and detailed disparity maps, two assumptions that were originally proposed by Marr and Poggio are adopted: uniqueness and continuity. That is, the disparity maps have a unique value per pixel and are continuous almost everywhere. These assumptions are enforced within a three-dimensional array of match values in disparity space. Each match value corresponds to a pixel in an image and a disparity relative to another image. An iterative algorithm updates the match values by diffusing support among neighboring values and inhibiting others along similar lines of sight. By applying the uniqueness assumption, occluded regions can be explicitly identified. To demonstrate the effectiveness of the algorithm we present the processing results from synthetic and real image pairs, including ones with ground-truth values for quantitative comparison with other method.

This paper formulates and solves a new variant of the stereo correspondence problem: simultaneously recovering the disparities, true colors, and opacities of visible surface elements. This problem arises in newer applications of stereo reconstruction, such as view interpolation and the layering of real imagery with synthetic graphics for special effects and virtual studio applications. While this problem is intrinsically more difficult than traditional stereo correspondence, where only the disparities are being recovered, it provides a principled way of dealing with commonly occurring problems such as occlusions and the handling of mixed (foreground/background) pixels near depth discontinuities. It also provides a novel means for separating foreground and background objects (matting), without the use of a special blue screen. We formulate the problem as the recovery of colors and opacities in a generalized 3-D (x, y, d) disparity space, and solve the problem using a combination of initial evidence aggregation followed by iterative energy minimization.

A stereo vision system that is required to support high-level object based tasks in a tele-operated environment is described. Stereo vision is computationally expensive, due to having to find corresponding pixels. Correlation is a fast, standard way to solve the correspondence problem. This paper analyses the behaviour of correlation based stereo to find ways to improve its quality while maintaining its realtime suitability. Three methods are suggested. Two of them aim to improve the disparity image especially at depth discontinuities, while one targets the identification of possible errors in general. Results are given on real stereo images with ground truth. A comparison with five standard correlation methods shows that improvements of simple stereo correlation are possible in real-time on current computer hardware.

This paper describes a working stereo-vision-based mobile robot that can navigate and autonomously explore its environment safely while building occupancy grid maps of the environment. We present a method for reducing stereo vision disparity images to two dimensional map information. Stereo vision has several attributes that set it apart from other sensors more commonly used for occupancy grid mapping. We discuss these attributes, the errors that some of them create, and how to overcome them. This includes the idea of segmenting disparity images based on continuous disparity surfaces to reject "spikes" caused by stereo mismatches. Stereo vision processing and map updates are done at 5Hz and the robot moves at speeds of 150 cm/s.

A video-rate stereo machine has been developed at CMU with the capability of generating a dense range map, aligned with an intensity image, at the video rate. The target performance of the CMU video-rate stereo machine is: 1) multi image input of 6 cameras; 2) high throughput of more than 1.2 million points of depth measurement per second; 3) high frame rate of 30 frame/sec; 4) a dense depth map of 200 × 200 pixels; 5) disparity search range of 30 pixels; 6) high precision of up to 7 bits (with interpolation); 7) uncertainty estimation available for each pixel; and 8) low latency (time after imaging) of 17 msec. INTRODUCTION Stereo ranging, which uses correspondence between sets of two or more images for depth measurement, has many advantages. It is passive and it does not emit any radio or light energy. With appropriate imaging geometry, optics, and high-resolution cameras, stereo can produce a dense, precise range image of even distant scenes. Stereo performs sensor fusion inherent...