No feature-based vision system can work unless good features can be identified and tracked from frame to frame. Although tracking itself is by and large a solved problem, selecting features that can be tracked well and correspond to physical points in the world is still hard. We propose a feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world. These methods are based on a new tracking algorithm that extends previous Newton-Raphson style search methods to work under affine image transformations. We test performance with several simulations and experiments.

This paper presents a stereo algorithm for obtaining disparity maps with occlusion explicitly detected. To produce smooth and detailed disparity maps, two assumptions that were originally proposed by Marr and Poggio are adopted: uniqueness and continuity. That is, the disparity maps have a unique value per pixel and are continuous almost everywhere. These assumptions are enforced within a three-dimensional array of match values in disparity space. Each match value corresponds to a pixel in an image and a disparity relative to another image. An iterative algorithm updates the match values by diffusing support among neighboring values and inhibiting others along similar lines of sight. By applying the uniqueness assumption, occluded regions can be explicitly identified. To demonstrate the effectiveness of the algorithm we present the processing results from synthetic and real image pairs, including ones with ground-truth values for quantitative comparison with other method.

This paper formulates and solves a new variant of the stereo correspondence problem: simultaneously recovering the disparities, true colors, and opacities of visible surface elements. This problem arises in newer applications of stereo reconstruction, such as view interpolation and the layering of real imagery with synthetic graphics for special effects and virtual studio applications. While this problem is intrinsically more difficult than traditional stereo correspondence, where only the disparities are being recovered, it provides a principled way of dealing with commonly occurring problems such as occlusions and the handling of mixed (foreground/background) pixels near depth discontinuities. It also provides a novel means for separating foreground and background objects (matting), without the use of a special blue screen. We formulate the problem as the recovery of colors and opacities in a generalized 3-D (x, y, d) disparity space, and solve the problem using a combination of initial evidence aggregation followed by iterative energy minimization.

This paper presents a feature tracker for long image sequences based on simultaneously estimating the motions and deformations of a collection of adjacent image patches. By sharing common corner nodes, the patches achieve greater stability than independent patch trackers. Modeling full bilinear deformations enables tracking in sequences which have large non-translational motions and/or foreshortening effects. We demonstrate the advantages of our technique with respect to previous algorithms using experimental results. Keywords: motion analysis, multiframe feature tracking, affine patches c flDigital Equipment Corporation 1995. All rights reserved. 1 The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213-3890 Contents i Contents 1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 2 Previous work : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 3 Spline-based image registration : : : : : : : :...

This paper presents a stereo algorithm for obtaining disparity maps with explicitly detected occlusion. To produce smooth and detailed disparity maps, two assumptions that were originally proposed by Marr and Poggio are adopted: uniqueness and continuity. That is, the disparity maps have unique values and are continuous almost everywhere. A volumetric approach is taken to utilize these assumptions. A 3D array of match likelihood values is constructed with each value corresponding to a pixel in an image and a disparity relative to another image. An iterative algorithm updates the match likelihood values by diffusing support among neighboring values and inhibiting others. After the values have converged, the region of occlusion is explicitly detected. To demonstrate the effectiveness of the algorithm we present the processing results from synthetic and real image pairs, with comparison to results by other methods. The resulting disparity maps are smooth and detailed with occlusions detec...