## Inference of Segmented Overlapping Surfaces from Binocular Stereo (2002)

Venue: | IEEE Trans. on PAMI |

Citations: | 13 - 3 self |

### BibTeX

@ARTICLE{Lee02inferenceof,

author = {Mi-suen Lee and Ieee Computer Society and Geâ Rard Medioni and Senior Member and Student Member},

title = {Inference of Segmented Overlapping Surfaces from Binocular Stereo},

journal = {IEEE Trans. on PAMI},

year = {2002},

volume = {24},

pages = {824--837}

}

### Years of Citing Articles

### OpenURL

### Abstract

AbstractÐWe present an integrated approach to the derivation of scene descriptions from a pair of stereo images, where the steps of feature correspondence and surface reconstruction are addressed within the same framework. Special attention is given to the development of a methodology with general applicability. In order to handle the issues of noise, lack of image features, surface discontinuities, and regions visible in one image only, we adopt a tensor representation for the data and introduce a robust computational technique called tensor voting for information propagation. The key contributions of this paper are twofold: First, we introduce ªsaliencyº instead of correlation scores as the criterion to determine the correctness of matches and the integration of feature matching and structure extraction. Second, our tensor representation and voting as a tool enables us to perform the complex computations associated with the formulation of the stereo problem in three dimensions at a reasonable computational cost. We illustrate the steps on an example, then provide results on both random dot stereograms and real stereo pairs, all processed with the same parameter set. Index TermsÐBinocular stereo, tensor voting, perceptual grouping, surface inference. 1

### Citations

1154 |
Vision: A computational investigation into the human representation and processing of visual information
- Marr
- 1982
(Show Context)
Citation Context ...n 3D space, or a set of visible surfaces. The general paradigm they adhere to was established by Marr and Poggio [23], the uniqueness and continuity constraints were introduced a little later by Marr =-=[24]-=-, the ordering constraint by Yuille and Poggio [39], and the gradient limit constraint by Burt and Julesz [5]. For early works on stereo, we refer readers to the reviews by Barnard and Fischler [1] an... |

337 |
Signal Processing for Computer Vision
- Granlund, Knutsson
- 1995
(Show Context)
Citation Context ...he ellipsoid. A.2 Tensor Decomposition As a result of the voting procedure, we produce arbitrary second-order, symmetric tensors; therefore, we need to handle any generic tensor. The spectrum theorem =-=[12]-=- states that any tensor can be expressed as a linear combination of three basis tensors, i.e., S ˆ… 1 2†^e^e1^e^e1 T ‡… 2 3†…^e^e1^e^e1 T ‡ ^e^e2^e^e2 T †‡ 3…^e^e1^e^e1 T ‡ ^e^e2^e^e2 T ‡ ^e^e3^e^e3 T... |

230 |
Marching cubes: a high resolution 3d surface reconstruction algorithm
- Lorensen, Cline
- 1987
(Show Context)
Citation Context ...ves, and curve junctions should be the desired output. Combining tensor voting with a marching process, the proposed framework extracts surfaces in the form of a triangulated mesh in a way similar to =-=[22]-=- and [35]. 3.2 Constraints Taking into consideration that the problem at hand is illposed, several constraints that should be imposed on the solution have been proposed. Besides the epipolar constrain... |

212 | A maximum likelihood stereo algorithm
- Cox, Hingorani, et al.
- 1996
(Show Context)
Citation Context ...also been attempted. Maximum a posteriori estimators were proposed by Geiger et al. [11], Belhumeur and Mumford [3], and Belhumeur [2], while a maximum-likelihood estimator was proposed by Cox et al. =-=[8]-=- and an approach based on Markov random fields by Boykov et al. [4]. These algorithms either demand explicit modeling of the surfaces, the discontinuities, and noise or assume that features and noise ... |

193 | A space-sweep approach to true multi-image matching
- Collins
- 1996
(Show Context)
Citation Context ...ty information is reprojected to the images for verification. Finally, there is a different class of algorithms that operate in three-dimensional space. It includes the algorithms proposed by Collins =-=[7]-=- and Seitz and Dyer [32]. Their characteristic is the use of the projection rays from the images to the scene and their relations in space to infer possible locations of world points. A combination of... |

149 |
A Computational Framework for Segmentation and Grouping
- Medioni, Lee, et al.
- 2000
(Show Context)
Citation Context ...e of our framework is tensor representation and tensor voting. The use of tensors in the field of signal processing and computer vision was introduced by Knutsson [19] and Westin [38]. Medioni et al. =-=[25]-=- formulated a methodology for inferring perceptual structures from sparse noisy data under which tensors communicate with their neighborhood sending and receiving information via a voting process. Thi... |

113 |
A bayesian-approach to binocular stereopsis
- Belhumeur
- 1996
(Show Context)
Citation Context ...formulation reduces computational cost. Stochastic formulations have also been attempted. Maximum a posteriori estimators were proposed by Geiger et al. [11], Belhumeur and Mumford [3], and Belhumeur =-=[2]-=-, while a maximum-likelihood estimator was proposed by Cox et al. [8] and an approach based on Markov random fields by Boykov et al. [4]. These algorithms either demand explicit modeling of the surfac... |

108 |
A guided Tour of Computer Vision
- Nalwa
- 1993
(Show Context)
Citation Context ...ramework to handle multiple layers. Fig. 8 shows another example of a random dot stereogram depicting a cross floating over two overlapping planes. This a classical example, adapted from Nalwa's book =-=[26]-=-. It is also properly handled without layer initialization or alterations of the parameters. Note that the boundaries of the cross are accurate and we even detect the corners explicitly. The random do... |

80 | A Bayesian treatment of the stereo correspondence problem using half-occluded regions
- Belhumeur, Mumford
- 1992
(Show Context)
Citation Context ...respectively. This formulation reduces computational cost. Stochastic formulations have also been attempted. Maximum a posteriori estimators were proposed by Geiger et al. [11], Belhumeur and Mumford =-=[3]-=-, and Belhumeur [2], while a maximum-likelihood estimator was proposed by Cox et al. [8] and an approach based on Markov random fields by Boykov et al. [4]. These algorithms either demand explicit mod... |

80 | Surfaces from stereo: Integrating feature matching, disparity estimation, and contour detection
- Hoff, Ahuja
- 1989
(Show Context)
Citation Context ...ramework for establishing correct correspondences and reconstructing the surfaces inferred from these correspondences. The integration of these two phases was first proposed by Hoff and Ahuja in 1989 =-=[14]-=- and was also used by Szeliski and Golland in 1998 [34]. The novelty of our approach stems from the use of a robust technique, tensor voting, that allows discontinuities and outliers to be handled pro... |

66 |
From multiple stereo views to multiple 3D surfaces
- Fua
- 1997
(Show Context)
Citation Context ...akes an approach based on the optimization of a cost function. Some of the most representative work in this category was published by Olsen [27], Okutomi and Kanade [28], Robert and Deriche [29], Fua =-=[10]-=-, Wei et al. [37]. The main challenge with these methods is the selection of an objective function that enforces the smoothness constraint and, at the same time, handles discontinuities correctly and ... |

59 | A Tensor Framework for Multidimensional Signal Processing
- Westin
- 1994
(Show Context)
Citation Context ...ON AND VOTING The core of our framework is tensor representation and tensor voting. The use of tensors in the field of signal processing and computer vision was introduced by Knutsson [19] and Westin =-=[38]-=-. Medioni et al. [25] formulated a methodology for inferring perceptual structures from sparse noisy data under which tensors communicate with their neighborhood sending and receiving information via ... |

54 |
A disparity gradient limit for binocular fusion
- Burt, Julesz
- 1980
(Show Context)
Citation Context ... occur even in the absence of monocular information [17]. An epitome of Julesz's work by the author himself can be found in [18]. The works of Marr and Poggio in 1979 [23] and Burt and Julesz in 1980 =-=[5]-=- are first attempts to define the problem and its fundamental constraints. Since then, progress has been made but the complete stereo problem remains unsolved. The derivation of scene descriptions fro... |

46 |
Binocular depth perception of computer-generated patterns
- JULESZ
- 1960
(Show Context)
Citation Context ... Four decades ago, Julesz shed new light on binocular stereo vision, introducing random dot stereograms, and demonstrating that depth perception can occur even in the absence of monocular information =-=[17]-=-. An epitome of Julesz's work by the author himself can be found in [18]. The works of Marr and Poggio in 1979 [23] and Burt and Julesz in 1980 [5] are first attempts to define the problem and its fun... |

42 |
A Generalized Ordering Constraint for Stereo Correspondence
- Yuille, Poggio
- 1984
(Show Context)
Citation Context ...al paradigm they adhere to was established by Marr and Poggio [23], the uniqueness and continuity constraints were introduced a little later by Marr [24], the ordering constraint by Yuille and Poggio =-=[39]-=-, and the gradient limit constraint by Burt and Julesz [5]. For early works on stereo, we refer readers to the reviews by Barnard and Fischler [1] and Dhond and Aggarwal [9]. This section discusses mo... |

39 | Dense depth map reconstruction: A minimization and regularization approach which preserves discontinuities
- Robert, Deriche
- 1996
(Show Context)
Citation Context ...gorithms takes an approach based on the optimization of a cost function. Some of the most representative work in this category was published by Olsen [27], Okutomi and Kanade [28], Robert and Deriche =-=[29]-=-, Fua [10], Wei et al. [37]. The main challenge with these methods is the selection of an objective function that enforces the smoothness constraint and, at the same time, handles discontinuities corr... |

38 |
Dialogues on perception
- Julesz
- 1995
(Show Context)
Citation Context ...roducing random dot stereograms, and demonstrating that depth perception can occur even in the absence of monocular information [17]. An epitome of Julesz's work by the author himself can be found in =-=[18]-=-. The works of Marr and Poggio in 1979 [23] and Burt and Julesz in 1980 [5] are first attempts to define the problem and its fundamental constraints. Since then, progress has been made but the complet... |

37 | A volumetric stereo matching method: Application to imagebased modeling
- Chen, Medioni
- 1999
(Show Context)
Citation Context ...and their relations in space to infer possible locations of world points. A combination of the latter techniques that is related to our approach of the stereo problem was proposed by Chen and Medioni =-=[6]-=-. The processing is performed in the three-dimensional disparity space, where correct matches are identified starting from locations where unique maxima of the cross-correlation of intensity values oc... |

32 | Inference of Integrated Surface, Curve, and Junction Descriptions From Sparse 3D Data
- Tang, Medioni
- 1998
(Show Context)
Citation Context ...curve junctions should be the desired output. Combining tensor voting with a marching process, the proposed framework extracts surfaces in the form of a triangulated mesh in a way similar to [22] and =-=[35]-=-. 3.2 Constraints Taking into consideration that the problem at hand is illposed, several constraints that should be imposed on the solution have been proposed. Besides the epipolar constraint, the mo... |

23 | Inferring Segmented Surface Description from Stereo Data
- Lee, Medioni
- 1998
(Show Context)
Citation Context ... NO. 6, JUNE 2002 Fig. 2. The ªwedding cakeº interpretation of overlapping planes. (a) Input images. (b) ªWedding cakeº output. and outliers, and we perform all the processing in 3D instead of 2 1/2D =-=[20]-=-. The 2 1/2D sketch was introduced by Marr [24] and is very popular among stereo researchers. The weakness of this representation lies in the fact that it is view-centered and viewpoint dependent elem... |

18 |
Intensity- and gradient-based stereo matching using hierarchical Gaussian basis functions
- Wei, Brauer, et al.
- 1998
(Show Context)
Citation Context ...based on the optimization of a cost function. Some of the most representative work in this category was published by Olsen [27], Okutomi and Kanade [28], Robert and Deriche [29], Fua [10], Wei et al. =-=[37]-=-. The main challenge with these methods is the selection of an objective function that enforces the smoothness constraint and, at the same time, handles discontinuities correctly and also degrades gra... |

13 |
A Robust Approach to Image Matching: Recovery of the Epipolar Geometry
- Zhang, Deriche, et al.
- 1994
(Show Context)
Citation Context ... Feature Correspondence The algorithm accepts as input a pair of stereo images of a static scene. If calibration information is not available, we use a method such as the one proposed by Zhang et al. =-=[40]-=- to obtain the epipolar geometry. In the remainder of this paper, we will assume, without loss of generality, that the images have already been rectified. The first preprocessing step is the extractio... |

7 |
I.J.: A maximum-flow formulation of the N-camera correspondence problem
- Roy, Cox
- 1998
(Show Context)
Citation Context ...rrectly and also degrades gracefully in the presence of noise. The proper treatment of all the issues that arise in binocular stereo usually results in very high-computational complexity. Roy and Cox =-=[30]-=- and, later, Ishikawa and Geiger [16] formulated the stereo problem as a maximum flow problem where the solution is a minimum cut of an undirected or a directed graph, respectively. This formulation r... |

4 |
ªMarkov Random Fields with Efficient Approximations,º
- Boykov, Veksler, et al.
- 1998
(Show Context)
Citation Context ...by Geiger et al. [11], Belhumeur and Mumford [3], and Belhumeur [2], while a maximum-likelihood estimator was proposed by Cox et al. [8] and an approach based on Markov random fields by Boykov et al. =-=[4]-=-. These algorithms either demand explicit modeling of the surfaces, the discontinuities, and noise or assume that features and noise follow a normal distribution. These complex models may be suitable ... |

4 |
Inference of surfaces, curves and junctions from sparse, noisy 3-D data
- MEDIONI, GUY
- 1997
(Show Context)
Citation Context ...t is able to survive corruption of noise up to a few times the order of the inlier data. Corruption by noisy data five times larger in number than the correct data was shown not to be catastrophic in =-=[13]-=-. Even if similar corruption is unlikely in the case of binocular stereo, this example demonstrates the noise tolerance of the tensor voting framework. In fact, the only assumption we make about noise... |

4 |
ªPhotorealistic Scene Reconstruction by Voxel Coloring,º
- Seitz, Dyer
- 1997
(Show Context)
Citation Context ...jected to the images for verification. Finally, there is a different class of algorithms that operate in three-dimensional space. It includes the algorithms proposed by Collins [7] and Seitz and Dyer =-=[32]-=-. Their characteristic is the use of the projection rays from the images to the scene and their relations in space to infer possible locations of world points. A combination of the latter techniques t... |

3 |
ªOcclusions and Binocular Stereo,º Int'l
- Geiger, Ladendorf, et al.
- 1995
(Show Context)
Citation Context ...rected or a directed graph, respectively. This formulation reduces computational cost. Stochastic formulations have also been attempted. Maximum a posteriori estimators were proposed by Geiger et al. =-=[11]-=-, Belhumeur and Mumford [3], and Belhumeur [2], while a maximum-likelihood estimator was proposed by Cox et al. [8] and an approach based on Markov random fields by Boykov et al. [4]. These algorithms... |

3 |
ªA Multiple-Baseline Stereo,º
- Okutomi, Kanade
- 1993
(Show Context)
Citation Context ...ficant class of stereo algorithms takes an approach based on the optimization of a cost function. Some of the most representative work in this category was published by Olsen [27], Okutomi and Kanade =-=[28]-=-, Robert and Deriche [29], Fua [10], Wei et al. [37]. The main challenge with these methods is the selection of an objective function that enforces the smoothness constraint and, at the same time, han... |

3 |
On Occluding Contour Artifacts
- Sara, Bajcsy
- 1997
(Show Context)
Citation Context ...tions. The shortcoming of MINPRAN lies in the need to know the type and number of surfaces we try to extract a priori and in its computational complexity for nonplanar surface models. Sara and Bajcsy =-=[31]-=-, after making a key observation on the cause of the shift of occluding boundaries in disparity maps, propose robust matching operators that can handle considerable amounts of noise and occlusion. The... |

2 |
ªMethods and Means for Recognising Complex
- Hough
- 1962
(Show Context)
Citation Context ...three basis components. 4.2 Tensor Voting The strength of our approach resides in the way data points communicate with each other through tensor voting. It is a process similar to the Hough transform =-=[15]-=- in the sense that we let the solution emerge from the data after measurements of compatibility among data items. It differs in the fact that there is no need to specify beforehand the parametric conf... |

2 |
ªRepresenting Local Structure Using Tensors,º
- Knutsson
- 1989
(Show Context)
Citation Context ...SOR REPRESENTATION AND VOTING The core of our framework is tensor representation and tensor voting. The use of tensors in the field of signal processing and computer vision was introduced by Knutsson =-=[19]-=- and Westin [38]. Medioni et al. [25] formulated a methodology for inferring perceptual structures from sparse noisy data under which tensors communicate with their neighborhood sending and receiving ... |

2 |
ªMINPRAN: A New Robust Estimator for Computer Vision,º
- Stewart
- 1995
(Show Context)
Citation Context ...mber of scenes which do not deviate much from their essential assumptions but may fail on other scenes. Given the nature of the problem, robust techniques seem a good choice. Stewart proposed MINPRAN =-=[33]-=-, a robust estimator that can infer surfaces, compatible with predetermined models, even in severe noise conditions. The shortcoming of MINPRAN lies in the need to know the type and number of surfaces... |

2 |
Curvature-Augmented Tensorial Framework for Integrated Shape Inference from Noisy, 3-D Data
- Tang, Medioni
- 2001
(Show Context)
Citation Context ...n that can be extracted from the data should be used to guide surface and curve extraction. So far, we have conducted experiments on curvature extraction from synthetic data with satisfactory results =-=[36]-=-. Finally, the efficiency of the extraction process is an area that can be considerably improved. The complexity of the Marching Cubes algorithm depends on the voxel size rather than the complexity of... |

1 |
ªStructure from StereoÐA Review,º
- Dhond, Aggarwal
- 1989
(Show Context)
Citation Context ...aint by Yuille and Poggio [39], and the gradient limit constraint by Burt and Julesz [5]. For early works on stereo, we refer readers to the reviews by Barnard and Fischler [1] and Dhond and Aggarwal =-=[9]-=-. This section discusses more recent publications in the field. A significant class of stereo algorithms takes an approach based on the optimization of a cost function. Some of the most representative... |

1 |
ªGrouping
- Lee, Medioni
- 1999
(Show Context)
Citation Context ...de the true surface boundary detection. Since edge detectors often fail to extract the entire occluding boundary, the detected edgels need to be linked. This is a curve inference problem addressed in =-=[21]-=-. Tensor voting and analysis of the results with respect to curve saliency allows the extraction of complete boundaries. The last step is the identification of which of the two segments of the surface... |

1 |
ªA Theory of Human Stereo Vision,º Proc
- Marr, Poggio
- 1979
(Show Context)
Citation Context ...trating that depth perception can occur even in the absence of monocular information [17]. An epitome of Julesz's work by the author himself can be found in [18]. The works of Marr and Poggio in 1979 =-=[23]-=- and Burt and Julesz in 1980 [5] are first attempts to define the problem and its fundamental constraints. Since then, progress has been made but the complete stereo problem remains unsolved. The deri... |

1 |
ªStereo Correspondence by Surface Reconstruction,º
- Olsen
- 1990
(Show Context)
Citation Context ...ons in the field. A significant class of stereo algorithms takes an approach based on the optimization of a cost function. Some of the most representative work in this category was published by Olsen =-=[27]-=-, Okutomi and Kanade [28], Robert and Deriche [29], Fua [10], Wei et al. [37]. The main challenge with these methods is the selection of an objective function that enforces the smoothness constraint a... |

1 |
ªStereo Matching with Transparency and Matting,º Int'l
- Szeliski, Golland
- 1999
(Show Context)
Citation Context ...econstructing the surfaces inferred from these correspondences. The integration of these two phases was first proposed by Hoff and Ahuja in 1989 [14] and was also used by Szeliski and Golland in 1998 =-=[34]-=-. The novelty of our approach stems from the use of a robust technique, tensor voting, that allows discontinuities and outliers to be handled properly when inferring surfaces and regions. As demonstra... |