## Stereo matching using belief propagation (2003)

### Cached

### Download Links

Citations: | 271 - 3 self |

### BibTeX

@MISC{Sun03stereomatching,

author = {Jian Sun and Nan-ning Zheng and Heung-yeung Shum},

title = {Stereo matching using belief propagation},

year = {2003}

}

### Years of Citing Articles

### OpenURL

### Abstract

In this paper, we formulate the stereo matching problem as a Markov network and solve it using Bayesian belief propagation. The stereo Markov network consists of three coupled Markov random fields that model the following: a smooth field for depth/disparity, a line process for depth discontinuity, and a binary process for occlusion. After eliminating the line process and the binary process by introducing two robust functions, we apply the belief propagation algorithm to obtain the maximum a posteriori (MAP) estimation in the Markov network. Other low-level visual cues (e.g., image segmentation) can also be easily incorporated in our stereo model to obtain better stereo results. Experiments demonstrate that our methods are comparable to the state-of-the-art stereo algorithms for many test cases.

### Citations

7441 |
Probabilistic Reasoning in Intelligent Systems
- Pearl
- 1988
(Show Context)
Citation Context ... [20]. Unlike Scharstein & Szeliski, where a nonlinear diffusion algorithm is used, we address this MAP problem by Belief Propagation. Belief Propagation is an exact inference method proposed by Pearl=-=[19]-=- in the belief networkwithout loops. Loopy Belief Propagation is just Belief Propagation that ignores the existence of loops in the networks. It has been applied successfully to some vision [9] and co... |

4020 |
Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images
- Geman, Geman
- 1984
(Show Context)
Citation Context ...n [A, C] and [D, F] In general, Bayesian stereo matching can be formulated as a maximum a posteriori MRF (MAP-MRF) problem. There are several methods to solve the MAP-MRF problem: simulated annealing =-=[12]-=-, Mean-Field annealing [10], the Graduated Non-Convexity algorithm(GNC) [4], and Variational approximation [14]. Finding a solution by simulated annealing can often take an unacceptably long time alth... |

1509 |
Nonlinear total variation based noise removal algorithms
- Rudin, Osher, et al.
- 1992
(Show Context)
Citation Context ...iscontinuity process explicitly into defining two robust functions that model occlusion and discontinuity implicitly. In this paper, our robust functions are derived from the Total Variance(TV) model =-=[18]-=- with the potential function ρ(x) =|x| because of its discontinuity preserving property. We truncate this potential function as our robust function: |F (s, ds,I)| ρd(ds) =− ln((1 − ed) exp(− )+ed) ρp(... |

1477 | Fast approximate energy minimization via graph cuts
- Boykov, Veksler, et al.
- 2001
(Show Context)
Citation Context ...process. It reduces execution time at the expense of solution quality. GNC can only be applied to some special energy functions. Variational approximation converges to a local minimum. Graph Cut (GC) =-=[6]-=- is a fast efficient algorithm to solve a MAP-MRF whose energy function is Potts or Generalized Potts. 3 Basic Stereo Model In our work, to handle occlusion and depth discontinuity explicitly, we mode... |

1119 | A taxonomy and evaluation of dense two-frame stereo correspondence algorithms
- Scharstein, Szeliski
- 2002
(Show Context)
Citation Context ...reo algorithms and especially those using the Bayesian approach. We refer the reader to a detailed and updated taxonomy of dense, two-frame stereo correspondence algorithms by Scharstein and Szeliski =-=[21]-=-. It also provides a testbed for quantitative evaluation of stereo algorithms. A stereo algorithm is called a global method if there is a global objective function to be optimized. Otherwise it is cal... |

990 |
Optimal approximations by piecewise smooth functions and associated variational problems
- MUMFORD, SHAH
- 1989
(Show Context)
Citation Context ...able required to estimate, the other representing its discontinuities. Similar models such as the “weak membrane” model[5] in surface reconstruction, and the “Mumford-Shah” model in image segmentation=-=[26]-=- have also been studied in computer vision. However, in image formation of stereo pairs, the piecewise-smooth scene is projected on a pair of stereo images. Some regions are only visible in one image.... |

758 | Visual Reconstruction
- BLAKE, ZISSERMAN
- 1987
(Show Context)
Citation Context ...s a maximum a posteriori MRF (MAP-MRF) problem. There are several methods to solve the MAP-MRF problem: simulated annealing [12], Mean-Field annealing [10], the Graduated Non-Convexity algorithm(GNC) =-=[4]-=-, and Variational approximation [14]. Finding a solution by simulated annealing can often take an unacceptably long time although global optimization is achieved in theory. Mean-Field annealing is a d... |

650 | Learning in Graphical Models
- Jordan
- 1998
(Show Context)
Citation Context ...ow the belief propagation algorithm is used to compute the MAP of the posterior distribution (12). B. Algorithm approximation: loopy belief propagation In the literature of probabilistic graph models =-=[19]-=-, a Markov network is an undirected graph as shown in Figure 3. Nodes {xs} are hidden variables and nodes {ys} are observed variables. By denoting X = {xs} and Y = {ys}, the posterior P (X|Y ) can be ... |

626 |
Markov chain Monte Carlo in practice
- Gilks, Richardson, et al.
- 1996
(Show Context)
Citation Context ...en if the forms and parameters are given, it is still difficult to find the MAP of a composition of a continuous MRFs D and two binary MRFs L and O. Although the Markov Chain Monte Carlo (MCMC) [14], =-=[15]-=- approach provides an effective way to explore a posterior distribution, the computational requirement makes MCMC impractical for stereo matching. The solution space of our model is Ω = Ωd × Ωl × Ωo, ... |

477 | Learning low-level vision
- Freeman, Pasztor, et al.
- 2000
(Show Context)
Citation Context ... Pearl[19] in the belief networkwithout loops. Loopy Belief Propagation is just Belief Propagation that ignores the existence of loops in the networks. It has been applied successfully to some vision =-=[9]-=- and communication [24] problems despite the presence of networkloops. The posterior probability (15) over D is exactly a Markov Network in the literature of probabilistic graph models as shown in Fig... |

404 | Layered Depth Images
- Shade, Gortler, et al.
- 1998
(Show Context)
Citation Context ...” and “Dayton”, we present results of new view synthesis shown in Figure 11(b), (d) and (f). The reference view is forward warped to a new viewpoint using a computed depth map by a two-pass algorithm =-=[31]-=-. Some large textureless regions, such as the sky in “Garden” and “Dayton”, are still hard to handle. A. Assumptions VIII. Discussion In Bayesian approaches, all assumptions must be made explicitly. I... |

296 | Computing Visual Correspondence with Occlusions Using
- Kolmogorov, Zabih
- 2001
(Show Context)
Citation Context ...rporating image segmentation improves stereo matching significantly, with 40% error reduction in B Ō. In fact, our algorithm ranks as the best for “Tsukuba” and outperforms Graph Cut (with occlusion) =-=[17]-=- which was widely considered the state-of-art stereo matching algorithm. Our algorithm competes well with other stereo algorithms for the three other data sets, “Sawtooth”, “Venus” and “Map”. It is in... |

286 |
A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment
- Kanade, Okutomi
- 1991
(Show Context)
Citation Context ...er in textureless regions and should be suspended at depth discontinuities. The fixed window is obviously invalid at depth discontinuities. Some improved windowbased methods, such as adaptive windows =-=[16]-=-, shiftable windows [5] and compact windows [23] try to avoid the windows that span depth discontinuities. Bayesian methods (e.g., [11,1,5,8,15]) are global methods that model discontinuities and occl... |

279 |
Cooperative computation of stereo disparity
- Marr, Poggio
- 1976
(Show Context)
Citation Context ...on is a measurement or observation. The most common matching costs, e.g. squared intensity difference(SD), absolute intensity difference[20], normalizedcross correlation[28], [7], binary matching cost=-=[25]-=-, gradient-based measurement[?], rank transform[35], phase and filter-bank responses[?], shifted absolute difference[3], are ways of computing the likelihood function. Different aggregation methods re... |

220 | Non-parametric local transforms for computing visual correspondence
- Zabih, Woodfill
- 1994
(Show Context)
Citation Context ... matching costs, e.g. squared intensity difference(SD), absolute intensity difference[20], normalizedcross correlation[28], [7], binary matching cost[25], gradient-based measurement[?], rank transform=-=[35]-=-, phase and filter-bank responses[?], shifted absolute difference[3], are ways of computing the likelihood function. Different aggregation methods reflect different priors assumed on scene structure. ... |

210 | A maximum-likelihood stereo algorithm
- Cox, Hingorani, et al.
- 1996
(Show Context)
Citation Context ...ies. Some improved windowbased methods, such as adaptive windows [16], shiftable windows [5] and compact windows [23] try to avoid the windows that span depth discontinuities. Bayesian methods (e.g., =-=[11,1,5,8,15]-=-) are global methods that model discontinuities and occlusion. Geiger et al. [11] derived an occlusion process and a disparity field from a matching process. Assuming an “order constraint” and “unique... |

200 | On the unification of line processes, outlier rejection, and robust statistics with applications in early vision
- Black, Rangarajan
- 1996
(Show Context)
Citation Context ...only to specify appropriate forms of ϕ(ds,dt), γ(ls,t) and ηc(os), but also to do inference in a continuous MRF and two binary MRFs. Fortunately, the unification of line process and robust statistics =-=[3]-=- provides us a way to eliminate the binary random variable from our MAP problem. If we simplify ηc(os) by ignoring the spatial interaction of occlusion sites1 ηc(os) =η(os) (10) we can rewrite our MAP... |

197 | Robust analysis of feature spaces: color image segmentation
- Comaniciu, Meer
- 1997
(Show Context)
Citation Context ...hen node xs and xt are in different regions words, the influence from neighbors becomes smaller as λseg increases. In our experiments, the segmentation labels are produced by the Mean-Shift algorithm =-=[7]-=-. The execution time is usually just a few seconds in all images used in our experiments. According to (15), the compatibility matrix ψst(xs,xt) can be rewritten as: ψst(xs,xt) = exp(−ρp(xs,xt))) exp(... |

182 | A pixel dissimilarity measure that is insensitive to image sampling
- Birchfield, Tomasi
- 1998
(Show Context)
Citation Context ...nsiders the pixels only in non-occluded areas s/∈ O because likelihood of the pixels in occluded areas can not be well defined. We use the pixel dissimilarity that is provably insensitive to sampling =-=[2]-=-: F (s, ds,I) = min{d(s, s ′ ,I)/σf , d(s ′ ,s,I)/σf } where d(s, s ′ ,I) = min{ � � IL(s) − I − R (s′ ) � � , |IL(s) − IR(s ′ )| , � � IL(s) − I + R (s′ ) � � }, s ′ is the matching pixel in right vi... |

149 |
Parallel and deterministic algorithms from MRFs: Surface reconstruction and integration
- Geiger, Girosi
- 1990
(Show Context)
Citation Context ...ral, Bayesian stereo matching can be formulated as a maximum a posteriori MRF (MAP-MRF) problem. There are several methods to solve the MAP-MRF problem: simulated annealing [12], Mean-Field annealing =-=[10]-=-, the Graduated Non-Convexity algorithm(GNC) [4], and Variational approximation [14]. Finding a solution by simulated annealing can often take an unacceptably long time although global optimization is... |

133 | Occlusions and binocular stereo
- Geiger, Ladendorf, et al.
- 1995
(Show Context)
Citation Context ...ies. Some improved windowbased methods, such as adaptive windows [16], shiftable windows [5] and compact windows [23] try to avoid the windows that span depth discontinuities. Bayesian methods (e.g., =-=[11,1,5,8,15]-=-) are global methods that model discontinuities and occlusion. Geiger et al. [11] derived an occlusion process and a disparity field from a matching process. Assuming an “order constraint” and “unique... |

128 |
Depth from edge and intensity based stereo
- Baker, Binford
- 1981
(Show Context)
Citation Context ...metric version of d(s, s ′ ,I)andσfis the image noise variance to be estimated. B. Prior There is no simple statistical relationship between coupled fields {D, L} and field O. The ordering constraint =-=[1]-=- assumes that the order of neighboring correspondences is always preserved. This ordering allows the construction of a dynamic programming scheme. However, this constraint may not always be true. For ... |

118 | Large occlusion stereo
- Bobick, Intille
- 1999
(Show Context)
Citation Context ...s and should be suspended at depth discontinuities. The fixed window is obviously invalid at depth discontinuities. Some improved windowbased methods, such as adaptive windows [16], shiftable windows =-=[5]-=- and compact windows [23] try to avoid the windows that span depth discontinuities. Bayesian methods (e.g., [11,1,5,8,15]) are global methods that model discontinuities and occlusion. Geiger et al. [1... |

118 | The variational approach to shape from shading
- Horn, Brooks
- 1986
(Show Context)
Citation Context ...RF) problem. There are several methods to solve the MAP-MRF problem: simulated annealing [12], Mean-Field annealing [10], the Graduated Non-Convexity algorithm(GNC) [4], and Variational approximation =-=[14]-=-. Finding a solution by simulated annealing can often take an unacceptably long time although global optimization is achieved in theory. Mean-Field annealing is a deterministic approximation to simula... |

118 | Handling occlusions in dense multi-view stereo
- Kang, Szeliski, et al.
- 2001
(Show Context)
Citation Context ...ts the disparity gradient. Obviously, the fixed window is obviously invalid at depth discontinuities. Some improved window-based methods, such as adaptive windows [20] and shiftable windows[6], [33], =-=[21]-=- try to avoid windows that span depth discontinuities. Bayesian methods (e.g., [13], [18], [2], [10], [6]) are global methods that model discontinuities and occlusion. Bayesian methods can be classifi... |

113 |
A bayesian-approach to binocular stereopsis
- Belhumeur
- 1996
(Show Context)
Citation Context ...ies. Some improved windowbased methods, such as adaptive windows [16], shiftable windows [5] and compact windows [23] try to avoid the windows that span depth discontinuities. Bayesian methods (e.g., =-=[11,1,5,8,15]-=-) are global methods that model discontinuities and occlusion. Geiger et al. [11] derived an occlusion process and a disparity field from a matching process. Assuming an “order constraint” and “unique... |

107 |
A global matching framework for stereo computation
- Tao, Sawhney, et al.
- 2001
(Show Context)
Citation Context ...IPLE CUES More low-level visual cues (e.g., segmentation, edges, corners) can be incorporated into the intensity constraint to improve stereo matching. Recently, a segmentation-based stereo algorithm =-=[32]-=- has been proposed based on the assumption that the depth discontinuities occur on the boundary of the segmented regions. In [32], the segmentationSUN ET AL.: STEREO MATCHING USING BELIEF PROPAGATION... |

104 | Stereo matching with nonlinear diffusion
- Scharstein, Szeliski
- 1998
(Show Context)
Citation Context ...)m1m3,1m4,1m5,1. x1 The belief at node x1 is computed as: b1 ← κm1m2,1m3,1m4,1m5,1 . 4.2 Belief Propagation The model that is most similar to our posterior probability (15) is Scharstein & Szeliski’s =-=[20]-=-. Unlike Scharstein & Szeliski, where a nonlinear diffusion algorithm is used, we address this MAP problem by Belief Propagation. Belief Propagation is an exact inference method proposed by Pearl[19] ... |

95 | Occlusions, discontinuities, and epipolar lines in stereo
- Ishikawa, Geiger
- 1998
(Show Context)
Citation Context |

77 | Bethe free energy, Kikuchi approximations, and belief propagation algorithms
- Yedidia, Freeman, et al.
- 2001
(Show Context)
Citation Context ...ef networkwithout loops. Loopy Belief Propagation is just Belief Propagation that ignores the existence of loops in the networks. It has been applied successfully to some vision [9] and communication =-=[24]-=- problems despite the presence of networkloops. The posterior probability (15) over D is exactly a Markov Network in the literature of probabilistic graph models as shown in Figure 3. In the Markov Ne... |

54 |
The JISCT Stereo Evaluation
- Bolles, Hannah
- 1993
(Show Context)
Citation Context ..., matching cost computation is a measurement or observation. The most common matching costs, e.g. squared intensity difference(SD), absolute intensity difference[20], normalizedcross correlation[28], =-=[7]-=-, binary matching cost[25], gradient-based measurement[?], rank transform[35], phase and filter-bank responses[?], shifted absolute difference[3], are ways of computing the likelihood function. Differ... |

22 | Stereo matching by compact windows via minimum ratio cycle
- Veksler
- 2001
(Show Context)
Citation Context ...d at depth discontinuities. The fixed window is obviously invalid at depth discontinuities. Some improved windowbased methods, such as adaptive windows [16], shiftable windows [5] and compact windows =-=[23]-=- try to avoid the windows that span depth discontinuities. Bayesian methods (e.g., [11,1,5,8,15]) are global methods that model discontinuities and occlusion. Geiger et al. [11] derived an occlusion p... |

21 | An intensity-based cooperative bidirectional stereo matching with simultaneous detection of discontinuities and occlusions
- Luo, Burkhardt
- 1995
(Show Context)
Citation Context ...o resort to a region-based method, such as neighborhood depth hypothesis [32] to infer occlusions. A more promising approach to handle occlusion for two-frame stereo matching is Left Right Check(LRC) =-=[24]-=-. In section IV-B, we simplify the basic stereo model from (7) to (12) by introducing two robust functions. The model that is most similar to our posterior probability (12) is Scharstein & Szeliski’s ... |

15 |
Robust Analysis of Feature Spaces
- Comaniciu, Meer
- 1997
(Show Context)
Citation Context ...age between neighbor sites becomes. In other words, the influence from neighbors becomes smaller as seg increases. In our experiments, the segmentation labels are produced by the Mean-Shift algorithm =-=[9]-=-. It takes just a few seconds for each image used in our experiments. With the introduction of pcueðds;dtÞ, the compatibility matrix stðxs;xtÞ becomes: stðxs;xtÞ expð pðxs;xtÞÞÞ expð pcueðxs;xtÞÞÞ: ð... |

10 |
Real-time correlation-based stereo vision with reduced border errors
- Hirschmueller, Innocent, et al.
(Show Context)
Citation Context ...6 9.41 1.36 0.23 6.57 1.36 1.75 6.63 0.33 4.40 GC+occl. [17] 1.27 0.43 6.90 0.36 0.00 3.65 2.79 5.39 2.54 1.79 10.08 Graph cuts [6] 1.86 1.00 9.35 0.42 0.14 3.76 1.69 2.30 5.40 2.39 9.35 Realtime SAD =-=[13]-=- 4.25 4.47 15.05 1.32 0.35 9.21 1.53 1.80 12.33 0.81 11.35 Bay. diff. [21] 6.49 11.62 12.29 1.43 0.69 9.29 3.89 7.15 18.17 0.20 2.49 SSD+MF [21] 5.26 3.86 24.65 2.14 0.72 13.08 3.81 6.93 12.94 0.66 9.... |

9 |
Prediction of correlation errors in stereo-pair images
- Ryan, Gray, et al.
- 1980
(Show Context)
Citation Context ...f view, matching cost computation is a measurement or observation. The most common matching costs, e.g. squared intensity difference(SD), absolute intensity difference[20], normalizedcross correlation=-=[28]-=-, [7], binary matching cost[25], gradient-based measurement[?], rank transform[35], phase and filter-bank responses[?], shifted absolute difference[3], are ways of computing the likelihood function. D... |

5 |
Handing Occlusions
- Kang, Szeliski, et al.
- 2001
(Show Context)
Citation Context ...thod limits the disparity gradient. Obviously, the fixed window is invalid at depth discontinuities. Some improved window-based methods, such as adaptive windows [20] and shiftable windows [6], [33], =-=[21]-=- try to avoid windows that span depth discontinuities. Bayesian methods (e.g., [13], [18], [2], [10], [6]) are global methods that model discontinuities and occlusion. Bayesian methods can be classifi... |