## Multi-view Stereo via Volumetric Graph-cuts and Occlusion Robust Photo-Consistency (2007)

### Cached

### Download Links

- [svr-www.eng.cam.ac.uk]
- [mi.eng.cam.ac.uk]
- [carlos-hernandez.org]
- [mi.eng.cam.ac.uk]
- [carlos-hernandez.org]
- [mi.eng.cam.ac.uk]
- [mi.eng.cam.ac.uk]
- [mi.eng.cam.ac.uk]
- [svr-www.eng.cam.ac.uk]
- [mi.eng.cam.ac.uk]
- [mi.eng.cam.ac.uk]
- [www.george-vogiatzis.org]
- [cms.brookes.ac.uk]
- [eprints.pascal-network.org]
- [mi.eng.cam.ac.uk]
- [www.george-vogiatzis.org]

Citations: | 135 - 10 self |

### BibTeX

@MISC{Vogiatzis07multi-viewstereo,

author = {George Vogiatzis and Carlos Hernández Esteban and Philip H. S. Torr and Roberto Cipolla},

title = { Multi-view Stereo via Volumetric Graph-cuts and Occlusion Robust Photo-Consistency},

year = {2007}

}

### OpenURL

### Abstract

This paper presents a volumetric formulation for the multi-view stereo problem which is amenable to a computationally tractable global optimisation using Graph-cuts. Our approach is to seek the optimal partitioning of 3D space into two regions labelled as ‘object’ and ‘empty’ under a cost functional consisting of the following two terms: (1) A term that forces the boundary between the two regions to pass through photo-consistent locations and (2) a ballooning term that inflates the ‘object ’ region. To take account of the effect of occlusion on the first term we use an occlusion robust photo-consistency metric based on Normalised Cross Correlation, which does not assume any geometric knowledge about the reconstructed object. The globally optimal 3D partitioning can be obtained as the minimum cut solution of a weighted graph.

### Citations

1384 | Fast Approximate Energy Minimization via Graph Cuts, Proc
- Boykov, Veksler, et al.
- 2001
(Show Context)
Citation Context ...ften formulated as minimisations of Markov Random Field (MRF) energy functions providing a clean and computationally-tractable formulation, for which good approximate solutions exist using Graph-cuts =-=[5]-=-, [14], [21] or Loopy Belief Propagation [27]. They can also be formulated as continuous PDE evolutions on the depth maps [26]. However, a key limitation of these solutions is that they can only repre... |

1032 | A taxonomy and evaluation of dense two-frame stereo correspondence algorithms
- Scharstein, Szeliski
- 2002
(Show Context)
Citation Context ...arity or depth value is assigned to each of its pixels using a combination of image correlation and regularisation. An excellent review for image based methods can be found in Scharstein and Szeliski =-=[22]-=-. These problems are often formulated as minimisations of Markov Random Field (MRF) energy functions providing a clean and computationally-tractable formulation, for which good approximate solutions e... |

970 |
Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations
- Osher, Sethian
- 1988
(Show Context)
Citation Context ...ecrease the probability of getting trapped in local minima (e.g. [7], [8], [11], [20]). Furthermore, explicit representations such as meshes are known to suffer from topological and sampling problems =-=[18]-=-. The approach described in this paper combines the advantages of both classes described above. We adopt an implicit volumetric representation based on voxel occupancy, but we pose the reconstruction ... |

797 | An experimental comparison of min-cut/max-flow algorithms for energy minimization
- Boykov, Kolmogorov
- 2004
(Show Context)
Citation Context ...ly with the total number of pixels in the sequence. The computation time required by the graph-cut computation for a 300 3 grid is approximately 45 minutes. We used the graphcut algorithm proposed in =-=[3]-=- and in particular the implementation available at the authors’ website. The first experiment was performed on a plaster bust of the Greek goddess Hygeia (36 images) photographed with a 5M pixel digit... |

694 | What energy functions can be minimized via graph cuts
- Kolmogorov, Zabih
- 2004
(Show Context)
Citation Context ...side the scene while the pairwise potential between two neighbour voxels i and j is equal to wij if the voxels have opposite labels and 0 otherwise. As a binary MRF with a sub-modular energy function =-=[15]-=- it can be solved exactly in polynomial time using Graph-cuts. V. RESULTS In this section we present some 3D reconstruction results obtained by our technique. The system used for all the models shown ... |

457 | A theory of shape by space carving
- Kutulakos, Seitz
- 1992
(Show Context)
Citation Context ...ple viewpoints can be easily integrated and surface smoothness can be enforced independent of viewpoint. This class consists of techniques using implicit representations such as voxel occupancy grids =-=[16]-=-, or level-sets of 3D scalar fields [7], [20], and explicit representations such as polygonal meshes [8], [11]. While some of these methods are known to produce high quality reconstructions their conv... |

314 | A comparison and evaluation of multi-view stereo reconstruction algorithms
- Seitz, Curless, et al.
- 2006
(Show Context)
Citation Context ... as the reference image the results may be different. The second class comprises of methods that use a volumetric representation of shape. For a recent, very thorough review of related techniques see =-=[23]-=-. Under this framework multiple viewpoints can be easily integrated and surface smoothness can be enforced independent of viewpoint. This class consists of techniques using implicit representations su... |

279 |
Finite-Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images
- Cohen
- 1993
(Show Context)
Citation Context ...ata-independent model where σ(x) is defined as a negative constant λ that produces an inflationary (ballooning) tendency. The motivation for this type of term in the active contour domain is given in =-=[6]-=-, but intuitively, it can be thought May 31, 2007 DRAFT 6sof as a shape prior that favours objects that fill the bounding volume in the absence of any other information. If the value of λ is too large... |

257 | Multi-camera scene reconstruction via graph cuts
- Kolmogorov, Zabih
- 2002
(Show Context)
Citation Context ...formulated as minimisations of Markov Random Field (MRF) energy functions providing a clean and computationally-tractable formulation, for which good approximate solutions exist using Graph-cuts [5], =-=[14]-=-, [21] or Loopy Belief Propagation [27]. They can also be formulated as continuous PDE evolutions on the depth maps [26]. However, a key limitation of these solutions is that they can only represent d... |

252 | Stereo matching using belief propagation
- Zheng, Shum
- 2003
(Show Context)
Citation Context ...andom Field (MRF) energy functions providing a clean and computationally-tractable formulation, for which good approximate solutions exist using Graph-cuts [5], [14], [21] or Loopy Belief Propagation =-=[27]-=-. They can also be formulated as continuous PDE evolutions on the depth maps [26]. However, a key limitation of these solutions is that they can only represent depth maps with a unique disparity per p... |

234 | A maximum-flow formulation of the ncamera stereo correspondence problem
- Roy, Cox
- 1998
(Show Context)
Citation Context ...ated as minimisations of Markov Random Field (MRF) energy functions providing a clean and computationally-tractable formulation, for which good approximate solutions exist using Graph-cuts [5], [14], =-=[21]-=- or Loopy Belief Propagation [27]. They can also be formulated as continuous PDE evolutions on the depth maps [26]. However, a key limitation of these solutions is that they can only represent depth m... |

230 |
On the estimation of a probability density function and the
- Parzen
- 1962
(Show Context)
Citation Context ...t be the global one. In order to take into account this observation, we build a new C by detecting all the local maxima dk of Sj, i.e., ∂Sj ∂d (dk) = 0, ∂2 Sj ∂d 2 (dk) > 0, and using a Parzen window =-=[19]-=- with a kernel W as follows: C w (d) = � j∈N(i) � Sj(dk)W(d − dk). (7) k The Parzen window technique provides an effective way of taking into account the actual scores of the local maxima and reinforc... |

194 | Variational principles, surface evolution, PDE’s, level set methods and the stereo problem
- Faugeras, Keriven
- 1998
(Show Context)
Citation Context ...and surface smoothness can be enforced independent of viewpoint. This class consists of techniques using implicit representations such as voxel occupancy grids [16], or level-sets of 3D scalar fields =-=[7]-=-, [20], and explicit representations such as polygonal meshes [8], [11]. While some of these methods are known to produce high quality reconstructions their convergence properties in the presence of n... |

180 | V.: Computing geodesics and minimal surfaces via graph cuts
- Boykov, Kolmogorov
- 2003
(Show Context)
Citation Context ...isation is computationally tractable, using existing max-flow algorithms. A. Background and previous work The inspiration for the approach presented in this paper is the work of Boykov and Kolmogorov =-=[2]-=- which establishes a theoretical link between maximum flow problems in discrete graphs and minimal surfaces in an arbitrary Riemannian metric. In particular the authors show how a continuous Riemannia... |

153 | Interactive image segmentation using an adaptive GMMRF model
- Blake, Rother, et al.
- 2004
(Show Context)
Citation Context ...cation described in that paper is interactive 2D or 3D segmentation. A probabilistic formulation of interactive segmentation with a more elaborate foreground/background model was given in Blake et al =-=[1]-=-. In [29] we showed how the basic idea of [2] can be applied to the volumetric, multiview stereo problem by computing a photo-consistency based Riemannian metric in which a minimal surface is computed... |

124 | Constructing virtual worlds using dense stereo - Narayanan, Rander, et al. - 1998 |

119 | Object-centered surface reconstruction: Combining multiimage stereo and shading
- Fua, Leclerc
- 1995
(Show Context)
Citation Context ...This class consists of techniques using implicit representations such as voxel occupancy grids [16], or level-sets of 3D scalar fields [7], [20], and explicit representations such as polygonal meshes =-=[8]-=-, [11]. While some of these methods are known to produce high quality reconstructions their convergence properties in the presence of noise are not well understood. Due to lack of regularisation, meth... |

93 | Segmentation by grouping junctions - Ishikawa, Geiger - 1998 |

82 | Exact Voxel Occupancy with Graph Cuts, in - Snow, Viola, et al. - 1999 |

69 | Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score
- Pons, Keriven, et al.
(Show Context)
Citation Context ...urface smoothness can be enforced independent of viewpoint. This class consists of techniques using implicit representations such as voxel occupancy grids [16], or level-sets of 3D scalar fields [7], =-=[20]-=-, and explicit representations such as polygonal meshes [8], [11]. While some of these methods are known to produce high quality reconstructions their convergence properties in the presence of noise a... |

61 | Gool. Dense matching of multiple widebaseline views
- Strecha, Tuytelaars, et al.
- 2003
(Show Context)
Citation Context ...e formulation, for which good approximate solutions exist using Graph-cuts [5], [14], [21] or Loopy Belief Propagation [27]. They can also be formulated as continuous PDE evolutions on the depth maps =-=[26]-=-. However, a key limitation of these solutions is that they can only represent depth maps with a unique disparity per pixel, i.e. depth is a function of image point. Capturing complete objects in this... |

55 |
Multi-view stereo revisited
- Goesele, Curless, et al.
- 2006
(Show Context)
Citation Context ...unique disparity per pixel, i.e. depth is a function of image point. Capturing complete objects in this manner requires further processing to merge multiple depth maps. This was recently attempted in =-=[10]-=- but resulted in only partially reconstructed object surfaces, leaving holes in areas of uncertainty. A second limitation is that the smoothness term imposed by the MRF is defined on image disparities... |

55 | Silhouette and stereo fusion for 3D object modeling
- Hernandez, Schmitt
- 2004
(Show Context)
Citation Context ...class consists of techniques using implicit representations such as voxel occupancy grids [16], or level-sets of 3D scalar fields [7], [20], and explicit representations such as polygonal meshes [8], =-=[11]-=-. While some of these methods are known to produce high quality reconstructions their convergence properties in the presence of noise are not well understood. Due to lack of regularisation, methods ba... |

51 | Carved visual hulls for image-based modeling
- Furukawa, Ponce
(Show Context)
Citation Context ...hod considerably. Nevertheless, by demonstrating promising results and highlighting the feasibility of solving multi-view stereo using volumetric graph cuts, [29] inspired a number of techniques [4], =-=[9]-=-, [13], [17], [24], [25], [28] that built on our formulation and attempted to address some of its shortcomings. In Furukawa et al. [9] and Sinha et al. [24] two different ways were proposed for incorp... |

51 | A surface reconstruction method using global graph cut optimization - Paris, Sillion, et al. - 2004 |

49 | Multi-View Reconstruction Using PhotoConsistency and Exact Silhouette Constraints: A Maximum Flow Formulation - Sinha, Pollefeys - 2005 |

34 | KOBBELT L.: Hierarchical volumetric multi-view stereo reconstruction of manifold surfaces based on dual graph embedding
- HORNUNG
- 2006
(Show Context)
Citation Context ...onsiderably. Nevertheless, by demonstrating promising results and highlighting the feasibility of solving multi-view stereo using volumetric graph cuts, [29] inspired a number of techniques [4], [9], =-=[13]-=-, [17], [24], [25], [28] that built on our formulation and attempted to address some of its shortcomings. In Furukawa et al. [9] and Sinha et al. [24] two different ways were proposed for incorporatin... |

33 | 3d surface reconstruction using graph cuts with surface constraints
- Tran, Davis
- 2006
(Show Context)
Citation Context ...s, by demonstrating promising results and highlighting the feasibility of solving multi-view stereo using volumetric graph cuts, [29] inspired a number of techniques [4], [9], [13], [17], [24], [25], =-=[28]-=- that built on our formulation and attempted to address some of its shortcomings. In Furukawa et al. [9] and Sinha et al. [24] two different ways were proposed for incorporating the powerful silhouett... |

31 | Silhouette coherence for camera calibration under circular motion
- Hernandez, Schmitt, et al.
- 2007
(Show Context)
Citation Context ...t method 0.50mm / 98.4% 0.64mm / 99.2% 0.69mm / 96.9% TABLE I COMPARISON OF OUR METHOD WITH STATE-OF-THE-ART TECHNIQUES AGAINST GROUND TRUTH DATA (FROM [23]). obtained automatically using silhouettes =-=[12]-=-. As in the first experiment however, we did not include these silhouettes in our foreground/background term. The mesh obtained from the 300 3 voxel grid contains accurately reconstructed sub-millimet... |

24 | Oriented Visibility for Multiview Reconstruction
- Lempitsky, Boykov, et al.
- 2006
(Show Context)
Citation Context ...rably. Nevertheless, by demonstrating promising results and highlighting the feasibility of solving multi-view stereo using volumetric graph cuts, [29] inspired a number of techniques [4], [9], [13], =-=[17]-=-, [24], [25], [28] that built on our formulation and attempted to address some of its shortcomings. In Furukawa et al. [9] and Sinha et al. [24] two different ways were proposed for incorporating the ... |

21 |
From photohulls to photoflux optimization, in
- Boykov, Lempitsky
- 2006
(Show Context)
Citation Context ...e method considerably. Nevertheless, by demonstrating promising results and highlighting the feasibility of solving multi-view stereo using volumetric graph cuts, [29] inspired a number of techniques =-=[4]-=-, [9], [13], [17], [24], [25], [28] that built on our formulation and attempted to address some of its shortcomings. In Furukawa et al. [9] and Sinha et al. [24] two different ways were proposed for i... |

17 | R.: Reconstructing relief surfaces - Vogiatzis, Torr, et al. |

15 | Volumetric stereo with silhouette and feature constraints
- Starck, Hilton, et al.
- 2006
(Show Context)
Citation Context ...theless, by demonstrating promising results and highlighting the feasibility of solving multi-view stereo using volumetric graph cuts, [29] inspired a number of techniques [4], [9], [13], [17], [24], =-=[25]-=-, [28] that built on our formulation and attempted to address some of its shortcomings. In Furukawa et al. [9] and Sinha et al. [24] two different ways were proposed for incorporating the powerful sil... |

3 | Variational principles in computational stereo - Faugeras, Gomes, et al. - 2003 |