## A geometric approach to shape from defocus (2005)

### Cached

### Download Links

- [emfs1.eps.hw.ac.uk]
- [www.vision.cs.ucla.edu]
- [vision.ucla.edu]
- [home.eps.hw.ac.uk]
- DBLP

### Other Repositories/Bibliography

Venue: | IEEE Trans. Pattern Anal. Mach. Intell |

Citations: | 43 - 1 self |

### BibTeX

@ARTICLE{Favaro05ageometric,

author = {Paolo Favaro and Stefano Soatto},

title = {A geometric approach to shape from defocus},

journal = {IEEE Trans. Pattern Anal. Mach. Intell},

year = {2005},

volume = {27},

pages = {406--417}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract—We introduce a novel approach to shape from defocus, i.e., the problem of inferring the three-dimensional (3D) geometry of a scene from a collection of defocused images. Typically, in shape from defocus, the task of extracting geometry also requires deblurring the given images. A common approach to bypass this task relies on approximating the scene locally by a plane parallel to the image (the so-called equifocal assumption). We show that this approximation is indeed not necessary, as one can estimate 3D geometry while avoiding deblurring without strong assumptions on the scene. Solving the problem of shape from defocus requires modeling how light interacts with the optics before reaching the imaging surface. This interaction is described by the so-called point spread function (PSF). When the form of the PSF is known, we propose an optimal method to infer 3D geometry from defocused images that involves computing orthogonal operators which are regularized via functional singular value decomposition. When the form of the PSF is unknown, we propose a simple and efficient method that first learns a set of projection operators from blurred images and then uses these operators to estimate the 3D geometry of the scene from novel blurred images. Our experiments on both real and synthetic images show that the performance of the algorithm is relatively insensitive to the form of the PSF. Our general approach is to minimize the Euclidean norm of the difference between the estimated images and the observed images. The method is geometric in that we reduce the minimization to performing projections onto linear subspaces, by using inner product structures on both infinite and finite-dimensional Hilbert spaces. Both proposed algorithms involve only simple matrix-vector multiplications which can be implemented in real-time. Index Terms—Shape from defocus, depth from defocus, blind deconvolution, image processing, deblurring, shape, 3D reconstruction, shape estimation, image restoration, learning subspaces. 1

### Citations

783 | Optimization by Vector Space Methods
- Luenberger
- 1969
(Show Context)
Citation Context ...lgorithm. We work in function space and use the geometry of operators between Hilbert spaces. For basic results on operators between finite and infinite-dimensional Hilbert spaces, see, for instance, =-=[17]-=-. 4.1 Notation and Formalization of the Problem For ease of notation we rearrange an image I in a column vector of dimension MN. Also, we collect a number K of images for different optics settings and... |

737 | Visual Reconstruction
- Blake, Zisserman
- 1987
(Show Context)
Citation Context ...In computer vision such a task is called shape-fromX, where X denotes the cue used to infer shape. For example, one can capture images from different vantage points as in stereo and motion [18], [8], =-=[4]-=-. In this paper, instead, we consider images that are captured with different optical settings of the imaging device, which leads to the problem of shape from defocus. In shape from defocus one can, f... |

717 |
Computer Vision: A Modern Approach
- Forsyth, Ponce
- 2000
(Show Context)
Citation Context ... formation model would require us to consider light interactions based on Maxwell’s equations [5]. A simpler way of proceeding is, instead, to resort to notions of optical geometry (see, for example, =-=[13]-=-). We shall be content with a reasonable approximation of light interaction based on assuming that the surface is Lambertian, 1 that there are no occlusions or self-occlusions (see [3], [11] for these... |

320 |
und E.Wolf. Principles of Optics
- Born
- 1986
(Show Context)
Citation Context ... formation model for defocused images and outline the assumptions we make in deriving it. A rigorous image formation model would require us to consider light interactions based on Maxwell’s equations =-=[5]-=-. A simpler way of proceeding is, instead, to resort to notions of optical geometry (see, for example, [13]). We shall be content with a reasonable approximation of light interaction based on assuming... |

189 |
A new sense for depth of field
- Pentland
- 1987
(Show Context)
Citation Context ...t at the price of a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], [24], =-=[25]-=-, [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow one to formally elimina... |

120 |
The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate
- Golub, Pereyra
- 1973
(Show Context)
Citation Context ...ELLIGENCE, VOL. 27, NO. 3, MARCH 2005 ðsÞ _H y sI: ð14Þ Then, ^s is also a local extremum of ðsÞ and ~s; ~r are also local extrema of ðs;rÞ. Proof. The proof extends the results of Golub and Pereyra =-=[16]-=-. For more details, see the Appendix. tu Remark 1. The significance of the proposition above is that (12) and (13) have the same minima in the 3D structure of the surfaces, but, while (12) is an optim... |

90 | Realtime focus range sensor
- Nayar, Watanabe, et al.
- 1995
(Show Context)
Citation Context ...nuities in the scene radiance [1], [19], [25], [34], [32], others that the radiance can be locally approximated by cubic polynomials [35], or that it can be controlled by using structured light [14], =-=[22]-=-, [24]. A more common simplification of the image formation model is the so-called equifocal assumption, which consists of assuming that the surface of the scene can be locally approximated by a plane... |

78 | Depth from defocus: a spatial domain approach
- Subbarao, Surya
- 1994
(Show Context)
Citation Context ..., some assume that the scene contains “sharp edges,” i.e., discontinuities in the scene radiance [1], [19], [25], [34], [32], others that the radiance can be locally approximated by cubic polynomials =-=[35]-=-, or that it can be controlled by using structured light [14], [22], [24]. A more common simplification of the image formation model is the so-called equifocal assumption, which consists of assuming t... |

73 |
Lawrence,“An investigation of methods for determining depth from focus
- Ens, P
- 1993
(Show Context)
Citation Context ...truction from defocused images, but at the price of a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model =-=[7]-=-, [9], [10], [12], [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since... |

72 |
Introduction to Inverse Problems
- Bertero, Boccacci
- 1998
(Show Context)
Citation Context ... we distinguish between those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain =-=[2]-=-, [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow one to formally eliminate undesired unknowns (the appearance, or “radiance”). However, the assumpt... |

59 | Depth from focusing and defocusing
- Xiong, Shafer
(Show Context)
Citation Context ...mation model is the so-called equifocal assumption, which consists of assuming that the surface of the scene can be locally approximated by a plane parallel to the image plane [19], [35], [25], [36], =-=[38]-=-, [41]. One advantage of such an assumption is that it allows one to avoid reconstructing the appearance of the scene while recovering its geometry. However, it also fails to 0162-8828/05/$20.00 ß 200... |

56 |
Depth from defocus: a real aperture imaging approach
- Chaudhuri, Rajagopalan
- 1999
(Show Context)
Citation Context .... 2 RELATION TO PREVIOUS WORK The general problem of shape from defocus has been addressed in a variety of contexts: Earlier approaches adopted Markov random fields to model both shape and appearance =-=[6]-=-, [29], [30]. This approach has been shown to be effective for surface reconstruction from defocused images, but at the price of a high computational cost. Among deterministic approaches, we distingui... |

48 | Rational filters for passive depth from defocus
- Watanabe, Nayar
- 1998
(Show Context)
Citation Context ...ge formation model is the so-called equifocal assumption, which consists of assuming that the surface of the scene can be locally approximated by a plane parallel to the image plane [19], [35], [25], =-=[36]-=-, [38], [41]. One advantage of such an assumption is that it allows one to avoid reconstructing the appearance of the scene while recovering its geometry. However, it also fails to 0162-8828/05/$20.00... |

35 |
Depth recovery from blurred edges
- Subbarao, Gurumoorthy
(Show Context)
Citation Context ...ape from defocus is based on simplifications of the image formation model. For example, some assume that the scene contains “sharp edges,” i.e., discontinuities in the scene radiance [1], [19], [25], =-=[34]-=-, [32], others that the radiance can be locally approximated by cubic polynomials [35], or that it can be controlled by using structured light [14], [22], [24]. A more common simplification of the ima... |

34 |
Three Dimensional Vision, a geometric viewpoint
- Faugeras
- 1993
(Show Context)
Citation Context ...ges. In computer vision such a task is called shape-fromX, where X denotes the cue used to infer shape. For example, one can capture images from different vantage points as in stereo and motion [18], =-=[8]-=-, [4]. In this paper, instead, we consider images that are captured with different optical settings of the imaging device, which leads to the problem of shape from defocus. In shape from defocus one c... |

29 |
Shape from focus: an effective approach for rough surfaces
- Nayar, Nakagawa
- 1990
(Show Context)
Citation Context ...d images, but at the price of a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model [7], [9], [10], [12], =-=[21]-=-, [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow one to form... |

28 |
An invitation to 3D vision, from images to models
- Ma, Soatto, et al.
- 2006
(Show Context)
Citation Context ...of images. In computer vision such a task is called shape-fromX, where X denotes the cue used to infer shape. For example, one can capture images from different vantage points as in stereo and motion =-=[18]-=-, [8], [4]. In this paper, instead, we consider images that are captured with different optical settings of the imaging device, which leads to the problem of shape from defocus. In shape from defocus ... |

27 | Occlusion edge blur: a cue to relative visual depth
- MARSHALL, BURBECK, et al.
- 1996
(Show Context)
Citation Context ...oaches to shape from defocus is based on simplifications of the image formation model. For example, some assume that the scene contains “sharp edges,” i.e., discontinuities in the scene radiance [1], =-=[19]-=-, [25], [34], [32], others that the radiance can be locally approximated by cubic polynomials [35], or that it can be controlled by using structured light [14], [22], [24]. A more common simplificatio... |

24 |
Edge and depth from focus
- Asada, Fujiwara, et al.
(Show Context)
Citation Context ... approaches to shape from defocus is based on simplifications of the image formation model. For example, some assume that the scene contains “sharp edges,” i.e., discontinuities in the scene radiance =-=[1]-=-, [19], [25], [34], [32], others that the radiance can be locally approximated by cubic polynomials [35], or that it can be controlled by using structured light [14], [22], [24]. A more common simplif... |

23 |
Simple range cameras based on focal error
- AP, Scherock, et al.
- 1994
(Show Context)
Citation Context ...ce of a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], [24], [25], [26], =-=[27]-=-, [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow one to formally eliminate undesired... |

21 | Microscopic shape from focus using active illumination
- Noguchi, Nayar
- 1994
(Show Context)
Citation Context ...es, but at the price of a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], =-=[24]-=-, [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow one to formally e... |

20 | Seeing beyond occlusions (and other marvels of a finite lens aperture
- Favaro, Soatto
(Show Context)
Citation Context ... example, [13]). We shall be content with a reasonable approximation of light interaction based on assuming that the surface is Lambertian, 1 that there are no occlusions or self-occlusions (see [3], =-=[11]-=- for these cases), that we are in a vacuum (see [23] when this hypothesis is removed), and that optics and surfaces are not dependent on light wavelength. These hypotheses allow us to describe the geo... |

18 | Observing shape from defocused images
- Favaro, Mennucci, et al.
(Show Context)
Citation Context ...ion from defocused images, but at the price of a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model [7], =-=[9]-=-, [10], [12], [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they... |

18 | Minimal operator set for passive depth from defocus - Watanabe, Nayar - 1996 |

14 | The optimal axial interval in estimating depth from defocus. Computer Vision
- Schechner, Kiryati
- 1999
(Show Context)
Citation Context ...etween those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], =-=[31]-=-, [39]. In particular, most of the latter approaches are appealing since they allow one to formally eliminate undesired unknowns (the appearance, or “radiance”). However, the assumptions required in o... |

10 | Learning shape from defocus
- Favaro, Soatto
(Show Context)
Citation Context ...rom defocused images, but at the price of a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model [7], [9], =-=[10]-=-, [12], [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow... |

9 | Optimal selection of camera parameters for recovery of depth from defocused images
- Rajagopalan, Chaudhuri
- 1997
(Show Context)
Citation Context ...ELATION TO PREVIOUS WORK The general problem of shape from defocus has been addressed in a variety of contexts: Earlier approaches adopted Markov random fields to model both shape and appearance [6], =-=[29]-=-, [30]. This approach has been shown to be effective for surface reconstruction from defocused images, but at the price of a high computational cost. Among deterministic approaches, we distinguish bet... |

8 | Optimal recovery of depth from defocused images using an MRF model
- Rajagopalan, Chaudhuri
- 1998
(Show Context)
Citation Context ...N TO PREVIOUS WORK The general problem of shape from defocus has been addressed in a variety of contexts: Earlier approaches adopted Markov random fields to model both shape and appearance [6], [29], =-=[30]-=-. This approach has been shown to be effective for surface reconstruction from defocused images, but at the price of a high computational cost. Among deterministic approaches, we distinguish between t... |

8 | Moment filters for high precision comput.ation of focus and stereo
- Xiong, Shafer
- 1995
(Show Context)
Citation Context ... those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], =-=[39]-=-. In particular, most of the latter approaches are appealing since they allow one to formally eliminate undesired unknowns (the appearance, or “radiance”). However, the assumptions required in order t... |

7 |
Computing depth from out-of-focus blur using a local frequency representation
- Gokstorp
- 1994
(Show Context)
Citation Context ...istinguish between those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], =-=[15]-=-, [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow one to formally eliminate undesired unknowns (the appearance, or “radiance”). However, the assumptions r... |

7 |
Monocular depth perception by evaluation of the blur in defocused images
- Schneider, Heit, et al.
- 1994
(Show Context)
Citation Context ...om defocus is based on simplifications of the image formation model. For example, some assume that the scene contains “sharp edges,” i.e., discontinuities in the scene radiance [1], [19], [25], [34], =-=[32]-=-, others that the radiance can be locally approximated by cubic polynomials [35], or that it can be controlled by using structured light [14], [22], [24]. A more common simplification of the image for... |

6 | Favaro,“A geometric approach to blind deconvolution with application to shape from defocus,”in Computer
- Soatto, P
- 2000
(Show Context)
Citation Context ...a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], [24], [25], [26], [27], =-=[33]-=-, [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow one to formally eliminate undesired unkno... |

5 | Shape and radiance estimation from the information divergence of blurred images
- Favaro, Soatto
- 2000
(Show Context)
Citation Context ...focused images, but at the price of a high computational cost. Among deterministic approaches, we distinguish between those that maintain a spatial representation of the imaging model [7], [9], [10], =-=[12]-=-, [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], [28], [31], [39]. In particular, most of the latter approaches are appealing since they allow one t... |

5 |
Depth from focus of structured light
- Scherock
- 1981
(Show Context)
Citation Context ...scontinuities in the scene radiance [1], [19], [25], [34], [32], others that the radiance can be locally approximated by cubic polynomials [35], or that it can be controlled by using structured light =-=[14]-=-, [22], [24]. A more common simplification of the image formation model is the so-called equifocal assumption, which consists of assuming that the surface of the scene can be locally approximated by a... |

4 |
Regularized depth from defocus
- Namboodiri, Chaudhuri, et al.
- 2008
(Show Context)
Citation Context ..., for example, [13]). We shall be content with a reasonable approximation of light interaction based on assuming that the surface is Lambertian, 1 that there are no occlusions or self-occlusions (see =-=[3]-=-, [11] for these cases), that we are in a vacuum (see [23] when this hypothesis is removed), and that optics and surfaces are not dependent on light wavelength. These hypotheses allow us to describe t... |

4 |
A block shift-variant blur model for recovering depth from defocused images
- Rajagopalan, Chaudhuri
- 1995
(Show Context)
Citation Context ...uish between those that maintain a spatial representation of the imaging model [7], [9], [10], [12], [21], [24], [25], [26], [27], [33], [37] and those that operate in the frequency domain [2], [15], =-=[28]-=-, [31], [39]. In particular, most of the latter approaches are appealing since they allow one to formally eliminate undesired unknowns (the appearance, or “radiance”). However, the assumptions require... |

4 |
Depth from Defocus Estimation
- Ziou, Deschenes
- 2001
(Show Context)
Citation Context ... model is the so-called equifocal assumption, which consists of assuming that the surface of the scene can be locally approximated by a plane parallel to the image plane [19], [35], [25], [36], [38], =-=[41]-=-. One advantage of such an assumption is that it allows one to avoid reconstructing the appearance of the scene while recovering its geometry. However, it also fails to 0162-8828/05/$20.00 ß 2005 IEEE... |

1 |
A Unified Approach to Image Focus and Defocus Analysis
- Yen-Fu
- 1998
(Show Context)
Citation Context ...pproaches that do not make this assumption yield accurate estimates of geometry, but are computationally challenging because they require estimating the radiance of the scene along with geometry [6], =-=[40]-=-. We propose a novel approach that is not based on the above assumptions. We show that it is not necessary to simplify the imaging model to avoid estimating the appearance of the scene while recoverin... |