## Image Mosaicing and Superresolution (2004)

Citations: | 50 - 4 self |

### BibTeX

@MISC{Capel04imagemosaicing,

author = {David Peter Capel},

title = {Image Mosaicing and Superresolution},

year = {2004}

}

### Years of Citing Articles

### OpenURL

### Abstract

The thesis investigates the problem of how information contained in multiple, overlapping images of the same scene may be combined to produce images of superior quality. This area, generically titled frame fusion, offers the possibility of reducing noise, extending the field of view, removal of moving objects, removing blur, increasing spatial resolution and improving dynamic range. As such, this research has many applications in fields as diverse as forensic image restoration, computer generated special effects, video image compression, and digital video editing. An essential enabling step prior to performing frame fusion is image registration, by which an accurate estimate of the point-to-point mapping between views is computed. A robust and efficient algorithm is described to automatically register multiple images using only information contained within the images themselves. The accuracy of this method, and the statistical assumptions upon which it relies, are investigated empirically. Two forms of frame-fusion are investigated. The first is image mosaicing, which is the alignment of multiple images into a single composition representing part of a 3D scene.

### Citations

3720 |
ªStochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,º
- Geman, Geman
- 1984
(Show Context)
Citation Context ...istance between the pair of pixels in C and not upon its orientation. For further details on the modelling of images with Gibbs distributions the reader is referred to Besag [14, 15], Geman and Geman =-=[67]-=- and Li [98]. In the case when the prior is a Gibbs distribution, equation (6.4) becomes f map = arg max f − ∑ ∀C∈C VC(C) − 1 2σ2 ‖Mf − g‖ n 2 (6.7) The difference between the many image priors propos... |

3159 |
Multiple View Geometry in Computer Vision
- Hartley, Zisserman
- 2003
(Show Context)
Citation Context ...an be found by linear methods. We now briefly review the linear method for estimating the homography H, from a set of N point correspondences x ↔ x ′ . Full details are given in Hartley and Zisserman =-=[77]-=-. From the definition of H, we have ⎛ x ⎝ ′ y ′ ⎞ ⎡ ⎠ = ⎣ 1 h11 h12 h13 h21 h22 h23 h31 h32 h33 ⎤ ⎛ ⎦ ⎝ where = is equality up to scale. Each inhomogeneous, 2D point correspondence generates two linea... |

3098 | A computational approach to edge detection
- Canny
- 1986
(Show Context)
Citation Context ...pixel edges extracted from each view of the calibration image. 37Objective Fit a line template to a view of the calibration image. Algorithm 1. Process the image with Canny’s sub-pixel edge detector =-=[27]-=-. This results in a point-sampled representation of the edges. 2. Transform the 16 × 16 line template under a putative homography H, which maps the template into the calibration image. 3. Measure the ... |

2452 |
Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography
- Fischler, Bolles
- 1981
(Show Context)
Citation Context ...77]. shows the resulting 268 matches super-imposed upon the first image. There are clearly many mismatches at this stage. RANSAC The RANdom SAmple Concensus algorithm, proposed by Fischler and Bolles =-=[62]-=-, is an extremely simple and powerful method for estimating model parameters given a data set heavily contaminated with outliers to the correct model. It has proved extremely effective in sifting 2-vi... |

1828 |
Robust Statistics
- Huber
- 1981
(Show Context)
Citation Context ...ution is evaluated. Instead of counting the number of inliers, as RANSAC does, MSAC determines the likelihood of the data given the proposed model parameters according to a simple, robust M-estimator =-=[79, 80]-=-. For each data point (i1, i2)n, the distance dn to the proposed line (α, β) is computed, and the overall cost associated with the solution is given by C = ∑ ρ(d 2 n) (3.13) ∀n 49where ρ(d 2 ) = d 2 ... |

1347 |
Practical Optimization
- Gill, Murray, et al.
- 1981
(Show Context)
Citation Context ...n be minimized with respect to the 8 parameters of H using an iterative, non-linear least-squares optimizer. Efficient algorithms for performing this optimization are based on the Gauss-Newton method =-=[50, 69, 70]-=-, the preferred variant being the Levenberg-Marquardt algorithm [117]. At every iteration, a finite difference approximation may be used to compute the Jacobian of the residuals with respect to each o... |

1138 |
Spatial interaction and the statistical analysis of lattice systems
- Besag
- 1974
(Show Context)
Citation Context ...V (C) depend only on the distance between the pair of pixels in C and not upon its orientation. For further details on the modelling of images with Gibbs distributions the reader is referred to Besag =-=[14, 15]-=-, Geman and Geman [67] and Li [98]. In the case when the prior is a Gibbs distribution, equation (6.4) becomes f map = arg max f − ∑ ∀C∈C VC(C) − 1 2σ2 ‖Mf − g‖ n 2 (6.7) The difference between the ma... |

941 |
Face recognition using eigenface
- Turk, Petland
- 1991
(Show Context)
Citation Context ...r vector y is far less than the number of pixels in the image (the dimension of f). This representation is applied here to registered face images. It is known from extensive use of PCA since at least =-=[167]-=- that a compact representation can be achieved in this case. Rather than learn the PC for the entire image, the face is divided into four key facial regions: the eyes, nose, mouth and cheek areas; and... |

766 | Texture synthesis by non-parametric sampling
- Efros, Leung
- 1999
(Show Context)
Citation Context ...des an interesting middle ground between the generic local priors previously used, and the type of priors based on sampling exemplars 216from training images (e.g. for homogeneous texture generation =-=[53]-=-) which have been applied to super-resolution by Candocia and Principe [26], and Freeman and Pasztor [63] The image will be modelled by its PCA components as ˆf = Vy + µ, where V represents the set of... |

737 | Visual Reconstruction
- Blake, Zisserman
- 1987
(Show Context)
Citation Context ...replaced by √ dx 2 + dy 2 + β, 187where β is a small constant. An alternative scheme with better global convergence properties is proposed by Chan et al. [33]. Weak-membrane MRFs Blake and Zisserman =-=[17]-=- propose a 2D analogue of the weakstring constraint, in which the gradient penalty is ρ(∆) = min(∆ 2 , α 2 ) (6.29) where ∆ is the image gradient. The parameter α controls the step-sensitivity, a thre... |

662 | Plenoptic modeling: an image-based rendering system - McMillan, Bishop - 1995 |

531 |
The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields
- Black, Anandan
- 1996
(Show Context)
Citation Context ...ustness in direct methods have been proposed. One approach is to use a similarity metric under which the influence of large residuals is reduced, such as the M-estimator approach of Black and Anandan =-=[16]-=-. An alternative and closely related scheme is a multiscale, iteratively re-weighted least-squares [82, 89] approach. However, both of these approaches are computationally expensive : optimizations in... |

524 | View interpolation for image synthesis - Chen, Williams - 1993 |

467 | E.: Learning low-level vision
- Freeman, Pasztor
(Show Context)
Citation Context ... based on sampling exemplars 216from training images (e.g. for homogeneous texture generation [53]) which have been applied to super-resolution by Candocia and Principe [26], and Freeman and Pasztor =-=[63]-=- The image will be modelled by its PCA components as ˆf = Vy + µ, where V represents the set of PC basis vectors and µ is the average of the training images. The aim is a low dimensional representatio... |

450 | Quicktime VR: an image-based approach to virtual environment navigation
- Chen
- 1995
(Show Context)
Citation Context ...hed together into a wide-field mosaic, such as Peleg & Herman’s VideoBrush system [113]. Mosaicing also forms the basic technology 6underlying node-based virtual reality systems such as QuickTime VR =-=[1, 35]-=- and SmoothMove [5]. The technique is also finding applications in video compression, digital steadycam, and in the enhancement of digital video editing and matte placement software. Super-resolution ... |

439 |
The Advanced Theory of Statistics
- Kendall, Stuart
- 1979
(Show Context)
Citation Context ...ence is Pr(g| ˆf) = ∏ Pr(gn| ˆf) (6.1) ∀n where g is the stack of vectors gn. The prior image model provides an additional pdf defined over the space of all images, Pr(f). By invoking Bayes’s theorem =-=[93, 96, 119, 139]-=-, we can combine the two to obtain the posterior distribution : Pr( ˆ f|g) = Pr(g|ˆ f) Pr( ˆ f) Pr(g) (6.2) 179The maximum a-posterior (MAP) estimate of f is then f map = arg max f = arg max f Pr(g| ... |

420 |
Regularization of Inverse Problems
- Engl, Hank, et al.
- 1996
(Show Context)
Citation Context ...e matrix L is typically chosen to be a discrete approximation of a first or second derivative operator. Equations (6.11) and (6.12) will be familiar to many people as forms of Tikhonov regularization =-=[59, 73, 150]-=- , a technique proposed by Tikhonov and Arsenin in the context of solving Fredholm integral equations of the first kind. Image deconvolution is one example of this class of problem. Another way to thi... |

398 | Generalized belief propagation
- Yedidia, Freeman, et al.
- 2000
(Show Context)
Citation Context ...or each individual character or pair of adjacent characters. Having reduced the configuration space in this way, a stochastic method may be applied more successfully. Mean-field or belief propagation =-=[172]-=- techniques may also offer an efficient method of solution. Learnt image models Part of the attractiveness of the model-based approach is the potential for learning image models from sets of training ... |

389 | Markov Random Field modeling in computer vision
- Li
- 1995
(Show Context)
Citation Context ...een the pair of pixels in C and not upon its orientation. For further details on the modelling of images with Gibbs distributions the reader is referred to Besag [14, 15], Geman and Geman [67] and Li =-=[98]-=-. In the case when the prior is a Gibbs distribution, equation (6.4) becomes f map = arg max f − ∑ ∀C∈C VC(C) − 1 2σ2 ‖Mf − g‖ n 2 (6.7) The difference between the many image priors proposed in the li... |

385 | Bundle Adjustment - A modern synthesis
- Triggs, McLauchlan, et al.
(Show Context)
Citation Context ...berg-Marquardt algorithm which takes advantage of this block-structure to allow efficient computation is described in the Manual of Photogrammetry [141] and also by Hartley [76, 78] and Triggs et al. =-=[162]-=-. However, the bundle-adjustment in this thesis was carried out using an implementation based on more modern approaches to large-scale non-linear optimization [24, 103]. The implementation is detailed... |

350 |
An efficient and accurate camera calibration technique for 3d machine vision
- Tsai
- 1986
(Show Context)
Citation Context ... a set of test images. The calibration image is designed to allow very high accuracy homographies to be computed when it is viewed from different positions. For this purpose we use an 8 × 8 Tsai grid =-=[165]-=- with a 3 : 1 mark–space ratio. Three different test images are then used : two Escher drawings and a Lego scene. The calibration image and three test images are shown in figure 3.10. The panel is vie... |

316 |
Numerical Methods for Unconstrained Optimization and Nonlinear Equations
- jun, Schnabel
- 1983
(Show Context)
Citation Context ...n be minimized with respect to the 8 parameters of H using an iterative, non-linear least-squares optimizer. Efficient algorithms for performing this optimization are based on the Gauss-Newton method =-=[50, 69, 70]-=-, the preferred variant being the Levenberg-Marquardt algorithm [117]. At every iteration, a finite difference approximation may be used to compute the Jacobian of the residuals with respect to each o... |

305 | Limits on super-resolution and how to break them
- Baker, Kanade
- 2002
(Show Context)
Citation Context ...e. Single-image methods are inherently limited by the amount of data available in an image and by the noise present in the image. However, Candocia [26], Freeman & Pasztor [63–66], and Baker & Kanade =-=[10]-=- have proposed Bayesian MAP estimators which utilize sophisticated prior models learnt from training images. These methods attempt to infer high-frequency detail from a single low-resolution image. 20... |

271 | Creating full view panoramic image mosaics and environment maps
- Szeliski, Shum
- 1997
(Show Context)
Citation Context ...here are many references made to spherical panoramic representations in the literature: in image-based rendering [101], wide-baseline 3D scene reconstruction [91, 92] and virtual-reality applications =-=[147]-=-. In each case, image sequences are used which cover part or all of the view-sphere, and the mosaic simply provides a convenient implicit 75representation of the relative placement and orientation of... |

261 |
Improving resolution by image registration
- Irani, Peleg
- 1991
(Show Context)
Citation Context ...ator, its sensitivity to observation noise and modelling error, is explored both empirically and analytically. The method is compared in detail to Irani and Peleg’s classic super-resolution algorithm =-=[86]-=-. Chapter 6: Super-resolution using generic image priors • The use of generic image priors in a Bayesian approach to super-resolution is discussed. The performance of several such models is investigat... |

261 |
Mathematical Statistics and Data Analysis
- Rice
- 1995
(Show Context)
Citation Context ...ence is Pr(g| ˆf) = ∏ Pr(gn| ˆf) (6.1) ∀n where g is the stack of vectors gn. The prior image model provides an additional pdf defined over the space of all images, Pr(f). By invoking Bayes’s theorem =-=[93, 96, 119, 139]-=-, we can combine the two to obtain the posterior distribution : Pr( ˆ f|g) = Pr(g|ˆ f) Pr( ˆ f) Pr(g) (6.2) 179The maximum a-posterior (MAP) estimate of f is then f map = arg max f = arg max f Pr(g| ... |

253 |
Digital image processing
- Pratt, William
- 1978
(Show Context)
Citation Context ...image. Image restoration from single images has generally concentrated on the removal of optical blur and motion blur. A good survey of these methods is given by Gonzalez and Wintz [72], and by Pratt =-=[116]-=-. The blurring process is modelled as convolution of the original image with a blurring operator (also called the point spread function). Hence go(x, y) = gt(x, y) ∗ hd(x, y) + n(x, y) (∗ = convolutio... |

240 | MLESAC: a new robust estimator with application to estimating image geometry
- S, Zisserman
- 2000
(Show Context)
Citation Context ...ion algorithm, it is vital that the line-fitting algorithm be robust to these model-outliers. The algorithm used for this purpose is Torr and Zisserman’s MSAC (M-estimator SAmple Consensus) algorithm =-=[160]-=-, a variation of the previously described RANSAC algorithm. Evidence suggests that this algorithm gives better results than the RANSAC algorithm in problems where the proportion of outliers is high, a... |

238 | A generalized gaussian image model for edge-preserving map estimation
- Bouman, Sauer
- 1993
(Show Context)
Citation Context ...mple of convex, but non-quadratic priors. Schultz and Stevenson use their 2nd derivative scheme with the Huber potentials as a piecewise-smooth image model. Generalized Gaussian MRFs Bouman and Sauer =-=[21, 125]-=-, and also Borman et al. [19, 20] propose a prior in which the image gradients are modelled by a Generalized Gaussian distribution, which is of the form Pr(x) = 1 Z exp ( − xp pσp ) (6.26) When 1 < p ... |

232 |
Projection pursuit
- Huber
- 1985
(Show Context)
Citation Context ...ution is evaluated. Instead of counting the number of inliers, as RANSAC does, MSAC determines the likelihood of the data given the proposed model parameters according to a simple, robust M-estimator =-=[79, 80]-=-. For each data point (i1, i2)n, the distance dn to the proposed line (α, β) is computed, and the overall cost associated with the solution is given by C = ∑ ρ(d 2 n) (3.13) ∀n 49where ρ(d 2 ) = d 2 ... |

230 | A method for the solution of certain non-linear problems in least squares - Levenberg - 1944 |

219 | Computing occluding and transparent motions
- Irani, Rousso, et al.
- 1994
(Show Context)
Citation Context ...e influence of large residuals is reduced, such as the M-estimator approach of Black and Anandan [16]. An alternative and closely related scheme is a multiscale, iteratively re-weighted least-squares =-=[82, 89]-=- approach. However, both of these approaches are computationally expensive : optimizations involving non-convex M-estimators are generally difficult and are not guaranteed to converge to a global opti... |

210 | Extraction of highresolution frames from video sequences
- Schultz, Stevenson
- 1996
(Show Context)
Citation Context ...mposing a first order smoothness constraint in the reconstructed high-resolution image. In this case the PSF can be obtained from the bench calibration of the satellite optics. 19Schultz & Stevenson =-=[132]-=- propose a MAP estimation method which differs from previous ones in two significant respects. Firstly, it does not use a geometric registration method, but a general full, dense correspondence method... |

209 |
Computing a trust region step
- Moré, Sorensen
- 1983
(Show Context)
Citation Context ... Hartley [76, 78] and Triggs et al. [162]. However, the bundle-adjustment in this thesis was carried out using an implementation based on more modern approaches to large-scale non-linear optimization =-=[24, 103]-=-. The implementation is detailed in appendix A. 4.4.3 A novel algorithm for feature-matching over N-views We now address the problem of exactly how to go about finding the N-view matches, Mj. Initiall... |

203 | 3D Model Acquisition from Extended Image Sequences
- Beardsley, Torr, et al.
- 1996
(Show Context)
Citation Context ...rrespondences. For certain types of images these point correspondences can be obtained automatically by matching image features such as interest points (such as generated by a Harris corner detector) =-=[13]-=-, and employing efficient non-linear optimization techniques to compute the required transformation. These feature based method have proved time and again to be extremely accurate, robust and efficien... |

188 |
Multiframe image restoration and registration
- Tsai, Huang
- 1984
(Show Context)
Citation Context ... sampling theorem [168]. The drawback with these methods is that they do not allow for blur or noise in the observed images. Another popular approach is to perform restoration in the frequency domain =-=[153, 164]-=-. These methods can deal with spatially invariant blur and noisy observa16tions, but the motion model is limited to pure translation. More recent approaches have used Projection onto Convex Sets (POC... |

165 | A nonlinear primal-dual method for total variation-based image restoration
- Golub, Mulet
- 1999
(Show Context)
Citation Context ...ensity. Each resampling location takes a weighted average of nearby samples, weighted according to distance (w = e −|δx| ). 5. Finally, deblur the resulting image using a standard single-image method =-=[33, 170]-=-. Figure 5.26 illustrates the process. low-resolution images registered images resampling kernels deblurred image resample deblur Figure 5.26: Schematic representation of Rudin’s forward-projection su... |

162 | Single view metrology
- Criminisi, Reid, et al.
(Show Context)
Citation Context ...gments, or by including a simple parametric model of the lens distortion in the geometric registration procedure. The former method is used in the context of single view metrology by Criminisi et al. =-=[39, 40]-=-. The latter method is used in the context of image mosaicing by Sawhney & Kumar [127] and by Dellaert et al. [48]. A detailed analysis of the lens distortion problem is performed by Devernay & Fauger... |

162 | Image mosaicing for tele-reality applications
- Szeliski
- 1994
(Show Context)
Citation Context ...ultiscale approach to estimate a quadratic approximation by minimizing an error based on projective flow. The multi-scale approach allows these algorithm to work for large image disparities. Szeliski =-=[146, 148]-=- computes the full 8 dof homography between images using a multiscale, iterative method. He uses a Gaussian pyramid and the Levenberg-Marquardt [117] algorithm to perform the estimation (see also appe... |

148 | Self-calibration from multiple views with a rotating camera
- Hartley
- 1994
(Show Context)
Citation Context .... Evidently, this method requires an estimate of the camera calibration matrices Kn. A non-linear method for accurately estimating the internal parameters of a rotating camera is described by Hartley =-=[76, 78]-=-. However, it is not essential for the calibration to be 74cylindrical reprojection manifold plan view optic centre Figure 4.4: Schematic illustration of mosaic rendering by reprojection onto a calib... |

147 |
The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind
- Groetsch
- 1984
(Show Context)
Citation Context ...e matrix L is typically chosen to be a discrete approximation of a first or second derivative operator. Equations (6.11) and (6.12) will be familiar to many people as forms of Tikhonov regularization =-=[59, 73, 150]-=- , a technique proposed by Tikhonov and Arsenin in the context of solving Fredholm integral equations of the first kind. Image deconvolution is one example of this class of problem. Another way to thi... |

147 |
Mosaic based representations of video sequences and their applications
- Irani, Anandan, et al.
- 1995
(Show Context)
Citation Context ... scanning [34, 133]. Under telephoto viewing conditions, where perspective effects such as the convergence of imaged parallel lines are negligible, images are related by a 6 dof affine transformation =-=[34, 84]-=-. Both similarity and affinity mappings are perfectly correct and valid transformations under certain imaging conditions, they are subgroups of a planar homography. By contrast, some authors have work... |

138 | Statistical models for images: compression restoration and synthesis
- Simoncelli
- 1997
(Show Context)
Citation Context ...(ω) −1 ⇒ PF (ω) ∝ 1 ω 2 Hence, roughly speaking, this GMRF is assuming that the signal power in f decays as 1 ω 2 . This is in agreement with investigations regarding the statistics of natural scenes =-=[105, 140]-=-, where it is observed that the power in the frequency spectrum decays as approximately 1 ω 2 . However, it has also been noted that it is phase coherence in a signal which is important in defining st... |

128 | Super-resolution Video Reconstruction with Arbitrary Sampling Latices and Nonzero Aperture Time
- Patti, Sezan, et al.
- 1997
(Show Context)
Citation Context ...on model, such as pure translation [10, 94, 104, 106, 114, 152, 164, 168], Euclidean [34, 54, 86] or affine [11, 87]. Others have chosen a more general motion model such as full dense stereo matching =-=[108, 132, 137, 138]-=-. It is also common to find other forms of point-spread function, such as a rectangular window [28], or a kernel extracted directly from calibration images [86, 99], or even from the observed image da... |

121 | R.: “3-d scene data recovery using omnidirectional multibaseline stereo
- Kang, Szeliski
- 1997
(Show Context)
Citation Context ...render views with arbitrary gaze directions and camera zoom. Excellent reviews of image mosaicing and its many applications are given in [83, 84, 95]. Some notable examples are due to Kang & Szeliski =-=[91, 92]-=- who use mosaics composed of a hemisphere of images to represent the view in every direction at a particular point in the world. By capturing mosaics at many points, and matching image features across... |

121 | Panoramic Mosaics by manifold projection
- Peleg, Herman
- 1997
(Show Context)
Citation Context ...l scenes and camera motions with varying degrees of success. Some of these methods are broadly based on simulating the linear push-broom cameras used in satellite imaging. Rousso [121, 122] and Peleg =-=[109, 112, 115]-=- consider mosaics composed of strips extracted from the input images. The strips are chosen such that the direction of optic flow is orthogonal to the axis of the strip. Using this method, and with su... |

117 | Virtual bellows: Constructing high-quality images from video
- Mann, Picard
- 1994
(Show Context)
Citation Context ..., but for reasons of numerical stability and/or simplicity of optimization have chosen to approximate the homography by a Taylor expansion to second-order, resulting in the biquadratic transformation =-=[95, 99, 100, 173]-=-. This has 12 dof, but is unable to correctly model perspective effects. The expansion is given in the following equations. x ′ = q x ′ x 2x 2 + qx ′ xyxy + q x ′ y 2y 2 + qx ′ xx + qx ′ yy + qx ′ y ′... |

114 | Video indexing based on mosaic representations
- Irani, Anandan
- 1998
(Show Context)
Citation Context ...ng standard techniques to form a very compact representation of the sequence — a point capitalized on in MPEG4. Further examples of off-line applications include event synopsis [83] and video-editing =-=[9, 81]-=-. Event synopsis involves constructing a static mosaic and significant residuals and then overlaying the residuals for a number of frames onto the mosaic simultaneously. This creates a synopsis of for... |

111 | Robust video mosaicing through topology inference and local to global alignment
- Sawhney, Hsu, et al.
- 1998
(Show Context)
Citation Context ...nal mosaic. Davis [46] solves a linear system, derived from a redundant set of pairwise homographies, so as to minimize an algebraic residual defined over the actual H matrix elements. Sawhney et al. =-=[128, 129]-=- propose a scheme in which the mosaic image is updated one frame at a time, and each additional frame is registered with and blended into the current mosaic image. Although these are all very practica... |

110 |
Detecting and tracking multiple moving objects using temporal integration
- Irani, Rousso, et al.
- 1992
(Show Context)
Citation Context ...s to use a temporal-median filter when combining the images. This removes the effect of temporary occlusions provided that the true background is unoccluded in more than half the frames. Irani et al. =-=[87, 88]-=- 70registration H01 H12 H0 H1 H2 reprojection (0,0) T blending Figure 4.1: The three basic steps in forming a mosaic representation of an image sequence and rendering a novel view. In this example, t... |