This paper aims to present a review of recent as well as classic image registration methods. Image registration is the process of overlaying images (two or more) of the same scene taken at different times, from different viewpoints, and/or by different sensors. The registration geometrically align two images (the reference and sensed images). The reviewed approaches are classified according to their nature (areabased and feature-based) and according to four basic steps of image registration procedure: feature detection, feature matching, mapping function design, and image transformation and resampling. Main contributions, advantages, and drawbacks of the methods are mentioned in the paper. Problematic issues of image registration and outlook for the future research are discussed too. The major goal of the paper is to provide a comprehensive reference source for the researchers involved in image registration, regardless of particular application areas. q 2003 Elsevier B.V. All rights reserved.

Feature-based methods for non-rigid registration frequently encounter the correspondence problem. Regardless of whether points, lines, curves or surface parameterizations are used, feature-based non-rigid matching requires us to automatically solve for correspondences between two sets of features. In addition, there could be many features in either set that have no counterparts in the other. This outlier rejection problem further complicates an already di#cult correspondence problem. We formulate feature-based non-rigid registration as a non-rigid point matching problem. After a careful review of the problem and an in-depth examination of two types of methods previously designed for rigid robust point matching (RPM), we propose a new general framework for non-rigid point matching. We consider it a general framework because it does not depend on any particular form of spatial mapping. We have also developed an algorithm---the TPS-RPM algorithm---with the thin-plate spline (TPS) as the parameterization of the non-rigid spatial mapping and the softassign for the correspondence. The performance of the TPS-RPM algorithm is demonstrated and validated in a series of carefully designed synthetic experiments. In each of these experiments, an empirical comparison with the popular iterated closest point (ICP) algorithm is also provided. Finally, we apply the algorithm to the problem of non-rigid registration of cortical anatomical structures which is required in brain mapping. While these results are somewhat preliminary, they clearly demonstrate the applicability of our approach to real world tasks involving feature-based non-rigid registration.

Facial expression passes through nonlinear shape and appearance deformations with variations in different people and expressions. We present nonlinear shape and appearance models for facial expression analysis and synthesis using nonlinear generative models for different facial expressions in different people. To achieve accurate shape normalized appearance models, we utilize nonlinear warping using thin plate spline (TPS). A novel nonlinear generative model using conceptual manifold embedding and empirical kernel maps for facial expressions provides facial shape and appearance samples according to the configuration, personal style, and expression parameters. We can recognize facial expressions based on estimated facial expression parameters after iterative estimations of facial expression and style. In addition, the model provides accurate synthesis of facial expression sequences even with high nonlinear deformations of shape and appearance during facial expressions. 1.

Abstract–Transformation functions play a major role in nonrigid image registration. In this pa-per, the characteristics of thin-plate spline (TPS), multiquadric (MQ), piecewise linear (PL), and weighted mean (WM) transformations are explored and their performances in nonrigid image reg-istration are compared. TPS and MQ are found to be most suitable when the set of control-point correspondences is not large (fewer than a thousand) and variation in spacing between the control points is not large. When spacing between the control points varies greatly, PL is found to produce a more accurate registration than TPS and MQ. When a very large set of control points is given and the control points contain positional inaccuracies, WM is preferred over TPS, MQ, and PL because it uses an averaging process that smoothes the noise and does not require the solution of a very large system of equations. Use of transformation functions in the detection of incorrect correspondences is also discussed. Index Terms–Image registration, transformation function, thin-plate spline, multiquadric, radial basis functions, piecewise linear, weighted-mean

Non-rigid registration of a pair of images depends on the generation of a dense deformation field across one of the images. Such deformation fields can be represented by the deformation of a set of knotpoints, interpolated to produce the continuous deformation field.

Abstract — We show that the Fisher-Rao Riemannian metric is a natural, intrinsic tool for computing shape geodesics. When a parameterized probability density function is used to represent a landmark-based shape, the modes of deformation are automatically established through the Fisher information of the density. Consequently, given two shapes parameterized by the same density model, the geodesic distance between them under the action of the Fisher-Rao metric is a convenient shape distance measure. It has the advantage of being an intrinsic distance measure and invariant to reparameterization. We first model shape landmarks using a Gaussian mixture model and then compute geodesic distances between two shapes using the Fisher-Rao metric corresponding to the mixture model. We illustrate our approach by computing Fisher geodesics between 2D corpus callosum shapes. Shape representation via the mixture model and shape deformation via the Fisher geodesic are hereby unified in this approach. I.

Abstract. Shape matching plays a prominent role in the analysis of medical and biological structures. Recently, a unifying framework was introduced for shape matching that uses mixture-models to couple both the shape representation and deformation. Essentially, shape distances were defined as geodesics induced by the Fisher-Rao metric on the manifold of mixture-model represented shapes. A fundamental drawback of the Fisher-Rao metric is that it is NOT available in closed-form for the mixture model. Consequently, shape comparisons are computationally very expensive. Here, we propose a new Riemannian metric based on generalized φ- entropy measures. In sharp contrast to the Fisher-Rao metric, our new metric is available in closed-form. Geodesic computations using the new metric are considerably more efficient. Discriminative capabilities of this new metric are studied by pairwise matching of corpus callosum shapes. Comparisons are conducted with the Fisher-Rao metric and the thin-plate spline bending energy. 1

In memory of my grandfather, Isadore Shore. May he rest in peace. ii ACKNOWLEDGEMENTS I would like to express sincerest thanks to my advisor, Prof. Jeff Fessler, for his guid-ance and support, as well as for many enjoyable and thought-provoking discussions. Work-ing for him has been a true privilege. I would also like to thank those who have served on my committee, Profs. Alfred Hero, Charles Meyer, Valen Johnson, and Romesh Saigal, for their efforts and contributions to the fruition of my dissertation. I further wish to acknowledge the many colleagues I have had throughout the years,

Abstract—The aim of this paper is to achieve seamless image stitching without producing visual artifact caused by severe intensity discrepancy and structure misalignment, given that the input images are roughly aligned or globally registered. Our new approach is based on structure deformation and propagation for achieving the overall consistency in image structure and intensity. The new stitching algorithm, which has found applications in image compositing, image blending, and intensity correction, consists of the following main processes. Depending on the compatibility and distinctiveness of the 2D features detected in the image plane, single or double optimal partitions are computed subject to the constraints of intensity coherence and structure continuity. Afterwards, specific 1D features are detected along the computed optimal partitions from which a set of sparse deformation vectors is derived to encode 1D feature matching between the partitions. These sparse deformation cues are robustly propagated into the input images by solving the associated minimization problem in gradient domain, thus providing a uniform framework for the simultaneous alignment of image structure and intensity. We present results in general image compositing and blending in order to show the effectiveness of our method in producing seamless stitching results from complex input images. Index Terms—Image stitching, structure deformation, image alignment. 1

Abstract. Shape matching plays a prominent role in the comparison of similar structures. We present a unifying framework for shape matching that uses mixture-models to couple both the shape repre-sentation and deformation. The theoretical foundation is drawn from information geometry wherein information matrices are used to establish intrinsic distances between parametric densities. When a parameterized probability density function is used to represent a landmark-based shape, the modes of deformation are automatically established through the information matrix of the density. We first show that given two shapes parameterized by Gaussian mixture models, the well known Fisher information matrix of the mixture model is also a Riemannian metric (actually the Fisher-Rao Riemannian metric) and can therefore be used for computing shape geodesics. The Fisher-Rao metric has the advantage of being an intrinsic metric and invariant to reparameterization. The geodesic—computed using this metric—establishes an intrinsic deformation between the shapes, thus unifying both shape representa-tion and deformation. A fundamental drawback of the Fisher-Rao metric is that it is NOT available in closed-form for the Gaussian mixture model. Consequently, shape comparisons are computation-ally very expensive. To address this, we develop a new Riemannian metric based on generalized φ-entropy measures. In sharp contrast to the Fisher-Rao metric, the new metric is available in closed-form. Geodesic computations using the new metric are considerably more efficient. We validate the performance and discriminative capabilities of these new information geometry based metrics by pair-wise matching of corpus callosum shapes. A comprehensive comparative analysis is also provided using other landmark based distances, including the Hausdorff distance, the Procrustes metric, landmark based diffeomorphisms, and the bending energies of the thin-plate (TPS) and Wendland splines. 1