Abstract — A new approach to the problem of multimodality medical image registration is proposed, using a basic concept from information theory, mutual information (MI), or relative entropy, as a new matching criterion. The method presented in this paper applies MI to measure the statistical dependence or information redundancy between the image intensities of corresponding voxels in both images, which is assumed to be maximal if the images are geometrically aligned. Maximization of MI is a very general and powerful criterion, because no assumptions are made regarding the nature of this dependence and no limiting constraints are imposed on the image content of the modalities involved. The accuracy of the MI criterion is validated for rigid body registration of computed tomography (CT), magnetic resonance (MR), and photon emission tomography (PET) images by comparison with the stereotactic registration solution, while robustness is evaluated with respect to implementation issues, such as interpolation and optimization, and image content, including partial overlap and image degradation. Our results demonstrate that subvoxel accuracy with respect to the stereotactic reference solution can be achieved completely automatically and without any prior segmentation, feature extraction, or other preprocessing steps which makes this method very well suited for clinical applications. Index Terms—Matching criterion, multimodality images, mutual information, registration. I.

A new information-theoretic approach is presented for finding the registration of volumetric medical images of differing modalities. Registration is achieved by adjustment of the relative pose until the mutual information between images is maximized. In our derivation of the registration procedure, few assumptions are made about the nature of the imaging process. As a result the algorithms are quite general and can foreseeably be used with a wide variety of imaging devices. This approach works directly with raw images; no preprocessing or feature detection is required. As opposed to feature-based techniques, all of the information in the scan is used to evaluate the registration. This technique is however more flexible and robust than other intensity based techniques like correlation. Additionally, it has an efficient implementation that is based on stochastic approximation. Experiments are presented that demonstrate the approach registering magnetic resonance (MR) images with comput...

Abstract—An overview is presented of the medical image processing literature on mutual-information-based registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application. Methods are classified according to the different aspects of mutual-information-based registration. The main division is in aspects of the methodology and of the application. The part on methodology describes choices made on facets such as preprocessing of images, gray value interpolation, optimization, adaptations to the mutual information measure, and different types of geometrical transformations. The part on applications is a reference of the literature available on different modalities, on interpatient registration and on different anatomical objects. Comparison studies including mutual information are also considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on past achievements and some future challenges. Index Terms—Image registration, literature survey, matching, mutual information. I.

Abstract not found

Over the last five years, new "voxel-based" approaches have allowed important progress in multimodal image registration, notably due to the increasing use of information-theoretic similarity measures.

This paper presents a new method for image registration based on jointly estimating the forward and reverse transformations between two images while constraining these transforms to be inverses of one another. This approach produces a consistent set of transformations that have less pairwise registration error, i.e., better correspondence, than traditional methods that estimate the forward and reverse transformations independently. The transformations are estimated iteratively and are restricted to preserve topology by constraining them to obey the laws of continuum mechanics. The transformations are parameterized by a Fourier series to diagonalize the covariance structure imposed by the continuum mechanics constraints and to provide a computationally efficient numerical implementation. Results using a linear elastic material constraint are presented using both magnetic resonance and X-ray computed tomography image data. The results show that the joint estimation of a consistent set of forward and reverse transformations constrained by linear-elasticity give better registration results than using either constraint alone or none at all.

We propose a new method for the intermodal registration of images using a criterion known as mutual information. Our main contribution is an optimizer that we specifically designed for this criterion. We show that this new optimizer is well adapted to a multiresolution approach because it typically converges in fewer criterion evaluations than other optimizers. We have built a multiresolution image pyramid, along with an interpolation process, an optimizer, and the criterion itself, around the unifying concept of spline-processing. This ensures coherence in the way we model data and yields good performance. We have tested our approach in a variety of experimental conditions and report excellent results. We claim an accuracy of about a hundredth of a pixel under ideal conditions. We are also robust since the accuracy is still about a tenth of a pixel under very noisy conditions. In addition, a blind evaluation of our results compares very favorably to the work of several other researchers.

While intensity-based similarity measures are increasingly used for medical image registration, they often rely on implicit assumptions regarding the imaging physics. The motivation of this paper is to clarify the assumptions on which a number of popular similarity measures rely. After formalizing registration based on general image acquisition models, we show that the search for an optimal measure can be cast into a maximum likelihood estimation problem. We then derive similarity measures corresponding to different modeling assumptions and retrieve some well-known measures (correlation coefficient, correlation ratio, mutual information). Finally, we present results of rigid registration between several image modalities to illustrate the importance of choosing an appropriate similarity measure.

Two new consistent image registration algorithms are presented: one is based on matching corresponding landmarks and the other is based on matching both landmark and intensity information. The consistent landmark and intensity registration algorithm produces good correspondences between images near landmark locations by matching corresponding landmarks and away from landmark locations by matching the image intensities. In contrast to similar unidirectional algorithms, these new consistent algorithms jointly estimate the forward and reverse transformation between two images while minimizing the inverse consistency error -- the error between the forward (reverse) transformation and the inverse of the the reverse (forward) transformation. This reduces the ambiguous correspondence between the forward and reverse transformations associated with large inverse consistency errors. In both algorithms a thin-plate spline (TPS) model is used to regularize the estimated transformations. Two-dimensional (2-D) examples are presented that show the inverse consistency error produced by the traditional unidirectional landmark TPS algorithm can be relatively large and that this error is minimized using the consistent landmark algorithm. Results using 2-D magnetic resonance imaging data are presented that demonstrate that using landmark and intensity information together produce better correspondence between medical images than using either landmarks or intensity information alone.

The notion of using context information for solving high-level vision and medical image segmentation problems has been increasingly realized in the field. However, how to learn an effective and efficient context model, together with an image appearance model, remains mostly unknown. The current literature using Markov Random Fields (MRFs) and Conditional Random Fields (CRFs) often involves specific algorithm design, in which the modeling and computing stages are studied in isolation. In this paper, we propose the auto-context algorithm. Given a set of training images and their corresponding label maps, we first learn a classifier on local image patches. The discriminative probability (or classification confidence) maps created by the learned classifier are then used as context information, in addition to the original image patches, to train a new classifier. The algorithm then iterates until convergence. Auto-context integrates low-level and context information by fusing a large number of low-level appearance features with context and implicit shape information. The resulting discriminative algorithm is general and easy to implement. Under nearly the same parameter settings in training, we apply the algorithm to three challenging vision applications: foreground/background segregation, human body configuration estimation, and scene region labeling. Moreover, context also plays a very important role in medical/brain images where the anatomical structures are mostly constrained to relatively fixed positions. With only some slight changes resulting from using 3D instead of 2D features, the auto-context algorithm applied to brain MRI image segmentation is shown to outperform state-of-the-art algorithms specifically designed for this domain. Furthermore, the scope of the proposed algorithm goes beyond image analysis and it has the potential to be used for a wide variety of problems in multi-variate labeling.