Results 1  10
of
35
Deterministic Annealing for Clustering, Compression, Classification, Regression, and Related Optimization Problems
 Proceedings of the IEEE
, 1998
"... this paper. Let us place it within the neural network perspective, and particularly that of learning. The area of neural networks has greatly benefited from its unique position at the crossroads of several diverse scientific and engineering disciplines including statistics and probability theory, ph ..."
Abstract

Cited by 248 (11 self)
 Add to MetaCart
this paper. Let us place it within the neural network perspective, and particularly that of learning. The area of neural networks has greatly benefited from its unique position at the crossroads of several diverse scientific and engineering disciplines including statistics and probability theory, physics, biology, control and signal processing, information theory, complexity theory, and psychology (see [45]). Neural networks have provided a fertile soil for the infusion (and occasionally confusion) of ideas, as well as a meeting ground for comparing viewpoints, sharing tools, and renovating approaches. It is within the illdefined boundaries of the field of neural networks that researchers in traditionally distant fields have come to the realization that they have been attacking fundamentally similar optimization problems.
A New Point Matching Algorithm for NonRigid Registration
, 2002
"... Featurebased methods for nonrigid registration frequently encounter the correspondence problem. Regardless of whether points, lines, curves or surface parameterizations are used, featurebased nonrigid matching requires us to automatically solve for correspondences between two sets of features. I ..."
Abstract

Cited by 235 (2 self)
 Add to MetaCart
Featurebased methods for nonrigid registration frequently encounter the correspondence problem. Regardless of whether points, lines, curves or surface parameterizations are used, featurebased nonrigid matching requires us to automatically solve for correspondences between two sets of features. In addition, there could be many features in either set that have no counterparts in the other. This outlier rejection problem further complicates an already di#cult correspondence problem. We formulate featurebased nonrigid registration as a nonrigid point matching problem. After a careful review of the problem and an indepth examination of two types of methods previously designed for rigid robust point matching (RPM), we propose a new general framework for nonrigid point matching. We consider it a general framework because it does not depend on any particular form of spatial mapping. We have also developed an algorithmthe TPSRPM algorithmwith the thinplate spline (TPS) as the parameterization of the nonrigid spatial mapping and the softassign for the correspondence. The performance of the TPSRPM algorithm is demonstrated and validated in a series of carefully designed synthetic experiments. In each of these experiments, an empirical comparison with the popular iterated closest point (ICP) algorithm is also provided. Finally, we apply the algorithm to the problem of nonrigid registration of cortical anatomical structures which is required in brain mapping. While these results are somewhat preliminary, they clearly demonstrate the applicability of our approach to real world tasks involving featurebased nonrigid registration.
Optimization and Dynamical Systems
, 1994
"... researchers in the areas of optimization, dynamical systems, control systems, signal processing, and linear algebra. The motivation for the results developed here arises from advanced engineering applications and the emergence of highly parallel computing machines for tackling such applications. The ..."
Abstract

Cited by 140 (18 self)
 Add to MetaCart
researchers in the areas of optimization, dynamical systems, control systems, signal processing, and linear algebra. The motivation for the results developed here arises from advanced engineering applications and the emergence of highly parallel computing machines for tackling such applications. The problems solved are those of linear algebra and linear systems theory, and include such topics as diagonalizing a symmetric matrix, singular value decomposition, balanced realizations, linear programming, sensitivity minimization, and eigenvalue assignment by feedback control. The tools are those, not only of linear algebra and systems theory, but also of differential geometry. The problems are solved via dynamical systems implementation, either in continuous time or discrete time, which is ideally suited to distributed parallel processing. The problems tackled are indirectly or directly concerned with dynamical systems themselves, so there is feedback in that dynamical systems are used to understand and optimize dynamical systems. One key to the new research results has been
Computable elastic distances between shapes
 SIAM J. of Applied Math
, 1998
"... Abstract. We define distances between geometric curves by the square root of the minimal energy required to transform one curve into the other. The energy is formally defined from a left invariant Riemannian distance on an infinite dimensional group acting on the curves, which can be explicitly comp ..."
Abstract

Cited by 120 (19 self)
 Add to MetaCart
Abstract. We define distances between geometric curves by the square root of the minimal energy required to transform one curve into the other. The energy is formally defined from a left invariant Riemannian distance on an infinite dimensional group acting on the curves, which can be explicitly computed. The obtained distance boils down to a variational problem for which an optimal matching between the curves has to be computed. An analysis of the distance when the curves are polygonal leads to a numerical procedure for the solution of the variational problem, which can efficiently be implemented, as illustrated by experiments.
Unsupervised Texture Segmentation in a Deterministic Annealing Framework
, 1998
"... We present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a data clustering problem based on sparse proximity data. Dissimilarities of pairs of textured regions are computed from ..."
Abstract

Cited by 91 (9 self)
 Add to MetaCart
We present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a data clustering problem based on sparse proximity data. Dissimilarities of pairs of textured regions are computed from a multiscale Gabor filter image representation. We discuss and compare a class of clustering objective functions which is systematically derived from invariance principles. As a general optimization framework we propose deterministic annealing based on a meanfield approximation. The canonical way to derive clustering algorithms within this framework as well as an efficient implementation of meanfield annealing and the closely related Gibbs sampler are presented. We apply both annealing variants to Brodatzlike microtexture mixtures and realword images.
New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence
"... A fundamental open problem in computer visiondetermining pose and correspondence between two sets of points in spaceis solved with a novel, fast [O(nm)], robust and easily implementable algorithm. The technique works on noisy 2D or 3D point sets that may be of unequal sizes and may differ by n ..."
Abstract

Cited by 85 (19 self)
 Add to MetaCart
A fundamental open problem in computer visiondetermining pose and correspondence between two sets of points in spaceis solved with a novel, fast [O(nm)], robust and easily implementable algorithm. The technique works on noisy 2D or 3D point sets that may be of unequal sizes and may differ by nonrigid transformations. Using a combination of optimization techniques such as deterministic annealing and the softassign, which have recently emerged out of the recurrent neural network/statistical physics framework, analog objective functions describing the problems are minimized. Over thirty thousand experiments, on randomly generated points sets with varying amounts of noise and missing and spurious points, and on handwritten character sets demonstrate the robustness of the algorithm. Keywords: Pointmatching, pose estimation, correspondence, neural networks, optimization, softassign, deterministic annealing, affine. 1 Introduction Matching the representations of two images has long...
Vector Quantization with Complexity Costs
, 1993
"... Vector quantization is a data compression method where a set of data points is encoded by a reduced set of reference vectors, the codebook. We discuss a vector quantization strategy which jointly optimizes distortion errors and the codebook complexity, thereby, determining the size of the codebook. ..."
Abstract

Cited by 54 (18 self)
 Add to MetaCart
Vector quantization is a data compression method where a set of data points is encoded by a reduced set of reference vectors, the codebook. We discuss a vector quantization strategy which jointly optimizes distortion errors and the codebook complexity, thereby, determining the size of the codebook. A maximum entropy estimation of the cost function yields an optimal number of reference vectors, their positions and their assignment probabilities. The dependence of the codebook density on the data density for different complexity functions is investigated in the limit of asymptotic quantization levels. How different complexity measures influence the efficiency of vector quantizers is studied for the task of image compression, i.e., we quantize the wavelet coefficients of gray level images and measure the reconstruction error. Our approach establishes a unifying framework for different quantization methods like Kmeans clustering and its fuzzy version, entropy constrained vector quantizati...
EM procedures using mean fieldlike approximations for Markov modelbased image segmentation
, 2001
"... This paper deals with Markov random field modelbased image segmentation. This involves parameter estimation in hidden Markov models for which one of the most widely used procedures is the EM algorithm. In practice, difficulties arise due to the dependence structure in the models and approximations ..."
Abstract

Cited by 46 (11 self)
 Add to MetaCart
This paper deals with Markov random field modelbased image segmentation. This involves parameter estimation in hidden Markov models for which one of the most widely used procedures is the EM algorithm. In practice, difficulties arise due to the dependence structure in the models and approximations are required to make the algorithm tractable. We propose a class of algorithms in which the idea is to deal with systems of independent variables. This corresponds to approximations of the pixels' interactions similar to the mean field approximation. It follows algorithms that have the advantage of taking the Markovian structure into account while preserving the good features of EM. In addition, this class, that includes new and already known procedures, is presented in a unified framework, showing that apparently distant algorithms come from similar approximation principles. We illustrate the algorithms performance on synthetic and real images. These experiments point out the ability of o...
The Invisible Hand Algorithm: Solving the Assignment Problem With Statistical Physics
, 1994
"... We propose a novel method for solving the assignment problem using techniques adapted from statistical physics. We derive a convex effective energy function whose unique minimum corresponds to the optimal assignment. Steepest descent results in a continuoustime dynamical system that is guaranteed t ..."
Abstract

Cited by 45 (4 self)
 Add to MetaCart
We propose a novel method for solving the assignment problem using techniques adapted from statistical physics. We derive a convex effective energy function whose unique minimum corresponds to the optimal assignment. Steepest descent results in a continuoustime dynamical system that is guaranteed to converge arbitrarily close to the optimal solution. Our algorithm has an appealing economic interpretation and has very interesting connections to the discrete auction algorithm proposed by Bertsekas. We also derive an alternative discrete algorithm for minimizing the effective energy based on a theorem by Sinkhorn.
Latent Variable Models for Neural Data Analysis
, 1999
"... The brain is perhaps the most complex system to have ever been subjected to rigorous scientific investigation. The scale is staggering: over 1011 neurons, each making an average of 10 3 synapses, with computation occurring on scales ranging from a single dendritic spine, to an entire cortical area. ..."
Abstract

Cited by 42 (5 self)
 Add to MetaCart
The brain is perhaps the most complex system to have ever been subjected to rigorous scientific investigation. The scale is staggering: over 1011 neurons, each making an average of 10 3 synapses, with computation occurring on scales ranging from a single dendritic spine, to an entire cortical area. Slowly, we are beginning to acquire experimental tools that can gather the massive amounts of data needed to characterize this system. However, to understand and interpret these data will also require substantial strides in inferential and statistical techniques. This dissertation attempts to meet this need, extending and applying the modern tools of latent variable modeling to problems in neural data analysis. It is divided