Results 1  10
of
11
Unsupervised Texture Segmentation in a Deterministic Annealing Framework
, 1998
"... We present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a data clustering problem based on sparse proximity data. Dissimilarities of pairs of textured regions are computed from ..."
Abstract

Cited by 91 (9 self)
 Add to MetaCart
We present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a data clustering problem based on sparse proximity data. Dissimilarities of pairs of textured regions are computed from a multiscale Gabor filter image representation. We discuss and compare a class of clustering objective functions which is systematically derived from invariance principles. As a general optimization framework we propose deterministic annealing based on a meanfield approximation. The canonical way to derive clustering algorithms within this framework as well as an efficient implementation of meanfield annealing and the closely related Gibbs sampler are presented. We apply both annealing variants to Brodatzlike microtexture mixtures and realword images.
A Theory of Proximity Based Clustering: Structure Detection by Optimization
 Pattern Recognition
, 1999
"... In this paper, a systematic optimization approach for clustering proximity or similarity data is developed. Starting from fundamental invariance and robustness properties, a set of axioms is proposed and discussed to distinguish different cluster compactness and separation criteria. The approach cov ..."
Abstract

Cited by 34 (8 self)
 Add to MetaCart
In this paper, a systematic optimization approach for clustering proximity or similarity data is developed. Starting from fundamental invariance and robustness properties, a set of axioms is proposed and discussed to distinguish different cluster compactness and separation criteria. The approach covers the case of sparse proximity matrices, and is extended to nested partitionings for hierarchical data clustering. To solve the associated optimization problems, a rigorous mathematical framework for deterministic annealing and meanfield approximation is presented. Efficient optimization heuristics are derived in a canonical way, which also clarifies the relation to stochastic optimization by Gibbs sampling. Similaritybased clustering techniques have a broad range of possible applications in computer vision, pattern recognition, and data analysis. As a major practical application we present a novel approach to the problem of unsupervised texture segmentation, which relies on statistical...
A Global Optimization Technique for Statistical Classifier Design
 IEEE Transactions on Signal Processing
"... A global optimization method is introduced for the design of statistical classifiers that minimize the rate of misclassification. We first derive the theoretical basis for the method, based on which we develop a novel design algorithm and demonstrate its effectiveness and superior performance in the ..."
Abstract

Cited by 25 (9 self)
 Add to MetaCart
A global optimization method is introduced for the design of statistical classifiers that minimize the rate of misclassification. We first derive the theoretical basis for the method, based on which we develop a novel design algorithm and demonstrate its effectiveness and superior performance in the design of practical classifiers for some of the most popular structures currently in use. The method, grounded in ideas from statistical physics and information theory, extends the deterministic annealing approach for optimization, both to incorporate structural constraints on data assignments to classes and to minimize the probability of error as the cost objective. During the design, data are assigned to classes in probability, so as to minimize the expected classification error given a specified level of randomness, as measured by Shannon's entropy. The constrained optimization is equivalent to a free energy minimization, motivating a deterministic annealing approach in which the entropy...
Deterministic annealing for semisupervised kernel machines
 In International Conference on Machine Learning
, 2006
"... An intuitive approach to utilizing unlabeled data in kernelbased classification algorithms is to simply treat unknown labels as additional optimization variables. For marginbased loss functions, one can view this approach as attempting to learn lowdensity separators. However, this is a hard optimi ..."
Abstract

Cited by 21 (9 self)
 Add to MetaCart
An intuitive approach to utilizing unlabeled data in kernelbased classification algorithms is to simply treat unknown labels as additional optimization variables. For marginbased loss functions, one can view this approach as attempting to learn lowdensity separators. However, this is a hard optimization problem to solve in typical semisupervised settings where unlabeled data is abundant. The popular Transductive SVM algorithm is a labelswitchingretraining procedure that is known to be susceptible to local minima. In this paper, we present a global optimization framework for semisupervised Kernel machines where an easier problem is parametrically deformed to the original hard problem and minimizers are smoothly tracked. Our approach is motivated from deterministic annealing techniques and involves a sequence of convex optimization problems that are exactly and efficiently solved. We present empirical results on several synthetic and real world datasets that demonstrate the effectiveness of our approach. 1.
A deterministic annealing framework for unsupervised texture segmentation
 Tech. Rep. IAITR962
, 1996
"... We present a novel framework for unsupervised texture segmentation, which relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a pairwise data clustering problem with a sparse neighborhood structure. The pairwise dissimilarities of texture blocks are compute ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
We present a novel framework for unsupervised texture segmentation, which relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a pairwise data clustering problem with a sparse neighborhood structure. The pairwise dissimilarities of texture blocks are computed using a multiscale image representation based on Gabor filters, which are tuned to spatial frequencies at different scales and orientations. We derive and discuss a family of objective functions to pose the segmentation problem in a precise mathematical formulation. An efficient optimization method, known as deterministic annealing, is applied to solve the associated optimization problem. The general framework of deterministic annealing and meanfield approximation is introduced and the canonical way to derive efficient algorithms within this framework is described in detail. Moreover the combinatorial optimization problem is examined from the viewpoint of scale space theory. The novel algorithm has been extensively tested on Brodatzlike microtexture mixtures and on realword images. In addition, benchmark studies with alternative segmentation techniques are reported.
An Informationtheoretic Framework for Optimization with Application to Supervised Learning
 In Abstracts of the International Symposium on Information Theory
, 1995
"... ble assignments, P [x1 2 C j(1) ; : : : ; xN 2 C j(N) ], and choose it to minimize the expected assignment cost ! E ?, given a constraint on Shannon's entropy, H. Thus, we seek the best random assignments in the sense of ! E ? for a given H. This formulation is equivalently stated by invoking the m ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
ble assignments, P [x1 2 C j(1) ; : : : ; xN 2 C j(N) ], and choose it to minimize the expected assignment cost ! E ?, given a constraint on Shannon's entropy, H. Thus, we seek the best random assignments in the sense of ! E ? for a given H. This formulation is equivalently stated by invoking the maximum entropy principle, but the former description is more appealing for optimization. The constrained minimization is equivalent to the unconstrained minimization of the Lagrangian: L j fi ! E ? \GammaH , where fi is the Lagrange multiplier controlling ! E ? and H. Physical inspiration for minimizing L is obtained by recognizing that it is the Helmholtz free energ
ON SEMISUPERVISED KERNEL METHODS
"... Semisupervised learning is an emerging computational paradigm for learning from limited supervision by utilizing large amounts of inexpensive, unsupervised observations. Not only does this paradigm carry appeal as a model for natural learning, but it also has an increasing practical need in most if ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Semisupervised learning is an emerging computational paradigm for learning from limited supervision by utilizing large amounts of inexpensive, unsupervised observations. Not only does this paradigm carry appeal as a model for natural learning, but it also has an increasing practical need in most if not all applications of machine learning – those where abundant amounts of data can be cheaply and automatically collected but manual labeling for the purposes of training learning algorithms is often slow, expensive, and errorprone. In this thesis, we develop families of algorithms for semisupervised inference. These algorithms are based on intuitions about the natural structure and geometry of probability distributions that underlie typical datasets for learning. The classical framework of Regularization in Reproducing Kernel Hilbert Spaces (which is the basis of stateoftheart supervised algorithms such as SVMs) is extended in several ways to utilize unlabeled data. These extensions are embodied in the following contributions: (1) Manifold Regularization is based on the assumption that highdimensional
Adaptive Methods for Netlist Partitioning
, 1997
"... An algorithm that remains in use at the core of many partitioning systems is the KernighanLin algorithm and a variant the FidduciaMatheysses (FM) algorithm. To understand the FM algorithm we applied principles of data engineering where visualization and statistical analysis are used to analyze the ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
An algorithm that remains in use at the core of many partitioning systems is the KernighanLin algorithm and a variant the FidduciaMatheysses (FM) algorithm. To understand the FM algorithm we applied principles of data engineering where visualization and statistical analysis are used to analyze the runtime behavior. We identified two improvements to the algorithm which, without clustering or an improved heuristic function, bring the performance of the algorithm near that of more sophisticated algorithms. One improvement is based on the observation, explored empirically, that the full passes in the FM algorithm appear comparable to a stochastic local restart in the search. We motivate this observation with a discussion of recent improvements in Monte Carlo Markov Chain methods in statistics. The other improvement is based on the observation that when an FMlike algorithm is run 20 times and the best run chosen, the performance trace of the algorithm on earlier runs is useful data for ...
Markov Random Field Modelling of fMRI Data Using a Mean Field EMalgorithm
"... Abstract. This paper considers the use of the EMalgorithm, combined with mean field theory, for parameter estimation in Markov random field models from unlabelled data. Special attention is given to the theoretical justification for this procedure, based on recent results from the machine learning ..."
Abstract
 Add to MetaCart
Abstract. This paper considers the use of the EMalgorithm, combined with mean field theory, for parameter estimation in Markov random field models from unlabelled data. Special attention is given to the theoretical justification for this procedure, based on recent results from the machine learning literature. With these results established, an example is given of the application of this technique for analysis of single trial functional magnetic resonance (fMR) imaging data of the human brain. The resulting model segments fMR images into regions with different ‘brain response ’ characteristics. 1
Hopfield Neural Networks, and Mean Field Annealing for Seismic Deconvolution and Multiple Attenuation
, 1997
"... We describe a global optimization method called mean field annealing (MFA) and its application to two basic problems in seismic data processing: Seismic deconvolution and surface related multiple attenuation. MFA replaces the stochastic nature of the simulated annealing method with a set of det ..."
Abstract
 Add to MetaCart
We describe a global optimization method called mean field annealing (MFA) and its application to two basic problems in seismic data processing: Seismic deconvolution and surface related multiple attenuation. MFA replaces the stochastic nature of the simulated annealing method with a set of deterministic update rules that act on the average value of the variables rather than on the variables themselves, based on the mean field approximation. As the temperature is lowered, the MFA rules update the variables in terms of their values at a previous temperature. By minimizing averages, it is possible to converge to an equilibrium state considerably faster than a standard simulated annealing method.The update rules are dependent on the form of the cost function and are obtained easily when the cost function resembles the energy function of a Hopfield network. The mapping of a problem onto a Hopfield network is not a precondition for using MFA, but it makes analytic calculati...