Results 1  10
of
10
Diffusion Kernels on Statistical Manifolds
, 2004
"... A family of kernels for statistical learning is introduced that exploits the geometric structure of statistical models. The kernels are based on the heat equation on the Riemannian manifold defined by the Fisher information metric associated with a statistical family, and generalize the Gaussian ker ..."
Abstract

Cited by 87 (6 self)
 Add to MetaCart
A family of kernels for statistical learning is introduced that exploits the geometric structure of statistical models. The kernels are based on the heat equation on the Riemannian manifold defined by the Fisher information metric associated with a statistical family, and generalize the Gaussian kernel of Euclidean space. As an important special case, kernels based on the geometry of multinomial families are derived, leading to kernelbased learning algorithms that apply naturally to discrete data. Bounds on covering numbers and Rademacher averages for the kernels are proved using bounds on the eigenvalues of the Laplacian on Riemannian manifolds. Experimental results are presented for document classification, for which the use of multinomial geometry is natural and well motivated, and improvements are obtained over the standard use of Gaussian or linear kernels, which have been the standard for text classification.
Symmetrizing the KullbackLeibler Distance
 IEEE Transactions on Information Theory
, 2000
"... We define a new distance measure the resistoraverage distance between two probability distributions that is closely related to the KullbackLeibler distance. While the KullbackLeibler distance is asymmetric in the two distributions, the resistoraverage distance is not. It arises from geometric ..."
Abstract

Cited by 24 (0 self)
 Add to MetaCart
We define a new distance measure the resistoraverage distance between two probability distributions that is closely related to the KullbackLeibler distance. While the KullbackLeibler distance is asymmetric in the two distributions, the resistoraverage distance is not. It arises from geometric considerations similar to those used to derive the Chernoff distance. Determining its relation to wellknown distance measures reveals a new way to depict how commonly used distance measures relate to each other. 1 Introduction The KullbackLeibler distance [15, 16] is perhaps the most frequently used informationtheoretic "distance" measure from a viewpoint of theory. If p 0 , p 1 are two probability densities, the KullbackLeibler distance is defined to be D(p 1 #p 0 )= # p 1 (x)log p 1 (x) p 0 (x) dx . (1) In this paper, log() has base two. The KullbackLeibler distance is but one example of the AliSilvey class of informationtheoretic distance measures [1], which are defined to ...
An informationtheoretic approach to detecting changes in multidimensional data streams
 In Proc. Symp. on the Interface of Statistics, Computing Science, and Applications
, 2006
"... Abstract An important problem in processing large data streams is detecting changes in the underlying distribution that generates the data. The challenge in designing change detection schemes is making them general, scalable, and statistically sound. In this paper, we take a general,informationthe ..."
Abstract

Cited by 22 (1 self)
 Add to MetaCart
Abstract An important problem in processing large data streams is detecting changes in the underlying distribution that generates the data. The challenge in designing change detection schemes is making them general, scalable, and statistically sound. In this paper, we take a general,informationtheoretic approach to the change detection problem, which works for multidimensional as well as categorical data. We use relative entropy, also called the KullbackLeiblerdistance, to measure the difference between two given distributions. The KLdistance is known to be related to the optimal error in determining whether the two distributions are the sameand draws on fundamental results in hypothesis testing. The KLdistance also generalizes traditional distance measures in statistics, and has invariance properties that make it ideally suitedfor comparing distributions. Our scheme is general; it is nonparametric and requires no assumptions on the underlyingdistributions. It employs a statistical inference procedure based on the theory of bootstrapping, which allows us to determine whether our measurements are statistically significant. The schemeis also quite flexible from a practical perspective; it can be implemented using any spatial partitioning scheme that scales well with dimensionality. In addition to providing change detections,our method generalizes Kulldorff's spatial scan statistic, allowing us to quantitatively identify specific regions in space where large changes have occurred.We provide a detailed experimental study that demonstrates the generality and efficiency of our approach with different kinds of multidimensional datasets, both synthetic and real. 1 Introduction We are collecting and storing data in unprecedented quantities and varietiesstreams, images, audio, text, metadata descriptions, and even simple numbers. Over time, these data streams change as the underlying processes that generate them change. Some changes are spurious and pertain to glitches in the data. Some are genuine, caused by changes in the underlying distributions. Some changes are gradual and some are more precipitous. We would like to detect changes in a variety of settings:
Toward a Theory of Information Processing
 IEEE Trans. Signal Processing
, 2002
"... Information processing theory endeavors to quantify how well signals encode information and how well systems, by acting on signals, process information. We use informationtheoretic distance measures, the KullbackLeibler distance in particular, to quantify how well signals represent information. ..."
Abstract

Cited by 12 (5 self)
 Add to MetaCart
Information processing theory endeavors to quantify how well signals encode information and how well systems, by acting on signals, process information. We use informationtheoretic distance measures, the KullbackLeibler distance in particular, to quantify how well signals represent information. The ratio of distances between a system's output and input quantifies the system's information processing properties.
FreeBSD CVS log for ports/INDEX with Asami' s song texts: http://www.freebsd.org/cgi/cvsweb.cgi/ports/INDEX FreeBSD porters Handbook: http://www.freebsd.org/doc/en_US.ISO88591/books/portershandbook OpenBSD: "Building an OpenBSD port" http://ww
 In the
"... Abstract. KullbackLeibler relativeentropy, in cases involving distributions resulting from relativeentropy minimization, has a celebrated property reminiscent of squared Euclidean distance: it satisfies an analogue of the Pythagoras ’ theorem. And hence, this property is referred to as Pythagoras ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Abstract. KullbackLeibler relativeentropy, in cases involving distributions resulting from relativeentropy minimization, has a celebrated property reminiscent of squared Euclidean distance: it satisfies an analogue of the Pythagoras ’ theorem. And hence, this property is referred to as Pythagoras ’ theorem of relativeentropy minimization or triangle equality and plays a fundamental role in geometrical approaches of statistical estimation theory like information geometry. Equvalent of Pythagoras’ theorem in the generalized nonextensive formalism is established in (Dukkipati at
Relations between KullbackLeibler distance and Fisher information
, 2002
"... The KullbackLeibler distance between two probability densities that are parametric perturbations of each other is related to the Fisher information. We generalize this relationship to the case when the perturbations may not be small and when the two densities are nonparametric. ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
The KullbackLeibler distance between two probability densities that are parametric perturbations of each other is related to the Fisher information. We generalize this relationship to the case when the perturbations may not be small and when the two densities are nonparametric.
Signal Estimation Under Random TimeWarpings and Nonlinear Signal Alignment
"... While signal estimation under random amplitudes, phase shifts, and additive noise is studied frequently, the problem of estimating a deterministic signal under random timewarpings has been relatively unexplored. We present a novel framework for estimating the unknown signal that utilizes the action ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
While signal estimation under random amplitudes, phase shifts, and additive noise is studied frequently, the problem of estimating a deterministic signal under random timewarpings has been relatively unexplored. We present a novel framework for estimating the unknown signal that utilizes the action of the warping group to form an equivalence relation between signals. First, we derive an estimator for the equivalence class of the unknown signal using the notion of Karcher mean on the quotient space of equivalence classes. This step requires the use of FisherRao Riemannian metric and a squareroot representation of signals to enable computations of distances and means under this metric. Then, we define a notion of the center of a class and show that the center of the estimated class is a consistent estimator of the underlying unknown signal. This estimation algorithm has many applications: (1) registration/alignment of functional data, (2) separation of phase/amplitude components of functional data, (3) joint demodulation and carrier estimation, and (4) sparse modeling of functional data. Here we demonstrate only (1) and (2): Given signals are temporally aligned using nonlinear warpings and, thus, separated into their phase and amplitude components. The proposed method for signal alignment is shown to have state of the art performance using Berkeley growth, handwritten signatures, and neuroscience spike train data. 1
174 Mathematical Foundations of Computational Anatomy (MFCA'06) Realizing Unbiased Deformation: A Theoretical Consideration
, 2011
"... Abstract — Maps of local tissue compression or expansion are often recovered by comparing MRI scans using nonlinear registration techniques. The resulting changes can be analyzed using tensorbased morphometry (TBM) to make inferences about anatomical differences. Numerous deformation techniques hav ..."
Abstract
 Add to MetaCart
Abstract — Maps of local tissue compression or expansion are often recovered by comparing MRI scans using nonlinear registration techniques. The resulting changes can be analyzed using tensorbased morphometry (TBM) to make inferences about anatomical differences. Numerous deformation techniques have been developed, although there has not been much theoretical development examining the mathematical/statistical validity of each technique. In this paper, we propose a basic principle that any registration technique should satisfy: realizing unbiased test statistics under null distribution of the displacement. In other words, any registration technique should recover zero change in the test statistic when comparing two images differing only in noise. Based on this principle, we propose a fundamental framework for the construction and analysis of image deformation. Moreover, we argue that logarithmic transform is instrumental in the analysis of deformation maps. Combined with the proposed framework, this leads to a theoretical connection between image registration and other branches of applied mathematics including information theory and grid generation. Index TermsMutual information, Image registration, Computational anatomy. 1.
Mathematical Foundations of Computational Anatomy Geometrical and Statistical Methods for Modelling Biological Shape Variability
"... Nonlinear registration and shape analysis are well developed research topic in the medical image analysis community. There is nowadays a growing number of methods that can faithfully deal with the underlying biomechanical behaviour of intrasubject shape deformations. However, it is more difficult ..."
Abstract
 Add to MetaCart
Nonlinear registration and shape analysis are well developed research topic in the medical image analysis community. There is nowadays a growing number of methods that can faithfully deal with the underlying biomechanical behaviour of intrasubject shape deformations. However, it is more difficult to relate the anatomical shape of different subjects. The goal of computational anatomy is to analyse and to statistically model this specific type of geometrical information. In the absence of any justified physical model, a natural attitude is to explore very general mathematical methods, for instance diffeomorphisms. However, working with such infinite dimensional space raises some deep computational and mathematical problems. In particular, one of the key problem is to do statistics. Likewise, modelling the variability of surfaces leads to rely on shape spaces that are much more complex than for curves. To cope with these, different methodological and computational frameworks have been proposed. The goal of the workshop was to foster interactions between researchers investigating the combination of geometry and statistics for modelling biological shape variability from image and surfaces. A special emphasis was put on theoretical developments, applications and results being welcomed as illustrations. Contributions were solicited in the following areas:
Forward
, 2011
"... Nonlinear registration and shape analysis are well developed research topic in the medical image analysis community. There is nowadays a growing number of methods that can faithfully deal with the underlying biomechanical behaviour of intrasubject shape deformations. However, it is more difficult ..."
Abstract
 Add to MetaCart
Nonlinear registration and shape analysis are well developed research topic in the medical image analysis community. There is nowadays a growing number of methods that can faithfully deal with the underlying biomechanical behaviour of intrasubject shape deformations. However, it is more difficult to relate the anatomical shape of different subjects. The goal of computational anatomy is to analyse and to statistically model this specific type of geometrical information. In the absence of any justified physical model, a natural attitude is to explore very general mathematical methods, for instance diffeomorphisms. However, working with such infinite dimensional space raises some deep computational and mathematical problems. In particular, one of the key problem is to do statistics. Likewise, modelling the variability of surfaces leads to rely on shape spaces that are much more complex than for curves. To cope with these, different methodological and computational frameworks have been proposed. The goal of the workshop was to foster interactions between researchers investigating the combination of geometry and statistics for modelling biological shape variability from image and surfaces. A special emphasis was put on theoretical developments, applications and results being welcomed as illustrations. inria00614989, version 1 17 Aug 2011