Results 1  10
of
90
ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context
 BMC Bioinformatics
, 2006
"... * To whom correspondence should be addressed ..."
Kernel measures of conditional dependence
 In Adv. NIPS
, 2008
"... We propose a new measure of conditional dependence of random variables, based on normalized crosscovariance operators on reproducing kernel Hilbert spaces. Unlike previous kernel dependence measures, the proposed criterion does not depend on the choice of kernel in the limit of infinite data, for a ..."
Abstract

Cited by 48 (32 self)
 Add to MetaCart
We propose a new measure of conditional dependence of random variables, based on normalized crosscovariance operators on reproducing kernel Hilbert spaces. Unlike previous kernel dependence measures, the proposed criterion does not depend on the choice of kernel in the limit of infinite data, for a wide class of kernels. At the same time, it has a straightforward empirical estimate with good convergence behaviour. We discuss the theoretical properties of the measure, and demonstrate its application in experiments. 1
Nonlinear Multivariate Analysis of Neurophysiological Signals
 Progress in Neurobiology
, 2005
"... Multivariate time series analysis is extensively used in neurophysiology with the aim of studying the relationship between simultaneously recorded signals. Recently, advances on information theory and nonlinear dynamical systems theory have allowed the study of various types of synchronization from ..."
Abstract

Cited by 39 (1 self)
 Add to MetaCart
Multivariate time series analysis is extensively used in neurophysiology with the aim of studying the relationship between simultaneously recorded signals. Recently, advances on information theory and nonlinear dynamical systems theory have allowed the study of various types of synchronization from time series. In this work, we first describe the multivariate linear methods most commonly used in neurophysiology and show that they can be extended to assess the existence of nonlinear interdependences between signals. We then review the concepts of entropy and mutual information followed by a detailed description of nonlinear methods based on the concepts of phase synchronization, generalized synchronization and event synchronization. In all cases, we show how to apply these methods to study different kinds of neurophysiological data. Finally, we illustrate the use of multivariate surrogate data test for the assessment of the strength (strong or weak) and the type (linear or nonlinear) of interdependence between neurophysiological signals.
A class of Rényi information estimators for multidimensional densities
 Annals of Statistics
, 2008
"... A class of estimators of the Rényi and Tsallis entropies of an unknown distribution f in R m is presented. These estimators are based on the kth nearestneighbor distances computed from a sample of N i.i.d. vectors with distribution f. We show that entropies of any order q, including Shannon’s entro ..."
Abstract

Cited by 34 (3 self)
 Add to MetaCart
A class of estimators of the Rényi and Tsallis entropies of an unknown distribution f in R m is presented. These estimators are based on the kth nearestneighbor distances computed from a sample of N i.i.d. vectors with distribution f. We show that entropies of any order q, including Shannon’s entropy, can be estimated consistently with minimal assumptions on f.Moreover, we show that it is straightforward to extend the nearestneighbor method to estimate the statistical distance between two distributions using one i.i.d. sample from each.
Separating a reallife nonlinear image mixture
 Journal of Machine Learning Research
, 2005
"... Note: The pagination of this version is slightly different from the pagination of the version published in the Journal of Machine Learning Research, and available in its web site. The contents are the same, however. When acquiring an image of a paper document, the image printed on the back page some ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
Note: The pagination of this version is slightly different from the pagination of the version published in the Journal of Machine Learning Research, and available in its web site. The contents are the same, however. When acquiring an image of a paper document, the image printed on the back page sometimes shows through. The mixture of the front and backpage images thus obtained is markedly nonlinear, and thus constitutes a good reallife test case for nonlinear blind source separation. This paper addresses a difficult version of this problem, corresponding to the use of “onion skin ” paper, which results in a relatively strong nonlinearity of the mixture, which becomes close to singular in the lighter regions of the images. The separation is achieved through the MISEP technique, which is an extension of the well known INFOMAX method. The separation results are assessed with objective quality measures. They show an improvement over the results obtained with linear separation, but have room for further improvement.
Hierarchical clustering based on mutual information. Arxiv preprint qbio/0311039
, 2003
"... Motivation: Clustering is a frequently used concept in variety of bioinformatical applications. We present a new method for hierarchical clustering of data called mutual information clustering (MIC) algorithm. It uses mutual information (MI) as a similarity measure and exploits its grouping property ..."
Abstract

Cited by 14 (0 self)
 Add to MetaCart
Motivation: Clustering is a frequently used concept in variety of bioinformatical applications. We present a new method for hierarchical clustering of data called mutual information clustering (MIC) algorithm. It uses mutual information (MI) as a similarity measure and exploits its grouping property: The MI between three objects X, Y, and Z is equal to the sum of the MI between X and Y, plus the MI between Z and the combined object (XY). Results: We use this both in the Shannon (probabilistic) version of information theory, where the “objects ” are probability distributions represented by random samples, and in the Kolmogorov (algorithmic) version, where the “objects ” are symbol sequences. We apply our method to the construction of mammal phylogenetic trees from mitochondrial DNA sequences and we reconstruct the fetal ECG from the output of independent components analysis (ICA) applied to the ECG of a pregnant woman. Availability: The programs for estimation of MI and for clustering (probabilistic version) are available
Nonlinear Extraction of Independent Components of Natural Images Using Radial Gaussianization
, 2009
"... We consider the problem of efficiently encoding a signal by transforming it to a new representation whose components are statistically independent. A widely studied linear solution, known as independent component analysis (ICA), exists for the case when the signal is generated as a linear transforma ..."
Abstract

Cited by 14 (4 self)
 Add to MetaCart
We consider the problem of efficiently encoding a signal by transforming it to a new representation whose components are statistically independent. A widely studied linear solution, known as independent component analysis (ICA), exists for the case when the signal is generated as a linear transformation of independent nongaussian sources. Here, we examine a complementary case, in which the source is nongaussian and elliptically symmetric. In this case, no invertible linear transform suffices to decompose the signal into independent components, but we show that a simple nonlinear transformation, which we call radial gaussianization (RG), is able to remove all dependencies. We then examine this methodology in the context of natural image statistics. We first show that distributions of spatially proximal bandpass filter responses are better described as elliptical than as linearly transformed independent sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either nearby pairs or blocks of bandpass filter responses is significantly greater than that achieved by ICA. Finally, we show that the RG transformation may be closely approximated by divisive normalization, which has been used to model the nonlinear response properties of visual neurons.
Estimation of Rényi entropy and mutual information based on generalized nearestneighbor graphs
, 2010
"... We present simple and computationally efficient nonparametric estimators of Rényi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over R d. The estimators are calculated as the sum of pth powers of the Euclidean lengths of the edges ..."
Abstract

Cited by 14 (2 self)
 Add to MetaCart
We present simple and computationally efficient nonparametric estimators of Rényi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over R d. The estimators are calculated as the sum of pth powers of the Euclidean lengths of the edges of the ‘generalized nearestneighbor ’ graph of the sample and the empirical copula of the sample respectively. For the first time, we prove the almost sure consistency of these estimators and upper bounds on their rates of convergence, the latter of which under the assumption that the density underlying the sample is Lipschitz continuous. Experiments demonstrate their usefulness in independent subspace analysis. 1
A NearestNeighbor Approach to Estimating Divergence between Continuous Random Vectors
, 2006
"... A method for divergence estimation between multidimensional distributions based on nearest neighbor distances is proposed. Given i.i.d. samples, both the bias and the variance of this estimator are proven to vanish as sample sizes go to infinity. In experiments on highdimensional data, the nearest ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
A method for divergence estimation between multidimensional distributions based on nearest neighbor distances is proposed. Given i.i.d. samples, both the bias and the variance of this estimator are proven to vanish as sample sizes go to infinity. In experiments on highdimensional data, the nearest neighbor approach generally exhibits faster convergence compared to previous algorithms based on partitioning.
Sufficient Dimension Reduction via Squaredloss Mutual Information Estimation
"... The goal of sufficient dimension reduction in supervised learning is to find the lowdimensional subspace of input features that is ‘sufficient ’ for predicting output values. In this paper, we propose a novel sufficient dimension reduction method using a squaredloss variant of mutual information as ..."
Abstract

Cited by 12 (11 self)
 Add to MetaCart
The goal of sufficient dimension reduction in supervised learning is to find the lowdimensional subspace of input features that is ‘sufficient ’ for predicting output values. In this paper, we propose a novel sufficient dimension reduction method using a squaredloss variant of mutual information as a dependency measure. We utilize an analytic approximator of squaredloss mutual information based on density ratio estimation, which is shown to possess suitable convergence properties. We then develop a natural gradient algorithm for sufficient subspace search. Numerical experiments show that the proposed method compares favorably with existing dimension reduction approaches. 1