Results 1  10
of
14
Quantifying and visualizing attribute interactions: An approach based on entropy
 http://arxiv.org/abs/cs.AI/0308002 v3
, 2004
"... Interactions are patterns between several attributes in data that cannot be inferred from any subset of these attributes. While mutual information is a wellestablished approach to evaluating the interactions between two attributes, we surveyed its generalizations as to quantify interactions between ..."
Abstract

Cited by 33 (4 self)
 Add to MetaCart
(Show Context)
Interactions are patterns between several attributes in data that cannot be inferred from any subset of these attributes. While mutual information is a wellestablished approach to evaluating the interactions between two attributes, we surveyed its generalizations as to quantify interactions between several attributes. We have chosen McGill’s interaction information, which has been independently rediscovered a number of times under various names in various disciplines, because of its many intuitively appealing properties. We apply interaction information to visually present the most important interactions of the data. Visualization of interactions has provided insight into the structure of data on a number of domains, identifying redundant attributes and opportunities for constructing new features, discovering unexpected regularities in data, and have helped during construction of predictive models; we illustrate the methods on numerous examples. A machine learning method that disregards interactions may get caught in two traps: myopia is caused by learning algorithms assuming independence in spite of interactions, whereas fragmentation arises from assuming an interaction in spite of independence.
Information geometric measure for neural spikes
 Neural Computation
, 2002
"... The present study introduces informationgeometric measures to analyze neural ring patterns by taking not only the secondorder but also higherorder interactions among neurons into account. Information geometry provides useful tools and concepts for this purpose, including the orthogonality of coo ..."
Abstract

Cited by 14 (5 self)
 Add to MetaCart
(Show Context)
The present study introduces informationgeometric measures to analyze neural ring patterns by taking not only the secondorder but also higherorder interactions among neurons into account. Information geometry provides useful tools and concepts for this purpose, including the orthogonality of coordinate parameters and the Pythagoras relation in the KullbackLeibler divergence. Based on this orthogonality, we show anovel method to analyze spike ring patterns by decomposing the interactions of neurons of various orders. As a result, purely pairwise, triplewise, and higherorder interactions are singled out. We also demonstrate the bene ts of our proposal by using real neural data, recorded in the prefrontal and parietal cortices of monkeys. 1
InformationGeometric Decomposition in Spike Analysis
 Diettrich, S. Becker, Z. Ghahramani (Eds.), NIPS
, 2001
"... We present an informationgeometric measure to systematically investigate neuronal firing patterns, taking account not only of the secondorder but also of higherorder interactions. We begin with the case of two neurons for illustration and show how to test whether or not any pairwise correlati ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
We present an informationgeometric measure to systematically investigate neuronal firing patterns, taking account not only of the secondorder but also of higherorder interactions. We begin with the case of two neurons for illustration and show how to test whether or not any pairwise correlation in one period is significantly different from that in the other period. In order to test such a hypothesis of different firing rates, the correlation term needs to be singled out `orthogonally' to the firing rates, where the null hypothesis might not be of independent firing. This method is also shown to directly associate neural firing with behavior via their mutual information, which is decomposed into two types of information, conveyed by mean firing rate and coincident firing, respectively.
Gene interaction in DNA microarray data is decomposed by information geometric measure
 Bioinformatics
, 2003
"... measure ..."
(Show Context)
Notes on information geometry and evolutionary processes
 CoRR
"... Abstract. In order to analyze and extract different structural properties of distributions, one can introduce different coordinate systems over the manifold of distributions. In Evolutionary Computation, the Walsh bases and the Building Block Bases are often used to describe populations, which simpl ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Abstract. In order to analyze and extract different structural properties of distributions, one can introduce different coordinate systems over the manifold of distributions. In Evolutionary Computation, the Walsh bases and the Building Block Bases are often used to describe populations, which simplifies the analysis of evolutionary operators applying on populations. Quite independent from these approaches, information geometry has been developed as a geometric way to analyze different order dependencies between random variables (e.g., neural activations or genes). In these notes I briefly review the essentials of various coordinate bases and of information geometry. The goal is to give an overview and make the approaches comparable. Besides introducing meaningful coordinate bases, information geometry also offers an explicit way to distinguish different order interactions and it offers a geometric view on the manifold and thereby also on operators that apply on the manifold. For instance, uniform crossover can be interpreted as an orthogonal projection of a population along an mgeodesic, monotonously reducing the θcoordinates that describe interactions between genes. 1
Neural Coding: Higher Order Temporal Patterns in the Neurostatistics of Cell Assemblies
, 2000
"... Recent advances in the technology of multiunit recordings make it possible to test Hebbs hypothesis that neurons do not function in isolation but are organized in assemblies. This has created the need for statistical approaches to detecting the presence of spatiotemporal patterns of more than two n ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Recent advances in the technology of multiunit recordings make it possible to test Hebbs hypothesis that neurons do not function in isolation but are organized in assemblies. This has created the need for statistical approaches to detecting the presence of spatiotemporal patterns of more than two neurons in neuron spike train data. We mention three possible measures for the presence of higher order patterns of neural activation  coefficients of loglinear models, connected cumulants, and redundancies  and present arguments in favor of the coefficients of loglinear models. We present test statistics for detecting the presence of higher order interactions in spike train data, by parametrizing these interactions in terms of coefficients of loglinear models. We also present a Bayesian approach for inferring the existence or absence of interactions and estimating their strength. The two methods, the frequentist and the Bayesian one, are shown to be consistent in the sense that interactions that are detected by either method also tend to be detected by the other. A heuristic for the analysis of temporal patterns is also proposed. Finally a Bayesian test is presented, that establishes stochastic differences between recorded segments of data. The methods are applied to experimental data and to synthetic data drawn from our statistical models. Our experimental data are drawn from multiunit recordings in the prefrontal cortex of behaving monkeys, from the somatosensory cortex of anesthetized rats, and from multiunit recordings in the visual cortex of behaving monkeys. 1
Mixture decompositions using a decomposition of the sample space. ArXiv 1008.0204
, 2010
"... We study the problem of finding the smallest m for which every element p of an exponential family E with finite sample space can be written as a mixture ofm elements of another exponential family E ′ as p = ∑mi=1 αifi, where fi ∈ E ′, αi ≥ 0 ∀i and∑mi=1 αi = 1. Our approach is based on coverings and ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
We study the problem of finding the smallest m for which every element p of an exponential family E with finite sample space can be written as a mixture ofm elements of another exponential family E ′ as p = ∑mi=1 αifi, where fi ∈ E ′, αi ≥ 0 ∀i and∑mi=1 αi = 1. Our approach is based on coverings and packings of the face lattice of the corresponding convex support polytopes. We use the notion of Ssets, subsets of the sample space such that every probability distribution that they support is contained in the closure of E. We find, in particular, that m = qN−1 yields the smallest mixtures of product distributions containing all distributions of N qary variables, and that any distribution ofN binary variables is a mixture ofm = 2N−(k+1)(1+1/(2k−1)) elements of the kinteraction exponential family (k = 1 describes product distributions). 1
Mixture Decomposition of Distributions using a Decomposition of the Sample Space
, 2010
"... We consider the set of join probability distributions of N binary random variables which can be written as a sum of m distributions in the following form p(x1,..., xN) =∑m i=1 αifi(x1,..., xN), where αi ≥ 0, ∑m i=1 αi = 1, and the fi(x1,..., xN) belong to some exponential family. For our analysis we ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
We consider the set of join probability distributions of N binary random variables which can be written as a sum of m distributions in the following form p(x1,..., xN) =∑m i=1 αifi(x1,..., xN), where αi ≥ 0, ∑m i=1 αi = 1, and the fi(x1,..., xN) belong to some exponential family. For our analysis we decompose the sample space into portions on which the mixture components fi can be chosen arbitrarily. We derive lower bounds on the number of mixture components from a given exponential family necessary to represent distributions with arbitrary correlations up to a certain order or to represent any distribution. For instance, in the case where fi are independent distributions we show that every distribution p on {0, 1}N is contained in the mixture model whenever m ≥ 2N−1, and furthermore, that there are distributions which are not contained in the mixture model whenever m < 2N−1.
decomposed by information geometric measure
, 2002
"... interaction in DNA microarray data is ..."
(Show Context)