Results 1  10
of
14
Modelbased decoding, information estimation, and changepoint detection in multineuron spike trains
 UNDER REVIEW, NEURAL COMPUTATION
, 2007
"... Understanding how stimulus information is encoded in spike trains is a central problem in computational neuroscience. Decoding methods provide an important tool for addressing this problem, by allowing us to explicitly read out the information contained in spike responses. Here we introduce several ..."
Abstract

Cited by 19 (12 self)
 Add to MetaCart
Understanding how stimulus information is encoded in spike trains is a central problem in computational neuroscience. Decoding methods provide an important tool for addressing this problem, by allowing us to explicitly read out the information contained in spike responses. Here we introduce several decoding methods based on pointprocess neural encoding models (i.e. “forward ” models that predict spike responses to novel stimuli). These models have concave loglikelihood functions, allowing for efficient fitting via maximum likelihood. Moreover, we may use the likelihood of the observed spike trains under the model to perform optimal decoding. We present: (1) a tractable algorithm for computing the maximum a posteriori (MAP) estimate of the stimulus — the most probable stimulus to have generated the observed single or multiplespike train response, given some prior distribution over the stimulus; (2) a Gaussian approximation to the posterior distribution, which allows us to quantify the fidelity with which various stimulus features are encoded; (3) an efficient method for estimating the mutual information between the stimulus and the response; and (4) a framework for the detection of changepoint times (e.g. the time at which the stimulus undergoes a change in mean or variance), by marginalizing over the posterior distribution of stimuli. We show several examples illustrating the performance of these estimators with simulated data.
Noisecontrastive estimation: A new estimation principle for unnormalized statistical models
"... We present a new estimation principle for parameterized statistical models. The idea is to perform nonlinear logistic regression to discriminate between the observed data and some artificially generated noise, using the model logdensity function in the regression nonlinearity. We show that this lea ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
We present a new estimation principle for parameterized statistical models. The idea is to perform nonlinear logistic regression to discriminate between the observed data and some artificially generated noise, using the model logdensity function in the regression nonlinearity. We show that this leads to a consistent (convergent) estimator of the parameters, and analyze the asymptotic variance. In particular, the method is shown to directly work for unnormalized models, i.e. models where the density function does not integrate to one. The normalization constant can be estimated just like any other parameter. For a tractable ICA model, we compare the method with other estimation methods that can be used to learn unnormalized models, including score matching, contrastive divergence, and maximumlikelihood where the normalization constant is estimated with importance sampling. Simulations show that noisecontrastive estimation offers the best tradeoff between computational and statistical efficiency. The method is then applied to the modeling of natural images: We show that the method can successfully estimate a largescale twolayer model and a Markov random field. 1
Learning IntermediateLevel Representations of Form and Motion from Natural Movies
 Neural Computation
, 2011
"... We present a model of intermediatelevel visual representation that is based on learning invariances from movies of the natural environment. The model is composed of two stages of processing: an early feature representation layer and a second layer in which invariances are explicitly represented. In ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
We present a model of intermediatelevel visual representation that is based on learning invariances from movies of the natural environment. The model is composed of two stages of processing: an early feature representation layer and a second layer in which invariances are explicitly represented. Invariances are learned as the result of factoring apart the temporally stable and dynamic components embedded in the early feature representation. The structure contained in these components is made explicit in the activities of secondlayer units that capture invariances in both form and motion. When trained on natural movies, the first layer produces a factorization, or separation, of image content into a temporally persistent part representing local edge structure and a dynamic part representing local motion structure, consistent with known response properties in early visual cortex (area V1). This factorization linearizes statistical dependencies among the firstlayer units, making them learnable by the second layer. The secondlayer units are split into two populations according to the factorization in the first layer. The formselective units receive their input from the temporally persistent part (local edge structure) and after training result in a diverse set of higherorder shape features consisting of extended contours, multiscale edges, textures, and texture boundaries. The motionselective units receive their input from the dynamic part (local motion structure) and after training result in a representation of image translation over different spatial scales and directions, in addition to more complex deformations. These representations provide a rich description of dynamic natural images and testable hypotheses regarding intermediatelevel representation in visual cortex. 1
Hierarchical Modeling of Local Image Features through LpNested Symmetric Distributions
"... We introduce a new family of distributions, called Lpnested symmetric distributions, whose densities are expressed in terms of a hierarchical cascade of Lpnorms. This class generalizes the family of spherically and Lpspherically symmetric distributions which have recently been successfully used fo ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
We introduce a new family of distributions, called Lpnested symmetric distributions, whose densities are expressed in terms of a hierarchical cascade of Lpnorms. This class generalizes the family of spherically and Lpspherically symmetric distributions which have recently been successfully used for natural image modeling. Similar to those distributions it allows for a nonlinear mechanism to reduce the dependencies between its variables. With suitable choices of the parameters and norms, this family includes the Independent Subspace Analysis (ISA) model as a special case, which has been proposed as a means of deriving filters that mimic complex cells found in mammalian primary visual cortex. Lpnested distributions are relatively easy to estimate and allow us to explore the variety of models between ISA and the Lpspherically symmetric models. By fitting the generalized Lpnested model to 8 × 8 image patches, we show that the subspaces obtained from ISA are in fact more dependent than the individual filter coefficients within a subspace. When first applying contrast gain control as preprocessing, however, there are no dependencies left that could be exploited by ISA. This suggests that complex cell modeling can only be useful for redundancy reduction for larger image patches. 1
Reducing statistical dependencies in natural signals using radial Gaussianization
"... We consider the problem of transforming a signal to a representation in which the components are statistically independent. When the signal is generated as a linear transformation of independent Gaussian or nonGaussian sources, the solution may be computed using a linear transformation (PCA or ICA, ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
We consider the problem of transforming a signal to a representation in which the components are statistically independent. When the signal is generated as a linear transformation of independent Gaussian or nonGaussian sources, the solution may be computed using a linear transformation (PCA or ICA, respectively). Here, we consider a complementary case, in which the source is nonGaussian but elliptically symmetric. Such a source cannot be decomposed into independent components using a linear transform, but we show that a simple nonlinear transformation, which we call radial Gaussianization (RG), is able to remove all dependencies. We apply this methodology to natural signals, demonstrating that the joint distributions of nearby bandpass filter responses, for both sounds and images, are closer to being elliptically symmetric than linearly transformed factorial sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either pairs or blocks of bandpass filter responses is significantly greater than that achieved by PCA or ICA. 1
The "treedependent components " of natural scenes are edge filters
"... We propose a new model for natural image statistics. Instead of minimizing dependency between components of natural images, we maximize a simple form of dependency in the form of treedependencies. By learning filters and tree structures which are best suited for natural images we observe that the r ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We propose a new model for natural image statistics. Instead of minimizing dependency between components of natural images, we maximize a simple form of dependency in the form of treedependencies. By learning filters and tree structures which are best suited for natural images we observe that the resulting filters are edge filters, similar to the famous ICA on natural images results. Calculating the likelihood of an image patch using our model requires estimating the squared output of pairs of filters connected in the tree. We observe that after learning, these pairs of filters are predominantly of similar orientations but different phases, so their joint energy resembles models of complex cells. 1 Introduction and related work Many models of natural image statistics have been proposed in recent years [1, 2, 3, 4]. A common goal of many of these models is finding a representation in which components or subcomponents of the image are made as independent or as sparse as possible [5, 6, 2]. This has been found to be a difficult goal, as natural images have a highly intricate structure and removing dependencies between
Divisive normalization: Justification and effectiveness as efficient coding transform
 In Advances in Neural Information Processing Systems 23
, 2010
"... Divisive normalization (DN) has been advocated as an effective nonlinear efficient coding transform for natural sensory signals with applications in biology and engineering. In this work, we aim to establish a connection between the DN transform and the statistical properties of natural sensory sign ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Divisive normalization (DN) has been advocated as an effective nonlinear efficient coding transform for natural sensory signals with applications in biology and engineering. In this work, we aim to establish a connection between the DN transform and the statistical properties of natural sensory signals. Our analysis is based on the use of multivariate t model to capture some important statistical properties of natural sensory signals. The multivariate t model justifies DN as an approximation to the transform that completely eliminates its statistical dependency. Furthermore, using the multivariate t model and measuring statistical dependency with multiinformation, we can precisely quantify the statistical dependency that is reduced by the DN transform. We compare this with the actual performance of the DN transform in reducing statistical dependencies of natural sensory signals. Our theoretical analysis and quantitative evaluations confirm DN as an effective efficient coding transform for natural sensory signals. On the other hand, we also observe a previously unreported phenomenon that DN may increase statistical dependencies when the size of pooling is small. 1
LpNESTED SYMMETRIC DISTRIBUTIONS
"... A important part in statistical analysis of data is to find a class of models that is flexible and rich enough to model the regularities in the data, but at the same time exhibits enough symmetry and structure itself to still be computationally and analytically tractable. One special way of introduc ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
A important part in statistical analysis of data is to find a class of models that is flexible and rich enough to model the regularities in the data, but at the same time exhibits enough symmetry and structure itself to still be computationally and analytically tractable. One special way of introducing such a symmetry is to fix the
RADIAL GAUSSIANIZATION WITH CLUSTERSPECIFIC BIAS COMPENSATION
"... In recent work, Lyu and Simoncelli [1] introduced radial Gaussianization (RG) as a very efficient procedure for transforming ndimensional random vectors into Gaussian vectors with independent and identically distributed (i.i.d.) components. This entails transforming the norms of the data so that th ..."
Abstract
 Add to MetaCart
In recent work, Lyu and Simoncelli [1] introduced radial Gaussianization (RG) as a very efficient procedure for transforming ndimensional random vectors into Gaussian vectors with independent and identically distributed (i.i.d.) components. This entails transforming the norms of the data so that they become chidistributed with n degrees of freedom. A necessary requirement is that the original data are generated by an isotropic distribution, that is, their probability density function (pdf) is constant over surfaces of ndimensional spheres (or, more general, ndimensional ellipsoids). The case of biases in the data, which is of great practical interest, is studied here; as we demonstrate with experiments, there are situations in which even very small amounts of bias can cause RG to fail. This becomes evident especially when the data form clusters in lowdimensional manifolds. To address this shortcoming, we propose a twostep approach which entails (i) first discovering clusters in the data and removing the bias from each, and (ii) performing RG on the biascompensated data. In experiments with synthetic data, the proposed bias compensation procedure results in significantly better Gaussianization than the noncompensated RG method. Index Terms — Gaussian distribution, chi distribution, isotropic distribution, singular distribution