Results 1 
4 of
4
Non Linear Neurons in the Low Noise Limit: A Factorial Code Maximizes Information Transfer
, 1994
"... We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focussing on the case of non linear transfer functions. We assume that both receptive fields (synaptic efficacies) and transfer functions can be adapted to the environm ..."
Abstract

Cited by 154 (18 self)
 Add to MetaCart
We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focussing on the case of non linear transfer functions. We assume that both receptive fields (synaptic efficacies) and transfer functions can be adapted to the environment. The main result is that, for bounded and invertible transfer functions, in the case of a vanishing additive output noise, and no input noise, maximization of information (Linsker'sinfomax principle) leads to a factorial code  hence to the same solution as required by the redundancy reduction principle of Barlow. We show also that this result is valid for linear, more generally unbounded, transfer functions, provided optimization is performed under an additive constraint, that is which can be written as a sum of terms, each one being specific to one output neuron. Finally we study the effect of a non zero input noise. We find that, at first order in the input noise, assumed to be small ...
Origins of Scaling in Natural Images
, 1997
"... One of the most robust qualities of our visual world is the scaleinvariance of natural images. Not only has scaling been found in different visual environments, but the phenomenon also appears to be calibration independent. This paper proposes a simple property of natural images which explains this ..."
Abstract

Cited by 105 (3 self)
 Add to MetaCart
One of the most robust qualities of our visual world is the scaleinvariance of natural images. Not only has scaling been found in different visual environments, but the phenomenon also appears to be calibration independent. This paper proposes a simple property of natural images which explains this robustness: They are collages of regions corresponding to statistically independent "objects". Evidence is provided for these objects having a powerlaw distribution of sizes within images, from which follows scaling in natural images. It is commonly suggested that scaling instead results from edges, each with power spectrum 1/k². This hypothesis is refuted by example.
Abstract Spectral Methods for MultiScale Feature Extraction and Data Clustering
, 2004
"... We address two issues that are fundamental to the analysis of naturallyoccurring datasets: how to extract features that arise at multiplescales and how to cluster items in a dataset using pairwise similarities between the elements. To this end we present two spectral methods: (1) Sparse Principal ..."
Abstract
 Add to MetaCart
(Show Context)
We address two issues that are fundamental to the analysis of naturallyoccurring datasets: how to extract features that arise at multiplescales and how to cluster items in a dataset using pairwise similarities between the elements. To this end we present two spectral methods: (1) Sparse Principal Component Analysis SPCA  a framework for learning a linear, orthonormal basis representation for structure intrinsic to a given dataset; and (2) EigenCuts  an algorithm for clustering items in a dataset using their pairwisesimilarities. SPCA is based on the discovery that natural images exhibit structure in a lowdimensional subspace in a local, scaledependent form. It is motivated by the observation that PCA does not typically recover such representations, due to its single minded pursuit of variance. In fact, it is widely believed that the analysis of secondorder statistics alone is insucient for extracting multiscale structure from data and there are many proposals in the literature showing how to harness higherorder image statistics to build multiscale representations. In this thesis, we show that resolving secondorder statistics with suitably constrained basis directions is indeed sucient to extract multiscale structure.
Printed in Great Britain 00426989/97 $17.00 + 0.00 Origins of Scaling in Natural Images
, 1996
"... One of the most robust qualities of our visual world is the scale invariance of natural images. Not only has scaling been found in different visual environments, but the phenomenon also appears to be calibrationindependent. This paper proposes a simple property of natural images which explains this ..."
Abstract
 Add to MetaCart
One of the most robust qualities of our visual world is the scale invariance of natural images. Not only has scaling been found in different visual environments, but the phenomenon also appears to be calibrationindependent. This paper proposes a simple property of natural images which explains this robustness: they are collages of regions corresponding to statistically independent "objects". Evidence is provided for these objects having a powerlaw distribution of sizes within images, from which follows scaling in natural images. It is commonly suggested that scaling instead results from edges, each with power spectrum 1/k2. This hypothesis is refuted by example. © 1997