Results 1  10
of
13
Exponential family harmoniums with an application to . . .
"... Directed graphical models with one layer of observed random variablesand one or more layers of hidden random variables have been the dominant modelling paradigm in many research fields. Although this approach has met with considerable success, the causal semantics of these models can make it diffi ..."
Abstract

Cited by 148 (22 self)
 Add to MetaCart
(Show Context)
Directed graphical models with one layer of observed random variablesand one or more layers of hidden random variables have been the dominant modelling paradigm in many research fields. Although this approach has met with considerable success, the causal semantics of these models can make it difficult to infer the posterior distribution over thehidden variables. In this paper we propose an alternative twolayer model based on exponential family distributions and the semantics of undirected models. Inference in these "exponential family harmoniums " is fast while learning is performed by minimizing contrastive divergence.A member of this family is then studied as an alternative probabilistic model for latent semantic indexing. In experiments it is shown that theyperform well on document retrieval tasks and provide an elegant solution to searching with keywords.
Objectbased attention and occlusion: Evidence from normal participants and a computational model
 Journal of Experimental Psychology: Human Perception and Performance
, 1998
"... One way of perceptually organizing a complex visual scene is to attend selectively to information in a particular physical location. Another way of reducing the complexity in the input is to attend selectively to an individual object in the scene and to process its elements preferentially. This latt ..."
Abstract

Cited by 89 (13 self)
 Add to MetaCart
One way of perceptually organizing a complex visual scene is to attend selectively to information in a particular physical location. Another way of reducing the complexity in the input is to attend selectively to an individual object in the scene and to process its elements preferentially. This latter, objectbased attention process was examined, and the predicted superiority for reporting features from 1 relative to 2 objects was replicated in a series of experiments. This objectbased process was robust even under conditions of occlusion, although there were some boundary conditions on its operation. Finally, an account of the data is provided via simulations of the findings in a computational model. The claim is that objectbased attention arises from a mechanism that groups together those features based on internal representations developed over perceptual experience and then preferentially gates these features for later, selective processing. Humans are exceptionally good at recognizing objects in natural visual scenes despite the fact that such scenes usually contain multiple, overlapping objects. One way in which individuals organize this complex input to minimize the
Transformation Equivariant Boltzmann Machines
"... Abstract. We develop a novel modeling framework for Boltzmann machines, augmenting each hidden unit with a latent transformation assignment variable which describes the selection of the transformed view of the canonical connection weights associated with the unit. This enables the inferences of the ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Abstract. We develop a novel modeling framework for Boltzmann machines, augmenting each hidden unit with a latent transformation assignment variable which describes the selection of the transformed view of the canonical connection weights associated with the unit. This enables the inferences of the model to transform in response to transformed input data in a stable and predictable way, and avoids learning multiple features differing only with respect to the set of transformations. Extending prior work on translation equivariant (convolutional) models, we develop translation and rotation equivariant restricted Boltzmann machines (RBMs) and deep belief nets (DBNs), and demonstrate their effectiveness in learning frequently occurring statistical structure from artificial and natural images.
Unsupervised segmentation with dynamical units
 IEEE Trans Neural Netw
, 1999
"... Abstract—In this paper, we present a novel network to separate mixtures of inputs that have been previously learned. A significant capability of the network is that it segments the components of each input object that most contribute to its classification. The network consists of amplitudephase uni ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Abstract—In this paper, we present a novel network to separate mixtures of inputs that have been previously learned. A significant capability of the network is that it segments the components of each input object that most contribute to its classification. The network consists of amplitudephase units that can synchronize their dynamics, so that separation is determined by the amplitude of units in an output layer, and segmentation by phase similarity between input and output layer units. Learning is unsupervised and based on a Hebbian update, and the architecture is very simple. Moreover, efficient segmentation can be achieved even when there is considerable superposition of the inputs. The network dynamics are derived from an objective function that rewards sparse coding in the generalized amplitudephase variables. We argue that this objective function can provide a possible formal interpretation of the binding problem and that the implementation of the network architecture and dynamics is biologically plausible. Index Terms—Binding problem, deconvolution, oscillations, phase correlation, separation of mixtures, synchronization.
The von Mises Graphical Model: Structure Learning
, 2011
"... The von Mises distribution is a continuous probability distribution on the circle used in directional statistics. In this paper, we introduce the undirected von Mises Graphical model and present an algorithm for structure learning using L1 regularization. We show that the learning algorithm is both ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
The von Mises distribution is a continuous probability distribution on the circle used in directional statistics. In this paper, we introduce the undirected von Mises Graphical model and present an algorithm for structure learning using L1 regularization. We show that the learning algorithm is both consistent and efficient. We also introduce a simple inference algorithm based on Gibbs sampling. We compare and contrast the von Mises Graphical Model (VGM) with a Gaussian Graphical Model (GGM) on both synthetic data and on data from protein structures and demonstrate that the The von Mises distribution is used in directional statistics to model angles and other circularlydistributed variables [3]. It closely approximates the wrapped normal distribution, but has the advantage that it is more tractable, mathematically [9]. Additionally, the von Mises distribution can be generalized to distributions over the (p − 1)dimensional sphere in R p, where it is known
The von Mises Graphical Model: Regularized Structure and Parameter Learning
, 2011
"... The von Mises distribution is a continuous probability distribution on the circle used in directional statistics. In this paper, we introduce the undirected von Mises Graphical model and present an algorithm for parameter and structure learning using L1 regularization. We show that the learning algo ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
The von Mises distribution is a continuous probability distribution on the circle used in directional statistics. In this paper, we introduce the undirected von Mises Graphical model and present an algorithm for parameter and structure learning using L1 regularization. We show that the learning algorithm is both consistent and statistically efficient. Additionally, we introduce a simple inference algorithm based on Gibbs sampling. We compare the von Mises Graphical Model (VGM) with a Gaussian Graphical Model (GGM) on both synthetic data and on data from protein structures, and demonstrate that the VGM achieves higher accuracy than the GGM.
1 A Mixture Model For Population codes of Gabor Filters
"... Abstract—Population coding is a coding scheme which is ubiquitous in neural systems, and is also of more general use in coding stimuli, for example in vision problems. A population of responses to a stimulus can be used to represent not only the value of some variable in the environment, but a full ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract—Population coding is a coding scheme which is ubiquitous in neural systems, and is also of more general use in coding stimuli, for example in vision problems. A population of responses to a stimulus can be used to represent not only the value of some variable in the environment, but a full probability distribution for that variable. The information is held in a distributed and encoded form which may in some situations be more robust to noise and failures than conventional representations. Gabor filters are a popular choice for detecting edges in the visual field for several reasons. They are easily tuned for a variety of edge widths and orientations, and are considered a close model of the edge filters in the human visual system. In this paper we consider population codes of Gabor filters with different orientations. A probabilistic model of Gabor filter responses is presented. Based on the analytically derived orientation tuning function and a parametric mixture model of the filter responses in the presence of local edge structure with single or multiple orientations a probability density function of the local orientation in any point (x, y) can be extracted through a parameter estimation procedure. The resulting pdf of the local contour orientation captures not only angular information at edges, corners or Tjunctions but also describes the certainty of the measurement which can be characterized in terms of the entropy of the individual mixture components.
A Phasor Model with Resting States
, 2000
"... this paper. The model considered here wil be referred to as phasor model with resting states. ..."
Abstract
 Add to MetaCart
this paper. The model considered here wil be referred to as phasor model with resting states.
ObjectBased Attention and Occlusion: Evidence From Normal Participants and a Computational Model
"... One way of perceptually organizing a complex visual scene is to attend selectively to information in a particular physical location. Another way of reducing the complexity in the input is to attend selectively to an individual object in the scene and to process its elements preferentially. This latt ..."
Abstract
 Add to MetaCart
(Show Context)
One way of perceptually organizing a complex visual scene is to attend selectively to information in a particular physical location. Another way of reducing the complexity in the input is to attend selectively to an individual object in the scene and to process its elements preferentially. This latter, objectbased attention process was examined, and the predicted superiority for reporting features from 1 relative to 2 objects was replicated in a series of experiments. This objectbased process was robust even under conditions of occlusion, although there were some boundary conditions on its operation. Finally, an account of the data is provided via simulations of the findings in a computational model. The claim is that objectbased attention arises from a mechanism that groups together those features based on internal representations developed over perceptual experience and then preferentially gates these features for later, selective processing. Humans are exceptionally good at recognizing objects in natural visual scenes despite the fact that such scenes usually contain multiple, overlapping objects. One way in which individuals organize this complex input to minimize the processing load is to divide the field on the basis of spatial location and then to attend selectively to particular physical regions. This selective attentional spotlight "illuminates" areas of interest and facilitates preferential processing of information from those chosen areas (e.g., Broadbent, 1982;
Exponential family harmoniums with an . . .
"... Directed graphical models with one layer of observed random variables and one or more layers of hidden random variables have been the dominant modelling paradigm in many research fields. Although this approach has met with considerable success, the causal semantics of these models can make it diffic ..."
Abstract
 Add to MetaCart
Directed graphical models with one layer of observed random variables and one or more layers of hidden random variables have been the dominant modelling paradigm in many research fields. Although this approach has met with considerable success, the causal semantics of these models can make it difficult to infer the posterior distribution over the hidden variables. In this paper we propose an alternative twolayer model based on exponential family distributions and the semantics of undirected models. Inference in these âexponential family harmoniumsâ is fast while learning is performed by minimizing contrastive divergence. A member of this family is then studied as an alternative probabilistic model for latent semantic indexing. In experiments it is shown that they perform well on document retrieval tasks and provide an elegant solution to searching with keywords.