Results 11  20
of
238
Markovian Models for Sequential Data
, 1996
"... Hidden Markov Models (HMMs) are statistical models of sequential data that have been used successfully in many machine learning applications, especially for speech recognition. Furthermore, in the last few years, many new and promising probabilistic models related to HMMs have been proposed. We firs ..."
Abstract

Cited by 119 (2 self)
 Add to MetaCart
Hidden Markov Models (HMMs) are statistical models of sequential data that have been used successfully in many machine learning applications, especially for speech recognition. Furthermore, in the last few years, many new and promising probabilistic models related to HMMs have been proposed. We first summarize the basics of HMMs, and then review several recent related learning algorithms and extensions of HMMs, including in particular hybrids of HMMs with artificial neural networks, InputOutput HMMs (which are conditional HMMs using neural networks to compute probabilities), weighted transducers, variablelength Markov models and Markov switching statespace models. Finally, we discuss some of the challenges of future research in this very active area. 1 Introduction Hidden Markov Models (HMMs) are statistical models of sequential data that have been used successfully in many applications in artificial intelligence, pattern recognition, speech recognition, and modeling of biological ...
Maximum Likelihood and Covariant Algorithms for Independent Component Analysis
, 1996
"... Bell and Sejnowski (1995) have derived a blind signal processing algorithm for a nonlinear feedforward network from an information maximization viewpoint. This paper first shows that the same algorithm can be viewed as a maximum likelihood algorithm for the optimization of a linear generative model ..."
Abstract

Cited by 117 (1 self)
 Add to MetaCart
Bell and Sejnowski (1995) have derived a blind signal processing algorithm for a nonlinear feedforward network from an information maximization viewpoint. This paper first shows that the same algorithm can be viewed as a maximum likelihood algorithm for the optimization of a linear generative model. Second, a covariant version of the algorithm is derived. This algorithm is simpler and somewhat more biologically plausible, involving no matrix inversions; and it converges in a smaller number of iterations. Third, this paper gives a partial proof of the `folktheorem' that any mixture of sources with highkurtosis histograms is separable by the classic ICA algorithm. Fourth, a collection of formulae are given that may be useful for the adaptation of the nonlinearity in the ICA algorithm. 1 Blind separation Algorithms for blind separation (Jutten and Herault 1991; Comon et al. 1991; Bell and Sejnowski 1995; Hendin et al. 1994) attempt to recover source signals s from observations x whic...
Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex
 Neural Computation
, 1995
"... this paper, we describe a hierarchical network model of visual recognition that explains these experimental observations by using a form of the extended Kalman filter as given by the Minimum Description Length (MDL) principle. The model dynamically combines inputdriven bottomup signals with expec ..."
Abstract

Cited by 113 (20 self)
 Add to MetaCart
this paper, we describe a hierarchical network model of visual recognition that explains these experimental observations by using a form of the extended Kalman filter as given by the Minimum Description Length (MDL) principle. The model dynamically combines inputdriven bottomup signals with expectationdriven topdown signals to predict current recognition state. Synaptic weights in the model are adapted in a Hebbian manner according to a learning rule also derived from the MDL principle. The resulting prediction/learning scheme can be viewed as implementing a form of the ExpectationMaximization (EM) algorithm. The architecture of the model posits an active computational role for the reciprocal connections between adjoining visual cortical areas in determining neural response properties. In particular, the model demonstrates the possible role of feedback from higher cortical areas in mediating neurophysiological effects due to stimuli from beyond the classical receptive field. Si
Representation is Representation of Similarities
 Behavioral and Brain Sciences
, 1996
"... Advanced perceptual systems are faced with the problem of securing a principled relationship between the world and its internal representation. I propose a unified approach to visual representation, based on Shepard's (1968) notion of secondorder isomorphism. According to the proposed theory, ..."
Abstract

Cited by 110 (21 self)
 Add to MetaCart
(Show Context)
Advanced perceptual systems are faced with the problem of securing a principled relationship between the world and its internal representation. I propose a unified approach to visual representation, based on Shepard's (1968) notion of secondorder isomorphism. According to the proposed theory, a shape is represented internally by the responses of a few tuned modules, each of which is broadly selective for some reference shape, whose similarity to the stimulus it measures. The result is a philosophically appealing, computationally feasible, biologically credible, and formally veridical representation of a distal shape space. This approach supports representation of and discrimination among shapes radically different from the reference ones, while bypassing the need for the computationally problematic decomposition into parts; it also addresses the needs of shape categorization, and can be used to derive a range of models of perceived similarity. Representation is Representation of Sim...
Bayesian computation in recurrent neural circuits
 Neural Computation
, 2004
"... A large number of human psychophysical results have been successfully explained in recent years using Bayesian models. However, the neural implementation of such models remains largely unclear. In this paper, we show that a network architecture commonly used to model the cerebral cortex can implem ..."
Abstract

Cited by 94 (4 self)
 Add to MetaCart
(Show Context)
A large number of human psychophysical results have been successfully explained in recent years using Bayesian models. However, the neural implementation of such models remains largely unclear. In this paper, we show that a network architecture commonly used to model the cerebral cortex can implement Bayesian inference for an arbitrary hidden Markov model. We illustrate the approach using an orientation discrimination task and a visual motion detection task. In the case of orientation discrimination, we show that the model network can infer the posterior distribution over orientations and correctly estimate stimulus orientation in the presence of significant noise. In the case of motion detection, we show that the resulting model network exhibits direction selectivity and correctly computes the posterior probabilities over motion direction and position. When used to solve the wellknown random dots motion discrimination task, the model generates responses that mimic the activities of evidenceaccumulating neurons in cortical areas LIP and FEF. The framework introduced in the paper posits a new interpretation of cortical activities in terms of log posterior probabilities of stimuli occurring in the natural world. 1 1
Exploring strategies for training deep neural networks
 Journal of Machine Learning Research
"... Département d’informatique et de recherche opérationnelle ..."
Abstract

Cited by 90 (12 self)
 Add to MetaCart
(Show Context)
Département d’informatique et de recherche opérationnelle
Tutorial on Variational Approximation Methods
 IN ADVANCED MEAN FIELD METHODS: THEORY AND PRACTICE
, 2000
"... We provide an introduction to the theory and use of variational methods for inference and estimation in the context of graphical models. Variational methods become useful as ecient approximate methods when the structure of the graph model no longer admits feasible exact probabilistic calculations. T ..."
Abstract

Cited by 88 (1 self)
 Add to MetaCart
(Show Context)
We provide an introduction to the theory and use of variational methods for inference and estimation in the context of graphical models. Variational methods become useful as ecient approximate methods when the structure of the graph model no longer admits feasible exact probabilistic calculations. The emphasis of this tutorial is on illustrating how inference and estimation problems can be transformed into variational form along with describing the resulting approximation algorithms and their properties insofar as these are currently known.
A Nonparametric MultiScale Statistical Model for Natural Images
 Advances in Neural Information Processing
, 1997
"... The observed distribution of natural images is far from uniform. On the contrary, real images have complex and important structure that can be exploited for image processing, recognition and analysis. There have been many proposed approaches to the principled statistical modeling of images, but each ..."
Abstract

Cited by 81 (2 self)
 Add to MetaCart
(Show Context)
The observed distribution of natural images is far from uniform. On the contrary, real images have complex and important structure that can be exploited for image processing, recognition and analysis. There have been many proposed approaches to the principled statistical modeling of images, but each has been limited in either the complexity of the models or the complexity of the images. We present a nonparametric multiscale statistical model for images that can be used for recognition, image denoising, and in a "generative mode" to synthesize high quality textures. Accepted Advanced in Neural Information Processing 10 (1997). 1 Introduction In this paper we describe a multiscale statistical model which can capture the structure of natural images across many scales. Once trained on example images, it can be used to recognize novel images, or to generate new images. Each of these tasks is reasonably efficient, requiring no more than a few seconds or minutes on a workstation. The sta...
Probabilistic Multimedia Objects (Multijects): A Novel Approach To Video Indexing And Retrieval In Multimedia Systems.
 In Proc. of ICIP
, 1998
"... This paper proposes a novel scheme for bridging the gap between low level media features and high level semantics using a probabilistic framework. We propose a framework, in which scenes can be indexed at a semantic level. The fundamental components of the framework are sites, objects and events. De ..."
Abstract

Cited by 80 (10 self)
 Add to MetaCart
(Show Context)
This paper proposes a novel scheme for bridging the gap between low level media features and high level semantics using a probabilistic framework. We propose a framework, in which scenes can be indexed at a semantic level. The fundamental components of the framework are sites, objects and events. Detection of presence of an instance of one of these influences the probability of the presence of instances within other classes. Detection of instances is done using probabilistic multimedia objects: Multijects. Indexing using Multijects can handle queries posed at semantic level. Multijects are built in a Markovian framework. Two ways of building the Multijects from low level features fusing features from multiple modalities are presented. A probabilistic framework is also envisioned to encode the higher level relationship between Multijects, which enhances or reduces the probabilities of concurrent existence of various Multijects. An actual implementation is presented by developing Multije...