Results 1  10
of
26
Transformationinvariant clustering using the EM algorithm
 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
, 2003
"... Clustering is a simple, effective way to derive useful representations of data, such as images and videos. Clustering explains the input as one of several prototypes, plus noise. In situations where each input has been randomly transformed (e.g., by translation, rotation, and shearing in images and ..."
Abstract

Cited by 73 (14 self)
 Add to MetaCart
(Show Context)
Clustering is a simple, effective way to derive useful representations of data, such as images and videos. Clustering explains the input as one of several prototypes, plus noise. In situations where each input has been randomly transformed (e.g., by translation, rotation, and shearing in images and videos), clustering techniques tend to extract cluster centers that account for variations in the input due to transformations, instead of more interesting and potentially useful structure. For example, if images from a video sequence of a person walking across a cluttered background are clustered, it would be more useful for the different clusters to represent different poses and expressions, instead of different positions of the person and different configurations of the background clutter. We describe a way to add transformation invariance to mixture models, by approximating the nonlinear transformation manifold by a discrete set of points. We show how the expectation maximization algorithm can be used to jointly learn clusters, while at the same time inferring the transformation associated with each input. We compare this technique with other methods for filtering noisy images obtained from a scanning electron microscope, clustering images from videos of faces into different categories of identification and pose and removing foreground obstructions from video. We also demonstrate that the new technique is quite insensitive to initial conditions and works better than standard techniques, even when the standard techniques are provided with extra data.
A comparison of algorithms for inference and learning in probabilistic graphical models
 IEEE Transactions on Pattern Analysis and Machine Intelligence
, 2005
"... Computer vision is currently one of the most exciting areas of artificial intelligence research, largely because it has recently become possible to record, store and process large amounts of visual data. While impressive achievements have been made in pattern classification problems such as handwr ..."
Abstract

Cited by 70 (4 self)
 Add to MetaCart
(Show Context)
Computer vision is currently one of the most exciting areas of artificial intelligence research, largely because it has recently become possible to record, store and process large amounts of visual data. While impressive achievements have been made in pattern classification problems such as handwritten character recognition and face detection, it is even more exciting that researchers may be on the verge of introducing computer vision systems that perform scene analysis, decomposing image input into its constituent objects, lighting conditions, motion patterns, and so on. Two of the main challenges in computer vision are finding efficient models of the physics of visual scenes and finding efficient algorithms for inference and learning in these models. In this paper, we advocate the use of graphbased probability models and their associated inference and learning algorithms for computer vision and scene analysis. We review exact techniques and various approximate, computationally efficient techniques, including iterative conditional modes, the expectation maximization (EM) algorithm, the mean field method, variational techniques, structured variational techniques, Gibbs sampling, the sumproduct algorithm and “loopy ” belief propagation. We describe how each technique can be applied in a model of multiple, occluding objects, and contrast the behaviors and performances of the techniques using a unifying cost function, free energy.
Continuous Speech Recognition Using Hidden Markov Models
 IEEE ASSP MAGAZINE
, 1990
"... Stochastic signal processing techniques have profoundly changed our perspective on speech processing. We have witnessed a progression from heuristic algorithms to detailed statistical approaches based on iterat ive analysis techniques. Markov modeling provides a mathematically rigorous approach t ..."
Abstract

Cited by 54 (9 self)
 Add to MetaCart
Stochastic signal processing techniques have profoundly changed our perspective on speech processing. We have witnessed a progression from heuristic algorithms to detailed statistical approaches based on iterat ive analysis techniques. Markov modeling provides a mathematically rigorous approach to developing robust s tat is t ica l signal models. Since t h e i n t roduc t i on of Markov models t o speech processing in t h e middle 1970s. continuous speech recognition technology has come of age. Dramatic advances have been made in characterizing the temporal and spectral evolution of the speech signal. A t the same time, our appreciation o f t he need to explain complex acoustic manifestations b y integration of application constraints in to low level signal processing has grown. In th is paper, w e review the use of Markov models in continuous speech recognition. Markov models are presented as a generalization of i t s predecessor technology, Dynamic Programming. A unified view is offered in which bo th linguistic decoding and acoustic matching are integrated in to a single optimal network search framework.
Recognizing handwritten digits using hierarchical products of experts
 IEEE Transactions on Pattern Analysis and Machine Intelligence
, 2001
"... Abstract—The product of experts learning procedure [1] can discover a set of stochastic binary features that constitute a nonlinear generative model of handwritten images of digits. The quality of generative models learned in this way can be assessed by learning a separate model for each class of di ..."
Abstract

Cited by 33 (5 self)
 Add to MetaCart
(Show Context)
Abstract—The product of experts learning procedure [1] can discover a set of stochastic binary features that constitute a nonlinear generative model of handwritten images of digits. The quality of generative models learned in this way can be assessed by learning a separate model for each class of digit and then comparing the unnormalized probabilities of test images under the 10 different classspecific models. To improve discriminative performance, a hierarchy of separate models can be learned for each digit class. Each model in the hierarchy learns a layer of binary feature detectors that model the probability distribution of vectors of activity of feature detectors in the layer below. The models in the hierarchy are trained sequentially and each model uses a layer of binary feature detectors to learn a generative model of the patterns of feature activities in the preceding layer. After training, each layer of feature dectectors produces a separate, unnormalized log probabilty score. With three layers of feature detectors for each of the 10 digit classes, a test image produces 30 scores which can be used as inputs to a supervised, logistic classification network that is trained on separate data. On the MNIST database, our system is comparable with current stateoftheart discriminative methods, demonstrating that the product of experts learning procedure can produce effective hierarchies of generative models of highdimensional data. Index Terms—Neural networks, products of experts, handwriting recognition, feature extraction, shape recognition, Boltzmann machines, modelbased recognition, generative models.
machine learning, and genetic neural nets
 Advances in Applied Mathematics
, 1989
"... We consider neural nets whose connections are defined by growth rules taking the form of recursion relations. These are called genetic neural nets. Learning in these nets is achieved by simulated annealing optimization of the net over the space of recursion relation parameters. The method is tested ..."
Abstract

Cited by 15 (1 self)
 Add to MetaCart
(Show Context)
We consider neural nets whose connections are defined by growth rules taking the form of recursion relations. These are called genetic neural nets. Learning in these nets is achieved by simulated annealing optimization of the net over the space of recursion relation parameters. The method is tested on a previously defined continuous coding problem. Results of control experiments are presented so that the success of the method can be judged. Genetic neural nets implement the ideas of scaling and parsimony, features which allow generalization in machine learning. © 1989 Academic Press, Inc. 1.
Recent Developments in Multilayer Perceptron Neural Networks
 Proceedings of the 7th Annual Memphis Area Engineering and Science Conference, MAESC
, 2005
"... Several neural network architectures have been developed over the past several years. One of the most popular and most powerful architectures is the multilayer perceptron. This architecture will be described in detail and recent advances in training of the multilayer perceptron will be presented. Mu ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
(Show Context)
Several neural network architectures have been developed over the past several years. One of the most popular and most powerful architectures is the multilayer perceptron. This architecture will be described in detail and recent advances in training of the multilayer perceptron will be presented. Multilayer perceptrons are trained using various techniques. For years the most used training method was back propagation and various derivatives of this to incorporate gradient information. Recent developments have used output weight optimizationhidden weight optimization (OWOHWO) and full conjugate gradient methods. OWOHWO is a very powerful technique in terms of accuracy and rapid convergence. OWOHWO has been used with a unique “network growing ” technique to ensure that the mean square error is monotonically nonincreasing as the network size increases (i.e., the number of hidden layer nodes increases). This “network growing ” technique was trained using OWOHWO but is amenable to any training technique. This technique significantly improves training and testing performance of the MLP.
Uncertainty and the Communication of Time
 Systems Research
, 1994
"... Prigogine and Stengers (1988) [47] have pointed to the centrality of the concepts of “time and eternity ” for the cosmology contained in Newtonian physics, but they have not addressed this issue beyond the domain of physics. The construction of “time ” in the cosmology dates back to debates among Hu ..."
Abstract

Cited by 7 (6 self)
 Add to MetaCart
(Show Context)
Prigogine and Stengers (1988) [47] have pointed to the centrality of the concepts of “time and eternity ” for the cosmology contained in Newtonian physics, but they have not addressed this issue beyond the domain of physics. The construction of “time ” in the cosmology dates back to debates among Huygens, Newton, and Leibniz. The deconstruction of this cosmology in terms of the philosophical questions of the 17th century suggests an uncertainty in the time dimension. While order has been conceived as an “harmonie préétablie, ” it is considered as emergent from an evolutionary perspective. In a “chaology”, one should fully appreciate that different systems may use different clocks. Communication systems can be considered as contingent in space and time: substances contain force or action, and they communicate not only in (observable) extension, but also over time. While each communication system can be considered as a system of reference for a special theory of communication, the addition of an evolutionary perspective to the mathematical theory of communication opens up the possibility of a general theory of communication.
Computational Analysis and Learning for a Biologically Motivated Model of Boundary Detection
"... In this work we address the problem of boundary detection by combining ideas and approaches from biological and computational vision. Initially, we propose a simple and efficient architecture that is inspired from models of biological vision. Subsequently, we interpret and learn the system using com ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
(Show Context)
In this work we address the problem of boundary detection by combining ideas and approaches from biological and computational vision. Initially, we propose a simple and efficient architecture that is inspired from models of biological vision. Subsequently, we interpret and learn the system using computer vision techniques: First, we present analogies between the system components and computer vision techniques and interpret the network as minimizing a cost functional, thereby establishing a link with variational techniques. Second, based on MeanField Theory the equations describing the network behavior are interpreted statistically. Third, we build on this interpretation to develop an algorithm to learn the network weights from manually segmented natural images. Using a systematic evaluation on the Berkeley benchmark we show that when using the learned connection weights our network outperforms classical edge detection algorithms.
Using sequence alignments to predict protein structure and stability with high accuracy. arXiv 1207.2484
 http://arxiv.org/abs/1207.2484 Lapedes AS, Giraud BG, Liu LC, Stormo GD
, 1999
"... ar ..."
(Show Context)