A Normalized Gaussian Network (NGnet) (Moody and Darken 1989) is a network of local linear regression units. The model softly partitions the input space by normalized Gaussian functions and each local unit linearly approximates the output within the partition. In this article, we propose a new on-line EM algorithm for the NGnet, which is derived from the batch EM algorithm (Xu, Jordan and Hinton 1995) by introducing a discount factor. We show that the on-line EM algorithm is equivalent to the batch EM algorithm if a specific scheduling of the discount factor is employed. In addition, we show that the on-line EM algorithm can be considered as a stochastic approximation method to find the maximum likelihood estimator. A new regularization method is proposed in order to deal with a singular input distribution. In order to manage dynamic environments, where the input-output distribution of data changes over time, unit manipulation mechanisms such as unit production, unit deletion...

Mixture of experts (ME) is a modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME architecture for adjusting the parameters and the iteratively reweighted least squares (IRLS) algorithm is used to perform maximization in the inner loop [Jordan, M.I., Jacobs, R.A. (1994). Hierarchical mixture of experts and the EM algorithm, Neural Computation, 6(2), 181--214]. However, it is reported in literature that the IRLS algorithm is of instability and the ME architecture trained by the EM algorithm, where IRLS algorithm is used in the inner loop, often produces the poor performance in multiclass classification. In this paper, the reason of this instability is explored. We find out that due to an implicitly imposed incorrect assumption on parameter independence in multiclass classification, an incomplete Hessian matrix is used in that IRLS algorithm. Based on this finding, we apply the Newton--Raphson met...

High-throughput methods can directly detect the set of interacting proteins in yeast but the results are often incomplete and exhibit high false positive and false negative rates. A number of researchers have recently presented methods for integrating direct and indirect data for predicting interactions. However, due to missing data and the high redundancy among the features used, different samples may benefit from different features based on the set of attributes available. In addition, in many cases it is hard to directly determine which of the datasets led to the prediction, which is an important issue for the biologists using these predications to design new experiments. To address these challenges we use a Mixture-of-Experts method. We split the data into four (roughly) homogeneous sets. The individual experts use logistic regression and their scores are combined using another logistic regression. However, when combining the scores the weighting of each expert depends on the set of input attributes. Thus different experts will have different influence on the prediction depending on the available features. We applied our method to predict the set of interacting proteins in yeast. Our method improved upon the best previous methods for this task. In addition, using the weighting of the experts the prediction can be easily evaluated by biologists based on the features that they feel are the most reliable. 1

in quality

genetic algorithms and neural networks

High-throughput methods can directly detect the set of interacting proteins in yeast but the results are often incomplete and exhibit high false positive and false negative rates. A number of researchers have recently presented methods for integrating direct and indirect data for predicting interactions. However, due to missing data and the high redundancy among the features used, di#erent samples may benefit from di#erent features based on the set of attributes available. In addition, in many cases it is hard to directly determine which of the datasets led to the prediction, which is an important issue for the biologists using these predications to design new experiments.

Today’s planetary exploration robots rarely travel beyond the yesterday imagery. However, advances in autonomous mobility will soon permit single-command site surveys of multiple kilometers. Here scientists cannot see the terrain in advance, and explorer robots must navigate and collect data autonomously. Onboard science data understanding can improve these surveys with image analysis, pattern recognition, learned classification, and information-theoretic planning. We report on field experiments near Amboy Crater, California, that demonstrate fundamental capabilities for autonomous surficial mapping of geologic phenomena with a visible near-infrared spectrometer. We develop an approach to “science on the fly ” that adapts the robot’s exploration using collected instrument data. We demonstrate feature detection and visual servoing to acquire spectra from dozens of targets without human intervention. The rover interprets spectra onboard, learning spatial models of science phenomena that guide it toward informative areas. It discovers spatial structure (correlations between neighboring regions) and cross-sensor structure (correlations between different scales). The rover uses surface observations to reinterpret satellite imagery and improve exploration efficiency. C ○ 2011 Wiley Periodicals, Inc. 1.

Most current machine vision systems suffer from a lack of flexibility to account for the high variability of unstructured environments. As the state of the world evolves, the potential knowledge provided by different visual attributes can change, breaking the initial assumptions of a non-adaptive vision system. This thesis develops a new comprehensive computational framework for the adaptive integration of information from different visual algorithms.

Dissertation submitted to the University of Cambridge for the degree of Doctor of Philosophy Although the performance of speech recognition systems has increased substantially over the last decades, there still remain a number of tasks which pose considerable problems for current state-of-the-art techniques. One of these tasks is the recognition of spontaneous speech which differs from read or planned speech in that its underlying dynamics change frequently over time. The negative effect of changes in acoustic background condition on recognition performance can also be observed in other situations as, for instance, in the case of speech that is corrupted by non-stationary noise. This thesis is concerned with the development of an acoustic model for speech recognition which automatically detects changes in the background condition of a signal and compensates for the model-data mismatch by combining the information of several expert models. These experts are specialised on the different acoustic conditions under consideration and their influ-ence on the recognition process is determined by how well their associated condition matches

. A mixture of experts consists of a gating network that learns to partition the input space and of experts networks attributed to these dierent regions. This paper focuses on the choice of the gating network. First, a localized gating network based on a mixture of linear latent variable models is proposed that extends a gating network introduced by Xu et al. [9], based on Gaussian mixture models. It is shown that this localized mixture of experts model, can be trained with the Expectation Maximization algorithm. The localized model is compared on a set of classication problems, with mixtures of experts having single or multilayer perceptrons as gating network. It is found that the standard mixture of experts with feedforward networks as gate often outperforms the other models. 1 Introduction A mixture of experts [5] is a probabilistic model that can be interpreted as a mixture model for estimating conditional probability distributions. The model consists of a gating network that ...