We present a system that takes realistic magnetoencephalographic (MEG) signals and localizes a single dipole to reasonable accuracy in real time. At its heart is a multilayer perceptron (MLP) which takes the sensor measurements as inputs, uses one hidden layer, and generates as outputs the amplitudes of receptive fields holding a distributed representation of the dipole location. We trained this Soft-MLP on dipolar sources with real brain noise and converted the network's output into an explicit Cartesian coordinate representation of the dipole location using two different decoding strategies. The proposed Soft-MLPs are much more accurate than previous networks which output source locations in Cartesian coordinates. Hybrid Soft-MLP-start-LM systems, in which the Soft-MLP output initializes Levenberg-Marquardt, retained their accuracy of 0.28 cm with a decrease in computation time from 36 ms to 30 ms. We apply the SoftMLP localizer to real MEG data separated by a blind source separation algorithm, and compare the Soft-MLP dipole locations to those of a conventional system.

This paper introduces a learning method for two-layer feedforward neural networks based on sensitivity analysis, which uses a linear training algorithm for each of the two layers. First, random values are assigned to the outputs of the first layer; later, these initial values are updated based on sensitivity formulas, which use the weights in each of the layers; the process is repeated until convergence. Since these

We describe a system that localizes a single dipole to reasonable accuracy from noisy magnetoencephalographic (MEG) measurements in real time. At its core is a multilayer perceptron (MLP) trained to map sensor signals and head position to dipole location. Including head position overcomes the previous need to retrain the MLP for each subject and session. The training dataset was generated by mapping randomly chosen dipoles and head positions through an analytic model and adding noise from real MEG recordings. After training, a localization took 0.7 ms with an average error of 0.90 cm. A few iterations of a Levenberg-Marquardt routine using the MLP output as its initial guess took 15 ms and improved accuracy to 0.53 cm, which approaches the natural limit on accuracy imposed by noise. We applied these methods to localize single dipole sources from MEG components isolated by blind source separation and compared the estimated locations to those generated by standard manually assisted commercial software. Hum Brain Mapp 24:21--34, 2005. 2004 Wiley-Liss, Inc.

: A set of new algorithms based on an on-line implementation of a well known global optimization strategy based on stochastic functional smoothing are proposed for training neural networks. These algorithms are different from other on-line global optimization approaches because they use not only first-order, but also second-order (Hessian) gradient information. Therefore, they have faster convergence than first-order gradient descent search methods. Convergence and sensitivity analysis of the proposed method are provided. The on-line algorithms are compared with second-order gradient method, momentum learning and conjugate gradients in order to claim their consistent and global convergence abilities; and are compared with conventional stochastic global optimization scheme in order to claim their faster learning rate. Computer simulation results are presented to support the analysis. Keywords---stochastic functional smoothing, on-line algorithms, global convergence, mean square error, ...

Source localization from MEG data in real time requires algorithms which are robust, fully automatic, and very fast. We present two neural network systems which are able to localize a single dipole to reasonable accuracy within a fraction of a millisecond, even when the signals are contaminated by considerable noise. The first network is a multilayer perceptron (MLP) which takes the sensor measurements as inputs, uses two hidden layers, and outputs source location in Cartesian coordinates. After training with random dipolar sources contaminated by real noise, localization of a single dipole could be performed within 300 microseconds on an 800 Mhz Athlon workstation, with an average localization error of 1.15 cm. To improve the accuracy to 0.28 cm, one can apply a few iterations of conventional Levenberg-Marquardt (LM) minimization using the MLP output as the initial guess. The combined method is about twenty times faster than multistart LM localization with comparable accuracy. In a second network with only one hidden layer, the outputs were the amplitudes of 193 evenly distributed Gaussian functions holding a soft distributed representation of the dipole location. We trained this network on dipolar sources with real noise, and externally converted the network's output into an explicit Cartesian coordinate representation of the dipole location. This new network had an improved localization accuracy of 0.87 cm, while localization time was lengthened to about 800 microseconds.

Iterative gradient methods like Levenberg-Marquardt (LM) are in widespread use for source localization from electroencephalographic (EEG) and magnetoencephalographic (MEG) signals. Unfortunately LM depends sensitively on the initial guess, necessitating repeated runs. This, combined with LM's high per-step cost, makes its computational burden quite high. To reduce this burden, we trained a multilayer perceptron (MLP) as a real-time localizer. We used an analytical model of quasistatic electromagnetic propagation through a spherical head to map randomly chosen dipoles to sensor activities according to the sensor geometry of a 4D Neuroimaging Neuromag-122 MEG system, and trained a MLP to invert this mapping in the absence of noise or in the presence of various sorts of noise such as white Gaussian noise, correlated noise, or real brain noise. A MLP structure was chosen to trade off computation and accuracy. This MLP was trained four times, with each type of noise. We measured the effects of initial guesses on LM performance, which motivated a hybrid MLPstart -LM method, in which the trained MLP initializes LM. We also compared the localization performance of LM, MLPs, and hybrid MLP-start-LMs for realistic brain signals. Trained MLPs are much faster than other methods, while the hybrid MLP-start-LMs are faster and more accurate than fixed-4-start-LM. In particular, the hybrid MLP-start-LM initialized by a MLP trained with the real brain noise dataset is 60 times faster and is comparable in accuracy to random-20-start-LM, and this hybrid system (localization error: 0.28 cm, computation time: 36 ms) shows almost as good performance as optimal-1-start-LM (localization error: 0.23 cm, computation time: 22 ms), which initializes LM with the correct dipole location. MLPs trai...

Log-linear models are widely used probability models for statistical pattern recognition. Typically, log-linear models are trained according to a convex criterion. In recent years, the interest in log-linear models has greatly increased. The optimization of log-linear model parameters is costly and therefore an important topic, in particular for large-scale applications. Different optimization algorithms have been evaluated empirically in many papers. In this work, we analyze the optimization problem analytically and show that the training of log-linear models can be highly ill-conditioned. We verify our findings on two handwriting tasks. By making use of our convergence analysis, we obtain good results on a large-scale continuous handwriting recognition task with a simple and generic approach. 1

An important issue with many sequential data analysis problems, such as those encountered in nfiancial data sets, is that different variables are known at different frequencies, at different times (asynchronicity), or are sometimes missing. To address this issue we propose to use recurrent networks with feedback into the input units, based on two fundamental ideas. The first motivation is that the "filled-in" value of the missing variable may not only depend in complicated ways on the value of this variable in the past of the sequence but also on the current and past values of other variables. The second motivation is that, for the purpose of making predictions or taking decisions, it is not always necessary to fill in the best possible value of the missing variables. In fact, it is sufficient to fill in a value which helps the system make better predictions or decisions. The advantages of this approach are demonstrated through experiments on several tasks.