We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent/sampling importance resampling algorithm (HySIR). In terms of both computational time and accuracy, the hybrid SIR is a clear improvement over conventional sequential Monte Carlo techniques. The new algorithm may be viewed as a global optimisation strategy, which allows us to learn the probability distributions of the network weights and outputs in a sequential framework. It is well suited to applications involving on-line, nonlinear and non-Gaussian signal processing. We show how the new algorithm outperforms extended Kalman filter training on several problems. In particular, we address the problem of pricing option contracts, traded in financial markets. In this context, we are able to estimate the one-step-ahead probability density functions of the options prices. i Contents 1 Introduction 1 2 State Space Neural Network Modelling 2 3 The Bayes...

In this paper, we show that a hierarchical Bayesian modelling approach to sequential learning leads to many interesting attributes such as regularisation and automatic relevance determination. We identify three inference levels within this hierarchy, namely model selection, parameter estimation and noise estimation. In environments where data arrives sequentially, techniques such as cross-validation to achieve regularisation or model selection are not possible. The Bayesian approach, with extended Kalman filtering at the parameter estimation level, allows for regularisation within a minimum variance framework. A multi-layer perceptron is used to generate the extended Kalman filter nonlinear measurements mapping. We describe several algorithms at the noise estimation level, which allow us to implement adaptive regularisation and automatic relevance determination of model inputs and basis functions. An important contribution of this paper is to show the theoretical links between adaptive...

In this paper, we derive an algorithm to train support vector machines sequentially. The algorithm makes use of the Kalman filter and is optimal in a minimum variance framework. It extends the support vector machine paradigm to applications involving real-time and non-stationary signal processing. It also provides a computationally efficient alternative to the problem of quadratic optimisation.

We consider sequential or online learning in dynamic neural regression models. By using a state space representation for the neural network's parameter evolution in time we obtain approximations to the unknown posterior by either deriving posterior modes via the Fisher scoring algorithm or by deriving approximate posterior means with the importance sampling method. Furthermore, we replace the commonly used Gaussian noise assumption in the neural regression model by a more flexible noise model based on the Student t-density. Since the t-density can be interpreted as being an infinite mixture of Gaussians, hyperparameters such as the degrees of freedom of the t-density can be learned from the data based on an online EM-type algorithm. We show experimentally that our novel methods outperform state-of-the art neural network online learning algorithms like the extended Kalman filter method for both, situations with standard Gaussian noise terms and situations with measurement outliers. 1 I...

Abstract — This paper develops an unscented grid-based filter and a smoother for accurate nonlinear modeling and analysis of time series. The filter uses unscented deterministic sampling during both the time and measurement updating phases, to approximate directly the distributions of the latent state variable. A complementary grid smoother is also made to enable computing of the likelihood. This helps us to formulate an expectation maximisation algorithm for maximum likelihood estimation of the state noise and the observation noise. Empirical investigations show that the proposed unscented grid filter/smoother compares favourably to other similar filters on nonlinear estimation tasks. I.

Volatility modeling is the lifeline of the derivative- and asset-pricing evaluation process. As such, it is understandable that a voluminous literature has evolved to discuss the temporal dependencies in financial market volatility. Much of this literature has been directed at daily and lower frequencies using ARCH and stochastic volatility type models. With access to high frequency and ultra high-frequency databases, more recent research has been able to explain about fifty percent of the interdaily forecasts of latent volatility. Relying upon hourly intervals, the GARCH(1,1) results presented here are consistent with prior studies. However, this paper adds to the tools available for conducting volatility exploration by introducing an adaptive radial basis function neural network that significantly lowers overall prediction error while maintaining a high explanatory ratio. The newly formulated RBF implements a closed-form regularization parameter with Bayesian prior information. It is an algorithmic extension that will permit more accurate and insightful analyses to be performed on high frequency financial time series. Over the past decade, research efforts increased significantly in the area of modeling volatility behavior in capital market high frequency data. Obtaining accurate

Over the recent past, stylized facts have not yielded a synthesis regarding the predictability of returns for alternative investment assets such as hedge funds. Recent studies on alternative asset return predictability have added to the ambiguity. These studies suggest that classification prediction methods may dominate more traditional return-level prediction methodology. This paper examines the predictive accuracy of three alternate radial basis function neural networks when applied to the returns of thirteen Credit Swiss First Boston/Tremont (CSFB) hedge fund indices. We provide evidence that the Kajiji-4 RBF neural network dominates within the RBF topology in the prediction of hedge fund returns by both level and classification. The results also show that the Kajiji-4 method is capable of near perfect directional prediction.