We propose e#cient particle smoothing methods for generalized state-spaces models.

In this paper we propose a new particle smoother that has a computational complexity of O(N), where N is the number of particles. This compares favourably with the O(N 2) computational cost of most smoothers and will result in faster rates of convergence for fixed computational cost. The new method also overcomes some of the degeneracy problems we identify in many existing algorithms. Through simulation studies we show that substantial gains in efficiency are obtained for practical amounts of computational cost. It is shown both through these simulation studies, and on the analysis of an athletics data set, that our new method also substantially outperforms the simple Filter-Smoother (the only other smoother with computational cost that is linear in the number of particles). 1

This paper is concerned with the parameter estimation of a relatively general class of nonlinear dynamic systems. A Maximum Likelihood (ML) framework is employed, and it is illustrated how an Expectation Maximisation (EM) algorithm may be used to compute these ML estimates. An essential ingredient is the employment of so-called “particle smoothing” methods to compute required conditional expectations via a sequential Monte Carlo approach. Simulation examples demonstrate the efficacy of these techniques.

Neural activity unfolding over time can be modeled using non-linear dynamical systems [1]. As neurons communicate via discrete action potentials, their activity can be characterized by the numbers of events occurring within short predefined time-bins (spike counts). Because the observed data are high-dimensional vectors of non-negative integers, nonlinear state estimation from spike counts presents a unique set of challenges. In this paper, we describe why the expectation propagation (EP) framework is particularly well-suited to this problem. We then demonstrate ways to improve the robustness and accuracy of Gaussian quadrature-based EP. Compared to the unscented Kalman smoother, we find that EPbased state estimators provide more accurate state estimates. 1.

We describe Bayesian learning in nonlinear state-space models (NSSMs). NSSMs are a general method for the probabilistic modelling of sequences and time-series. They take the form of iterated maps on continuous state-spaces, and can have either discrete or continuous valued output functions. They are generalizations of the more well known state-space models such as Hidden Markov models (HMMs), and Linear-Gaussian statespace models (LSSMs). In this paper, we describe the problems of Bayesian learning and inference and in NSSMs. We present an MCMC methods of sampling from the posterior of the parameters given observed data.

Abstract — The primary contribution of this paper is an algorithm capable of identifying parameters in certain mixed linear/nonlinear state-space models, containing conditionally linear Gaussian substructures. More specifically, we employ the standard maximum likelihood framework and derive an expectation maximization type algorithm. This involves a nonlinear smoothing problem for the state variables, which for the conditionally linear Gaussian system can be efficiently solved using a so called Rao-Blackwellized particle smoother (RBPS). As a secondary contribution of this paper we extend an existing RBPS to be able to handle the fully interconnected model under study. I.

Abstract: In this article, we develop a new Rao-Blackwellized Monte Carlo smoothing algorithm for conditionally linear Gaussian models. The algorithm is based on the forwardfiltering backward-simulation Monte Carlo smoother concept and performs the backward simulation directly in the marginal space of the non-Gaussian state component while treating the linear part analytically. Unlike the previously proposed backward-simulation based Rao-Blackwellized smoothing approaches, it does not require sampling of the Gaussian state component and is also able to overcome certain normalization problems of two-filter smoother based approaches. The performance of the algorithm is illustrated in a simulated application.

This paper is about the estimation of fixed model parameters in hidden Markov models using an online (or recursive) version of the Expectation-Maximization (EM) algorithm. It is first shown that under suitable mixing assumptions, the large sample behavior of the traditional (batch) EM algorithm may be analyzed through the notion of a limiting EM recursion, which is deterministic. This observation generalizes results previously obtained for latent data model with independent observations. By using the recursive implementation of smoothing computations associated with sum functionals of the hidden state, it is then possible to propose an online EM algorithm that generalizes an approach recently proposed in the case of HMMs with finite-valued observations. The performance of the proposed algorithm is numerically evaluated through simulations in the case of a noisily observed Markov chain.

Abstract Two-filter smoothing is a principled approach for performing optimal smoothing in non-linear non-Gaussian state–space models where the smoothing distributions are computed through the combination of ‘forward ’ and ‘backward ’ time filters. The ‘forward ’ filter is the standard Bayesian filter but the ‘backward ’ filter, generally referred to as the backward information filter, is not a probability measure on the space of the hidden Markov process. In cases where the backward information filter can be computed in closed form, this technical point is not important. However, for general state–space models where there is no closed form expression, this prohibits the use of flexible numerical techniques such as Sequential Monte Carlo (SMC) to approximate the two-filter smoothing formula. We propose here a generalised twofilter smoothing formula which only requires approximating probability distributions and applies to any state–space model, removing the need to make restrictive assumptions used in previous approaches to this problem. SMC algorithms are developed to implement this generalised recursion and we illustrate their performance on various problems.

Inference in graphical models has emerged as a promising technique for planning. A recent approach to decision-theoretic planning in relational domains uses forward inference in dynamic Bayesian networks compiled from learned probabilistic relational rules. Inspired by work in non-relational domains with small state spaces, we derive a backpropagation method for such nets in relational domains starting from a goal state mixture distribution. We combine this with forward reasoning in a bidirectional two-filter approach. We perform experiments in a complex 3D simulated desktop environment with an articulated manipulator and realistic physics. Empirical results show that bidirectional probabilistic reasoning can lead to more efficient and accurate planning in comparison to pure forward reasoning. 1.