Increasingly, for many application areas, it is becoming important to include elements of nonlinearity and non-Gaussianity in order to model accurately the underlying dynamics of a physical system. Moreover, it is typically crucial to process data on-line as it arrives, both from the point of view of storage costs as well as for rapid adaptation to changing signal characteristics. In this paper, we review both optimal and suboptimal Bayesian algorithms for nonlinear/non-Gaussian tracking problems, with a focus on particle filters. Particle filters are sequential Monte Carlo methods based on point mass (or “particle”) representations of probability densities, which can be applied to any state-space model and which generalize the traditional Kalman filtering methods. Several variants of the particle filter such as SIR, ASIR, and RPF are introduced within a generic framework of the sequential importance sampling (SIS) algorithm. These are discussed and compared with the standard EKF through an illustrative example.

Modelling sequential data is important in many areas of science and engineering. Hidden Markov models (HMMs) and Kalman filter models (KFMs) are popular for this because they are simple and flexible. For example, HMMs have been used for speech recognition and bio-sequence analysis, and KFMs have been used for problems ranging from tracking planes and missiles to predicting the economy. However, HMMs and KFMs are limited in their “expressive power”. Dynamic Bayesian Networks (DBNs) generalize HMMs by allowing the state space to be represented in factored form, instead of as a single discrete random variable. DBNs generalize KFMs by allowing arbitrary probability distributions, not just (unimodal) linear-Gaussian. In this thesis, I will discuss how to represent many different kinds of models as DBNs, how to perform exact and approximate inference in DBNs, and how to learn DBN models from sequential data. In particular, the main novel technical contributions of this thesis are as follows: a way of representing Hierarchical HMMs as DBNs, which enables inference to be done in O(T) time instead of O(T 3), where T is the length of the sequence; an exact smoothing algorithm that takes O(log T) space instead of O(T); a simple way of using the junction tree algorithm for online inference in DBNs; new complexity bounds on exact online inference in DBNs; a new deterministic approximate inference algorithm called factored frontier; an analysis of the relationship between the BK algorithm and loopy belief propagation; a way of applying Rao-Blackwellised particle filtering to DBNs in general, and the SLAM (simultaneous localization and mapping) problem in particular; a way of extending the structural EM algorithm to DBNs; and a variety of different applications of DBNs. However, perhaps the main value of the thesis is its catholic presentation of the field of sequential data modelling.

Bayesian estimation problems where the posterior distribution evolves over time through the accumulation of data arise in many applications in statistics and related fields. Recently, a large number of algorithms and applications based on sequential Monte Carlo methods (also known as particle filtering methods) have appeared in the literature to solve this class of problems; see (Doucet, de Freitas & Gordon, 2001) for a survey. However, few of these methods have been proved to converge rigorously. The purpose of this paper is to address this issue. We present a general sequential Monte Carlo (SMC) method which includes most of the important features present in current SMC methods. This method generalizes and encompasses many recent algorithms. Under mild regularity conditions, we obtain rigorous convergence results for this general SMC method and therefore give theoretical backing for the validity of all the algorithms that can be obtained as particular cases of it. Keywords: Bayesian...

Particle filters are used for hidden state estimation with nonlinear dynamical systems. The inference of 3-d human motion is a natural application, given the nonlinear dynamics of the body and the nonlinear relation between states and image observations. However, the application of particle filters has been limited to cases where the number of state variables is relatively small, because the number of samples needed with high dimensional problems can be prohibitive. We describe a filter that uses hybrid Monte Carlo (HMC) to obtain samples in high dimensional spaces. It uses multiple Markov chains that use posterior gradients to rapidly explore the state space, yielding fair samples from the posterior. We find that the HMC filter is several thousand times faster than a conventional particle filter on a 28D people tracking problem.

We develop methods for performing smoothing computations in general state-space models. The methods rely on a particle representation of the filtering distributions, and their evolution through time using sequential importance sampling and resampling ideas. In particular, novel techniques are presented for generation of sample realizations of historical state sequences. This is carried out in a forwardfiltering backward-smoothing procedure which can be viewed as the non-linear, non-Gaussian counterpart of standard Kalman filter-based simulation smoothers in the linear Gaussian case. Convergence in the mean-squared error sense of the smoothed trajectories is proved, showing the validity of our proposed method. The methods are tested in a substantial application for the processing of speech signals represented by a time-varying autoregression and parameterised in terms of timevarying partial correlation coe#cients, comparing the results of our algorithm with those from a simple smoother based upon the filtered trajectories.

This article is available in electronic form on the ISDS web site, http://www.stat.duke.edu

this paper, we shall concentrate on the following class of state space models.Let t 1; 2;... denote discrete time: then t x t-1 B t v t F t u t ;x 0 .x 0 ;P 0 /; .1/ C t x t t " t t u t ;.2/ y t /;. 3/ where u t n u is an exogenous process and x t n x and y t n y are unobserved processes. The sequences v t .0;I n v / n v " t .0;I n " / n " are independent identically distributed (IID) Gaussian.We assume that P 0 > 0; x 0 ;v t and w t are mutually independent for all t, and the model parameters # # . x 0 ;P 0;A t ;B t ;C t ;D t ;F t ;G t 1; 2;... / arekU wn.The processes .x t / and .y t / define a standard linear Gaussian state space model.We do not observe .y t / in our case, but .z t /.The observations .z t / are conditionally independent given the processes .x t / and .y t / and marginally distributed according to p.z t t /; it is assumed that p.z t t / can be evaluated pointwise up to a normalizing constant.Typically p.z t t / belongs to the exponential family.Alternatively z t may be a censored or quantized version of y t . This class of partially observed Gaussian state space models has numerous applications; many examples are discussed for instance in de Jong (1997), Manrique and Shephard (1998) and West and Harrison (1997). We want to estimate sequentially in time some characteristics of the posterior distribution 1:t /.Typically, we are interested in computing E.x t 1:t / (filtering), E.x t+L 1:t / (prediction) 1:t / (fixed lag smoothing), where L is a positive integer.These estimates do not in general admit analytical expressions and we must resort to numerical methods

In this paper particle filters for dynamic state space models handling unknown static parameters are discussed. The approach is based on marginalizing the static parameters out of the posterior distribution such that only the state vector needs to be considered. Such a marginalization can always be applied. However, real-time applications are only possible when the distribution of the unknown parameters given both observations and the hidden state vector depends on some low-dimensional sufficient statistics. Such sufficient statistics are present in many of the commonly used state space models. Marginalizing the static parameters avoids the problem of impoverishment which typically occur when static parameters are included as part of the state vector. The filters are tested on several different models, with promising results.

We develop methods for performing maximum a posteriori (MAP) se- quence estimation in non-linear non-Gaussian dynamic models. The methods rely on a particle cloud representation of the filtering distribution which evolves through time using importance sampling and resampling ideas. MAP sequence estimation is then performed using a classical dynamic programming technique applied to the discretised version of the state space. In contrast with standard approaches to the problem which essentially compare only the trajectories generated directly during the filtering stage, our method efficiently computes the optimal trajectory over all combinations of the filtered states. A particular strength of the method is that MAP sequence estimation is performed sequentially in one single forwards pass through the data without the requirement of an additional backward sweep. An application to estimation of a non-linear time series model and to spectral estimation for time-varying autoregressions is described.

In this paper a method is introduced for approximating the likelihood for the unknown parameters of a state space model. The approximation conver99 to thetr3 likelihood as the simulation size goes to infinity. In addition, the approximating likelihood is continuous as a function of the unknownpar"5KM3 under rder gener' conditions. The appr2j h advocated is fast,rt,K' and avoids many of the pitfalls associated withcurK3 t techniques based upon imporK'95 sampling. We assess the per";'KM33 of the method by consider33 alinear state space model, compar'K therK;55" with the Kalman filter which deliver the tr' likelihood. We also apply the method to a non-Gaussian state space model, the Stochastic Volatility model, finding that the appr53 h is efficient and effective. Applications to continuous time finance models ar also consider2" A result is established which allows the likelihood to be estimated quickly and efficiently using the output from the general auxilary particle filter.