Variational learning for switching state-space models

DMCA

Variational learning for switching state-space models (1998)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Zoubin Ghahramani , Geoffrey E. Hinton

Venue:	Neural Computation
Citations:	172 - 5 self

BibTeX

@ARTICLE{Ghahramani98variationallearning,
    author = {Zoubin Ghahramani and Geoffrey E. Hinton},
    title = {Variational learning for switching state-space models},
    journal = {Neural Computation},
    year = {1998},
    volume = {12},
    pages = {963--996}
}

OpenURL

Abstract

We introduce a new statistical model for time series which iteratively segments data into regimes with approximately linear dynamics and learns the parameters of each of these linear regimes. This model combines and generalizes two of the most widely used stochastic time series models -- hidden Markov models and linear dynamical systems -- and is closely related to models that are widely used in the control and econometrics literatures. It can also be derived by extending the mixture of experts neural network (Jacobs et al., 1991) to its fully dynamical version, in which both expert and gating networks are recurrent. Inferring the posterior probabilities of the hidden states of this model is computationally intractable, and therefore the exact Expectation Maximization (EM) algorithm cannot be applied. However, we present a variational approximation that maximizes a lower bound on the log likelihood and makes use of both the forward-backward recursions for hidden Markov models and the Kalman lter recursions for linear dynamical systems. We tested the algorithm both on artificial data sets and on a natural data set of respiration force from a patient with sleep apnea. The results suggest that variational approximations are a viable method for inference and learning in switching state-space models.

Keyphrases

state-space model variational learning linear dynamical system variational approximation dynamical version artificial data set stochastic time series model viable method econometrics literature time series kalman lter recursion gating network log likelihood expert neural network exact expectation maximization linear dynamic sleep apnea new statistical model respiration force posterior probability natural data set linear regime forward-backward recursion hidden state hidden markov model model combine markov model

DMCA