Results 1  10
of
44
Dynamic Bayesian Networks: Representation, Inference and Learning
, 2002
"... Modelling sequential data is important in many areas of science and engineering. Hidden Markov models (HMMs) and Kalman filter models (KFMs) are popular for this because they are simple and flexible. For example, HMMs have been used for speech recognition and biosequence analysis, and KFMs have bee ..."
Abstract

Cited by 564 (3 self)
 Add to MetaCart
Modelling sequential data is important in many areas of science and engineering. Hidden Markov models (HMMs) and Kalman filter models (KFMs) are popular for this because they are simple and flexible. For example, HMMs have been used for speech recognition and biosequence analysis, and KFMs have been used for problems ranging from tracking planes and missiles to predicting the economy. However, HMMs
and KFMs are limited in their “expressive power”. Dynamic Bayesian Networks (DBNs) generalize HMMs by allowing the state space to be represented in factored form, instead of as a single discrete random variable. DBNs generalize KFMs by allowing arbitrary probability distributions, not just (unimodal) linearGaussian. In this thesis, I will discuss how to represent many different kinds of models as DBNs, how to perform exact and approximate inference in DBNs, and how to learn DBN models from sequential data.
In particular, the main novel technical contributions of this thesis are as follows: a way of representing
Hierarchical HMMs as DBNs, which enables inference to be done in O(T) time instead of O(T 3), where T is the length of the sequence; an exact smoothing algorithm that takes O(log T) space instead of O(T); a simple way of using the junction tree algorithm for online inference in DBNs; new complexity bounds on exact online inference in DBNs; a new deterministic approximate inference algorithm called factored frontier; an analysis of the relationship between the BK algorithm and loopy belief propagation; a way of
applying RaoBlackwellised particle filtering to DBNs in general, and the SLAM (simultaneous localization
and mapping) problem in particular; a way of extending the structural EM algorithm to DBNs; and a variety of different applications of DBNs. However, perhaps the main value of the thesis is its catholic presentation of the field of sequential data modelling.
On Tempo Tracking: Tempogram Representation and Kalman Filtering
, 2000
"... We formulate tempo tracking in a Bayesian framework where a tempo tracker is modeled as a stochastic dynamical system. The tempo is modeled as a hidden state variable of the system and is estimated by a Kalman filter. The Kalman filter operates on a Tempogram, a waveletlike multiscale expansion ..."
Abstract

Cited by 78 (8 self)
 Add to MetaCart
We formulate tempo tracking in a Bayesian framework where a tempo tracker is modeled as a stochastic dynamical system. The tempo is modeled as a hidden state variable of the system and is estimated by a Kalman filter. The Kalman filter operates on a Tempogram, a waveletlike multiscale expansion of a real performance. An important advantage of our approach is that it is possible to formulate both offline or realtime algorithms. The simulation results on a systematically collected set of MIDI piano performances of Yesterday and Michelle by the Beatles shows accurate tracking of approximately %90 of the beats.
Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter
, 2005
"... Effective neural motor prostheses require a method for decoding neural activity representing desired movement. In particular, the accurate reconstruction of a continuous motion signal is necessary for the control of devices such as computer cursors, robots, or a patient's own paralyzed limbs. For su ..."
Abstract

Cited by 48 (7 self)
 Add to MetaCart
Effective neural motor prostheses require a method for decoding neural activity representing desired movement. In particular, the accurate reconstruction of a continuous motion signal is necessary for the control of devices such as computer cursors, robots, or a patient's own paralyzed limbs. For such applications we developed a realtime system that uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons. In this study, we used recordings that were previously made in the arm area of primary motor cortex in awake behaving monkeys using a chronically implanted multielectrode microarray. Bayesian inference involves computing the posterior probability of the hand motion conditioned on a sequence of observed firing rates; this is formulated in terms of the product of a likelihood and a prior. The likelihood term models the probability of firing rates given a particular hand motion. We found that a linear Gaussian model could be used to approximate this likelihood and could be readily learned from a small amount of training data. The prior term defines a probabilistic model of hand kinematics and was also taken to be a linear Gaussian model. Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference when the likelihood and prior are linear and Gaussian. In offline experiments, the Kalmanfilter reconstructions of hand trajectory were more accurate than previously reported results. The resulting decoding algorithm provides a principled probabilistic model of motorcortical coding, decodes hand motion in real time, provides an estimate of uncertainty, and is straightfor3 ward to implement. Additionally the formulation unifies and extends previous models of neural coding while prov...
A Generative Model for Music Transcription
, 2005
"... In this paper we present a graphical model for polyphonic music transcription. Our model, formulated as a Dynamical Bayesian Network, embodies a transparent and computationally tractable approach to this acoustic analysis problem. An advantage of our approach is that it places emphasis on explicitl ..."
Abstract

Cited by 42 (14 self)
 Add to MetaCart
In this paper we present a graphical model for polyphonic music transcription. Our model, formulated as a Dynamical Bayesian Network, embodies a transparent and computationally tractable approach to this acoustic analysis problem. An advantage of our approach is that it places emphasis on explicitly modelling the sound generation procedure. It provides a clear framework in which both high level (cognitive) prior information on music structure can be coupled with low level (acoustic physical) information in a principled manner to perform the analysis. The model is a special case of the, generally intractable, switching Kalman filter model. Where possible, we derive, exact polynomial time inference procedures, and otherwise efficient approximations. We argue that our generative model based approach is computationally feasible for many music applications and is readily extensible to more general auditory scene analysis scenarios.
Observability and Identifiability of Jump Linear Systems
 In Proc. of IEEE Conference on Decision and Control
, 2002
"... We analyze the observability of the continuous and discrete states of a class of linear hybrid systems. We derive rank conditions that the structural parameters of the model must satisfy in order for filtering and smoothing algorithms to operate correctly. We also study the identifiability of the mo ..."
Abstract

Cited by 37 (8 self)
 Add to MetaCart
We analyze the observability of the continuous and discrete states of a class of linear hybrid systems. We derive rank conditions that the structural parameters of the model must satisfy in order for filtering and smoothing algorithms to operate correctly. We also study the identifiability of the model parameters by characterizing the set of models that produce the same output measurements. Finally, when the data are generated by a model in the class, we give conditions under which the true model can be identified.
An Algebraic Geometric Approach to the Identification of a Class of Linear Hybrid Systems
 In Proc. of IEEE Conference on Decision and Control
, 2003
"... We propose an algebraic geometric solution to the identification of a class of linear hybrid systems. We show that the identification of the model parameters can be decoupled from the inference of the hybrid state and the switching mechanism generating the transitions, hence we do not constraint the ..."
Abstract

Cited by 37 (11 self)
 Add to MetaCart
We propose an algebraic geometric solution to the identification of a class of linear hybrid systems. We show that the identification of the model parameters can be decoupled from the inference of the hybrid state and the switching mechanism generating the transitions, hence we do not constraint the switches to be separated by a minimum dwell time. The decoupling is obtained from the socalled hybrid decoupling constraint, which establishes a connection between linear hybrid system identification, polynomial factorization and hyperplane clustering. In essence, we represent the number of discrete states n as the degree of a homogeneous polynomial p and the model parameters as factors of p. We then show that one can estimate n from a rank constraint on the data, the coe#cients of p from a linear system, and the model parameters from the derivatives of p. The solution is closed form if and only if n 4. Once the model parameters have been identified, the estimation of the hybrid state becomes a simpler problem. Although our algorithm is designed for noiseless data, we also present simulation results with noisy data. 1
Factorial Switching Linear Dynamical Systems applied to Physiological Condition Monitoring
 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
"... Condition monitoring often involves the analysis of systems with hidden factors that switch between different modes of operation in some way. Given a sequence of observations, the task is to infer the filtering distribution of the switch setting at each time step. In this paper we present factorial ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
Condition monitoring often involves the analysis of systems with hidden factors that switch between different modes of operation in some way. Given a sequence of observations, the task is to infer the filtering distribution of the switch setting at each time step. In this paper we present factorial switching linear dynamical systems as a general framework for handling such problems. We show how domain knowledge and learning can be successfully combined in this framework, and introduce a new factor (the “Xfactor”) for dealing with unmodelled variation. We demonstrate the flexibility of this type of model by applying it to the problem of monitoring the condition of a premature baby receiving intensive care. The state of health of a baby cannot be observed directly, but different underlying factors are associated with particular patterns of physiological measurements and artifacts. We have explicit knowledge of common factors and use the Xfactor to model novel patterns which are clinically significant but have unknown cause. Experimental results are given which show the developed methods to be effective on typical intensive care unit monitoring data.
Identification of PWARX Hybrid Models with Unknown and Possibly Different Orders
 In Proceedings of IEEE American Control Conference
, 2004
"... We consider the problem of identifying the orders and the model parameters of PWARX hybrid models from noiseless input/output data. We cast the identification problem in an algebraic geometric framework in which the number of discrete states corresponds to the degree of a multivariate polynomial p a ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
We consider the problem of identifying the orders and the model parameters of PWARX hybrid models from noiseless input/output data. We cast the identification problem in an algebraic geometric framework in which the number of discrete states corresponds to the degree of a multivariate polynomial p and the orders and the model parameters are encoded on the factors of p. We derive a rank constraint on the input/output data from which one can estimate the coefficients of p. Given p, we show that one can estimate the orders and the parameters of each ARX model from the derivatives of p at a collection of regressors that minimize a certain objective function. Our solution does not require previous knowledge about the orders of the ARX models (only an upper bound is needed), nor does it constraint the orders to be equal. Also the switching mechanism can be arbitrary, hence the switches need not be separated by a minimum dwell time. We illustrate our approach with an algebraic example of a switching circuit and with simulation results in the presence of noisy data.
Unsupervised anomaly detection and diagnosis for liquid rocket engine propulsion
 In Proceedings of the IEEE Aerospace Conference, Big Sky
, 2007
"... Abstract — The results of a comprehensive array of unsupervised anomaly detection algorithms applied to Space Shuttle Main Engine (SSME) data are presented. Most of the algorithms are based upon variants of the wellknown unconditional Gaussian mixture model (GMM). One goal of the paper is to demons ..."
Abstract

Cited by 5 (4 self)
 Add to MetaCart
Abstract — The results of a comprehensive array of unsupervised anomaly detection algorithms applied to Space Shuttle Main Engine (SSME) data are presented. Most of the algorithms are based upon variants of the wellknown unconditional Gaussian mixture model (GMM). One goal of the paper is to demonstrate the maximum utility of these algorithms by the exhaustive development of a very simple GMM. Selected variants will provide us with the added benefit of diagnostic capability. Another algorithm that shares a common technique for detection with the GMM is presented, but instead uses a different modeling paradigm. The model provides a more rich description of the dynamics of the data, however the data requirements are quite modest. We will show that this very simple and straightforward method finds an event that characterizes
Fitting a Conditional Gaussian Distribution
, 1998
"... Introduction We consider the problem of nding the Maximum Likelihood (ML) estimates of the parameters of a conditional Gaussian node Y with continuous parent X and discrete parent Q, i.e., p(yjx; Q = i) = cj i j 1 2 exp 1 2 (y B i x) 0 1 i (y B i x) where c = (2) d=2 is a constant and j ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
Introduction We consider the problem of nding the Maximum Likelihood (ML) estimates of the parameters of a conditional Gaussian node Y with continuous parent X and discrete parent Q, i.e., p(yjx; Q = i) = cj i j 1 2 exp 1 2 (y B i x) 0 1 i (y B i x) where c = (2) d=2 is a constant and jyj = d. The j'th row of B i is the regression vector for the j component of y given that Q = i. To allo