## The Factored Frontier Algorithm for Approximate Inference in DBNs (0)

### Cached

### Download Links

- [http.cs.berkeley.edu]
- [www.cs.berkeley.edu]
- CiteULike
- DBLP

### Other Repositories/Bibliography

Venue: | In UAI |

Citations: | 49 - 4 self |

### BibTeX

@INPROCEEDINGS{Murphy_thefactored,

author = {Kevin Murphy and Yair Weiss},

title = {The Factored Frontier Algorithm for Approximate Inference in DBNs},

booktitle = {In UAI},

year = {},

pages = {378--385}

}

### Years of Citing Articles

### OpenURL

### Abstract

The Factored Frontier (FF) algorithm is a simple approximate inference algorithm for Dynamic Bayesian Networks (DBNs). It is very similar to the fully factorized version of the Boyen-Koller (BK) algorithm, but instead of doing an exact update at every step followed by marginalisation (projection), it always works with factored distributions. Hence it can be applied to models for which the exact update step is intractable. We show that FF is equivalent to (one iteration of) loopy belief propagation (LBP) on the original DBN, and that BK is equivalent (to one iteration of) LBP on a DBN where we cluster some of the nodes. We then show empirically that by iterating more than once, LBP can improve on the accuracy of both FF and BK. We compare these algorithms on two real-world DBNs: the first is a model of a water treatment plant, and the second is a coupled HMM, used to model freeway trac.

### Citations

7493 |
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- Pearl
- 1988
(Show Context)
Citation Context ...ction tree algorithm. In Section 4, we show how both FF and BK are related to loopy belief propagation (LBP) [12, 19, 17, 18, 5, 11], which is the method of applying Pearl's message passing algorithm =-=[13]-=- to a Bayes net even if it contains (undirected) cycles or loops. In Section 5, we experimentally compare all four algorithms | exact, FF, BK, and LBP | on a number of problems, and in Section 6, we c... |

4597 | A tutorial on hidden Markov models and selected applications in speech processing
- Rabiner
- 1989
(Show Context)
Citation Context ... Q possible values. The observed nodes can be discrete or continuous. The simplest way to perform exact inference in a DBN is to convert the model to an HMM and apply the forwards-backwards algorithm =-=[15]-=-. This takes O(TQ 2N ) time. By exploiting the conditional independencies within a slice, it is possible to reduce this tosTNQ N+F ) time, where F is the maximum fan-in of any node. Unfortunately, thi... |

670 |
Probabilistic Networks and Expert Systems
- Cowell, Dawid, et al.
- 1999
(Show Context)
Citation Context ...dden nodes, so there are no constraints on the order in which nodes are added to or removed from the frontier.) For regular 1 DBNs, the frontier algorithm is equivalent to the junction tree algorithm =-=[3, 9, 16] appl-=-ied to the \unrolled" DBN. In particular, the frontier sets correspond to the maximal cliques in the moralized, triangulated graph; in the junction tree, these cliques are connected together in a... |

515 | Factorial hidden markov models
- Ghahramani, Jordan
- 1997
(Show Context)
Citation Context ... such as greedy search [8], often perform as well as exhaustive search using branch and bound [21]. A special case of the frontier algorithm, applied to factorial HMMs, was published in Appendix B of =-=[6]-=-. (In an FHMM, there are no cross links between the hidden nodes, so there are no constraints on the order in which nodes are added to or removed from the frontier.) For regular 1 DBNs, the frontier a... |

495 | Loopy belief propagation for approximate inference: An empirical study
- Murphy, Weiss, et al.
- 1999
(Show Context)
Citation Context ...aph. In addition, FF is much simpler than BK, since it does not explicitely rely on the junction tree algorithm. In Section 4, we show how both FF and BK are related to loopy belief propagation (LBP) =-=[12, 19, 17, 18, 5, 11]-=-, which is the method of applying Pearl's message passing algorithm [13] to a Bayes net even if it contains (undirected) cycles or loops. In Section 5, we experimentally compare all four algorithms | ... |

410 | Generalized belief propagation
- Yedidia, Freeman, et al.
- 2000
(Show Context)
Citation Context ...ations can help dramatically. 4.1 A free energy for iterated BK The equivalence between BK and a single iteration of LBP on the clustered graph allows us to utilize the recent result of Yedidia et al =-=[20] to obtai-=-n a free energy for \iterated" BK. We dene the \iterated" BK algorithm as running LBP on the clustered graph using a FB schedule until convergence. Thesrst iteration of iterated BK is equiva... |

340 | Turbo decoding as an instance of Pearl’s belief propagation algorithm
- McEliece, MacKay, et al.
- 1998
(Show Context)
Citation Context ...aph. In addition, FF is much simpler than BK, since it does not explicitely rely on the junction tree algorithm. In Section 4, we show how both FF and BK are related to loopy belief propagation (LBP) =-=[12, 19, 17, 18, 5, 11]-=-, which is the method of applying Pearl's message passing algorithm [13] to a Bayes net even if it contains (undirected) cycles or loops. In Section 5, we experimentally compare all four algorithms | ... |

269 | Tractable inference for complex stochastic processes
- Boyen, Koller
- 1998
(Show Context)
Citation Context ... 9 10 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 36 35 Figure 2: A DBN designed to monitor a waste water treatement plant. This model is originally from [7], and was modied by [=-=2]-=- to include (discrete) evidence nodes. 2 Exact inference We start by reviewing the forwards-backwards (FB) algorithm [15] for HMMs, and then the frontier algorithm [21] for DBNs, since this will form ... |

234 | Correctness of belief propagation in Gaussian graphical models of arbitrary topology
- Weiss, Freeman
(Show Context)
Citation Context ...aph. In addition, FF is much simpler than BK, since it does not explicitely rely on the junction tree algorithm. In Section 4, we show how both FF and BK are related to loopy belief propagation (LBP) =-=[12, 19, 17, 18, 5, 11]-=-, which is the method of applying Pearl's message passing algorithm [13] to a Bayes net even if it contains (undirected) cycles or loops. In Section 5, we experimentally compare all four algorithms | ... |

193 | On the Optimality of Solutions of the Max-Product Belief Propagation Algorithm in Arbitrary Graphs
- Weiss, Freeman
- 2001
(Show Context)
Citation Context |

183 | Correctness of local probability propagation in graphical models with loops
- Weiss, Freeman
(Show Context)
Citation Context |

173 | Probabilistic Independence Networks for Hidden Markov Probability Models
- Smyth, Heckerman, et al.
- 1996
(Show Context)
Citation Context ...dden nodes, so there are no constraints on the order in which nodes are added to or removed from the frontier.) For regular 1 DBNs, the frontier algorithm is equivalent to the junction tree algorithm =-=[3, 9, 16] appl-=-ied to the \unrolled" DBN. In particular, the frontier sets correspond to the maximal cliques in the moralized, triangulated graph; in the junction tree, these cliques are connected together in a... |

39 | Approximate learning of dynamic models
- Boyen, Koller
- 1998
(Show Context)
Citation Context ...omputing the marginal on each cluster. The product of these marginals then gives the approximate posterior, ~ t . We can use a similar method for computing the backward messages in an ecient manner [=-=1]-=-. Boyen and Koller prove, roughly speaking, that if the error introduced by the projection step isn't much greater than the error incurred by using an approximate prior, both errors relative to the tr... |

13 |
A forward-backward algorithm for inference in Bayesian networks and an empirical comparison with HMMs
- Zweig
- 1996
(Show Context)
Citation Context ...lly from [7], and was modied by [2] to include (discrete) evidence nodes. 2 Exact inference We start by reviewing the forwards-backwards (FB) algorithm [15] for HMMs, and then the frontier algorithm [=-=21-=-] for DBNs, since this will form the basis of our generalisation. 2.1 The forwards backwards algorithm The basic idea of the FB algorithm is to compute i t def = P (X t = ijy 1:t ) in the forwards pa... |

12 |
A computational scheme for reasoning in dynamic probabilistic networks
- Kjaerul
- 1992
(Show Context)
Citation Context ...dden nodes, so there are no constraints on the order in which nodes are added to or removed from the frontier.) For regular 1 DBNs, the frontier algorithm is equivalent to the junction tree algorithm =-=[3, 9, 16] appl-=-ied to the \unrolled" DBN. In particular, the frontier sets correspond to the maximal cliques in the moralized, triangulated graph; in the junction tree, these cliques are connected together in a... |

11 |
On the fixed points of the max-product algorithm
- Freeman, Weiss
(Show Context)
Citation Context ..., for networks in which there is only a single loop [17], and for general networks but using the max-product (Viterbi) version instead of the sum-product (forwards-backwards) version of the algorithm =-=[4]-=-. The key assumption in LBP is that the messages coming into a node are independent. But this is exactly the same assumption that we make in the FF algorithm! Indeed, we can show that both algorithms ... |

11 | Turbo factor analysis
- Frey
- 1999
(Show Context)
Citation Context |

8 |
An expert system for control of waste water treatment — a pilot project
- Jensen, Kjaerulff, et al.
- 1989
(Show Context)
Citation Context ...s. 1 2 3 4 5 6 7 8 11 12 9 10 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 36 35 Figure 2: A DBN designed to monitor a waste water treatement plant. This model is originally from =-=[7-=-], and was modied by [2] to include (discrete) evidence nodes. 2 Exact inference We start by reviewing the forwards-backwards (FB) algorithm [15] for HMMs, and then the frontier algorithm [21] for DBN... |

6 |
Fusion and propogation with multiple observations in belief networks
- Peot, Shachter
- 1991
(Show Context)
Citation Context ...posteriors, whereas it requires T iterations of the decentralized protocol (each iteration computing 2TN messages in parallel) to reach the same result; hence the centralized algorithm is more ecient =-=[14]-=-. For loopy graphs, it is not clear which protocol is better; it depends on whether local or global information is more important for computing the posteriors. It is also easy to see that the fully-fa... |

4 | The belief in tap - Kabashima, Saad - 1999 |

3 |
Triangulation of graphs { algorithms giving small total state space
- Kjaerul
- 1990
(Show Context)
Citation Context ...ove nodes so as to minimize the sum of the frontier sizes is equivalent tosnding an optimal elimination ordering, which is known to be NP-hard. Nevertheless, heuristics methods, such as greedy search =-=[8]-=-, often perform as well as exhaustive search using branch and bound [21]. A special case of the frontier algorithm, applied to factorial HMMs, was published in Appendix B of [6]. (In an FHMM, there ar... |

1 |
Modeling freeway trac with coupled HMMs
- Kwon, Murphy
- 2000
(Show Context)
Citation Context ... of as asrst approximation to iterated BK. 5 Experimental results To compare the accuracy of LBP, BK, and FF, we used a CHMM model with 10 chains trained on some real freeway trac data using exact EM =-=[1-=-0]. We dene the L 1 error as t = P N i=1 P Q s=1 jP (X i t = 0 5 10 15 20 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 iterations L1 error 0 0.1 0.2 0.4 0.6 0.8 Figure 7: Results of applying LBP to the ... |