Results 1  10
of
48
Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms
 IEEE Transactions on Information Theory
, 2005
"... Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems t ..."
Abstract

Cited by 519 (13 self)
 Add to MetaCart
(Show Context)
Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain regionbased free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a “valid ” or “maxentnormal ” approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the “Bethe method, ” the “junction graph method, ” the “cluster variation method, ” and the “region graph method.” Finally, we explain how to tell whether a regionbased approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.
Linear programming relaxations and belief propagation – an empirical study
 Jourmal of Machine Learning Research
, 2006
"... The problem of finding the most probable (MAP) configuration in graphical models comes up in a wide range of applications. In a general graphical model this problem is NP hard, but various approximate algorithms have been developed. Linear programming (LP) relaxations are a standard method in comput ..."
Abstract

Cited by 77 (4 self)
 Add to MetaCart
(Show Context)
The problem of finding the most probable (MAP) configuration in graphical models comes up in a wide range of applications. In a general graphical model this problem is NP hard, but various approximate algorithms have been developed. Linear programming (LP) relaxations are a standard method in computer science for approximating combinatorial problems and have been used for finding the most probable assignment in small graphical models. However, applying this powerful method to realworld problems is extremely challenging due to the large numbers of variables and constraints in the linear program. TreeReweighted Belief Propagation is a promising recent algorithm for solving LP relaxations, but little is known about its running time on large problems. In this paper we compare treereweighted belief propagation (TRBP) and powerful generalpurpose LP solvers (CPLEX) on relaxations of realworld graphical models from the fields of computer vision and computational biology. We find that TRBP almost always finds the solution significantly faster than all the solvers in CPLEX and more importantly, TRBP can be applied to large scale problems for which the solvers in CPLEX cannot be applied. Using TRBP we can find the MAP configurations in a matter of minutes for a large range of real world problems. 1.
Approximate inference and protein folding
 Advances in Neural Information Processing Systems
, 2002
"... Sidechain prediction is an important subtask in the proteinfolding problem. We show that finding a minimal energy sidechain configuration is equivalent to performing inference in an undirected graphical model. The graphical model is relatively sparse yet has many cycles. We used this equivalence ..."
Abstract

Cited by 70 (8 self)
 Add to MetaCart
(Show Context)
Sidechain prediction is an important subtask in the proteinfolding problem. We show that finding a minimal energy sidechain configuration is equivalent to performing inference in an undirected graphical model. The graphical model is relatively sparse yet has many cycles. We used this equivalence to assess the performance of approximate inference algorithms in a realworld setting. Specifically we compared belief propagation (BP), generalized BP (GBP) and naive mean field (MF). In cases where exact inference was possible, maxproduct BP always found the global minimum of the energy (except in few cases where it failed to converge), while other approximation algorithms of similar complexity did not. In the full protein data set, maxproduct BP always found a lower energy configuration than the other algorithms, including a widely used proteinfolding software (SCWRL). 1
Finding the m most probable configurations using loopy belief propagation
 In NIPS 16
, 2004
"... Loopy belief propagation (BP) has been successfully used in a number of difficult graphical models to find the most probable configuration of the hidden variables. In applications ranging from protein folding to image analysis one would like to find not just the best configuration but rather the top ..."
Abstract

Cited by 42 (2 self)
 Add to MetaCart
(Show Context)
Loopy belief propagation (BP) has been successfully used in a number of difficult graphical models to find the most probable configuration of the hidden variables. In applications ranging from protein folding to image analysis one would like to find not just the best configuration but rather the top M. While this problem has been solved using the junction tree formalism, in many real world problems the clique size in the junction tree is prohibitively large. In this work we address the problem of finding the M best configurations when exact inference is impossible. We start by developing a new exact inference algorithm for calculating the best configurations that uses only maxmarginals. For approximate inference, we replace the maxmarginals with the beliefs calculated using maxproduct BP and generalized BP. We show empirically that the algorithm can accurately and rapidly approximate the M best configurations in graphs with hundreds of variables. 1
Phylogenetic hidden Markov models
 IN STATISTICAL METHODS IN MOLECULAR EVOLUTION
, 2005
"... Phylogenetic hidden Markov models, or phyloHMMs, are probabilistic models that consider not only the way substitutions occur through evolutionary history at each site of a genome, but also the way this process changes from one site to the next. By treating molecular evolution as a combination of tw ..."
Abstract

Cited by 32 (6 self)
 Add to MetaCart
(Show Context)
Phylogenetic hidden Markov models, or phyloHMMs, are probabilistic models that consider not only the way substitutions occur through evolutionary history at each site of a genome, but also the way this process changes from one site to the next. By treating molecular evolution as a combination of two Markov processes—one that operates in the dimension of space (along a genome) and one that operates in the dimension of time (along the branches of a phylogenetic tree)—these models allow aspects of both sequence structure and sequence evolution to be captured. Moreover, as we will discuss, they permit key computations to be performed exactly and efficiently. PhyloHMMs allow evolutionary information to be brought to bear on a wide variety of problems of sequence “segmentation, ” such as gene prediction and the identification of conserved elements. PhyloHMMs were first proposed as a way of improving phylogenetic models that allow for variation among sites in the rate of substitution [8, 52]. Soon afterward, they were adapted for the problem of secondary structure
Dynamic Programming for Parsing and Estimation of Stochastic UnificationBased Grammars
 In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics
, 2002
"... Stochastic unificationbased grammars (SUBGs) define exponential distributions over the parses generated by a unificationbased grammar (UBG). Existing algorithms for parsing and estimation require the enumeration of all of the parses of a string in order to determine the most likely one, or in order ..."
Abstract

Cited by 24 (0 self)
 Add to MetaCart
Stochastic unificationbased grammars (SUBGs) define exponential distributions over the parses generated by a unificationbased grammar (UBG). Existing algorithms for parsing and estimation require the enumeration of all of the parses of a string in order to determine the most likely one, or in order to calculate the statistics needed to estimate a grammar from a training corpus. This paper describes a graphbased dynamic programming algorithm for calculating these statistics from the packed UBG parse representations of Maxwell and Kaplan (1995) which does not require enumerating all parses. Like many graphical algorithms, the dynamic programming algorithm's complexity is worstcase exponential, but is often polynomial.
An Introduction to Bayesian Network Theory and Usage
, 2000
"... . I present an introduction to some of the concepts within Bayesian networks to help a beginner become familiar with this eld's theory. Bayesian networks are a combination of two dierent mathematical areas: graph theory and probability theory. So, I rst give the basic denition of Bayesian netwo ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
. I present an introduction to some of the concepts within Bayesian networks to help a beginner become familiar with this eld's theory. Bayesian networks are a combination of two dierent mathematical areas: graph theory and probability theory. So, I rst give the basic denition of Bayesian networks. This is followed by an elaboration of the underlying graph theory that involves the arrangements of nodes and edges in a graph. Since Bayesian networks encode one's beliefs for a system of variables, I then proceed to discuss, in general, how to update these beliefs when one or more of the variables' values are no longer unknown (i.e., you have observed their values). Learning algorithms involve a combination of learning the probability distributions along with learning the network topology. I then conclude Part I by showing how Bayesian networks can be used in various domains, such as in the timeseries problem of automatic speech recognition. In Part II I then give in more detail some ...
Robust deployment of dynamic sensor networks for cooperative track detection
 IEEE Sensors
, 2009
"... Abstract—The problem of cooperative track detection by a dynamic sensor network arises in many applications, including security and surveillance, and tracking of endangered species. Several authors have recently shown that the qualityofservice of these networks can be statically optimized by placi ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
(Show Context)
Abstract—The problem of cooperative track detection by a dynamic sensor network arises in many applications, including security and surveillance, and tracking of endangered species. Several authors have recently shown that the qualityofservice of these networks can be statically optimized by placing the sensors in the region of interest (ROI) via mathematical programming. However, if the sensors are subject to external forcing, such as winds or currents, they may be rapidly displaced, and their qualityofservice may be significantly deteriorated over time. The novel approach presented in this paper consists of placing the sensors in the ROI based on their future displacement, which can be estimated from environmental forecasts and sensor dynamic models. The sensor network deployment is viewed as a new problem in dynamic computational geometry, in which the initial positions of a family of circles with timevarying radii and positions are to be optimized subject to sets of algebraic and differential equations. When these equations are nonlinear and timevarying, the optimization problem does not have an exact solution, or global optimum, but can be approximated as a finitedimensional nonlinear program by discretizing the qualityofservice and the dynamic models with respect to time. Then, a nearoptimal solution for the initial sensor positions is sought by means of sequential quadratic programming. The numerical results show that this approach can improve qualityofservice by up to a factor of five compared to existing techniques, and its performance is robust to propagated modeling and deployment errors. Index Terms—Cooperative, coverage, current, deployment, de
Bayes Blocks: An implementation of the variational Bayesian building blocks framework
 In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, UAI 2005
, 2005
"... A software library for constructing and learning probabilistic models is presented. The library offers a set of building blocks from which a large variety of static and dynamic models can be built. These include hierarchical models for variances of other variables and many nonlinear models. The unde ..."
Abstract

Cited by 7 (5 self)
 Add to MetaCart
A software library for constructing and learning probabilistic models is presented. The library offers a set of building blocks from which a large variety of static and dynamic models can be built. These include hierarchical models for variances of other variables and many nonlinear models. The underlying variational Bayesian machinery, providing for fast and robust estimation but being mathematically rather involved, is almost completely hidden from the user thus making it very easy to use the library. The building blocks include Gaussian, rectified Gaussian and mixtureofGaussians variables and computational nodes which can be combined rather freely. 1
Lifted Relational Kalman Filtering
"... Kalman Filtering is a computational tool with widespread applications in robotics, financial and weather forecasting, environmental engineering and defense. Given observation and state transition models, the Kalman Filter (KF) recursively estimates the state variables of a dynamic system. However, t ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
Kalman Filtering is a computational tool with widespread applications in robotics, financial and weather forecasting, environmental engineering and defense. Given observation and state transition models, the Kalman Filter (KF) recursively estimates the state variables of a dynamic system. However, the KF requires a cubic time matrix inversion operation at every timestep which prevents its application in domains with large numbers of state variables. We propose Relational Gaussian Models to represent and model dynamic systems with large numbers of variables efficiently. Furthermore, we devise an exact lifted Kalman Filtering algorithm which takes only linear time in the number of random variables at every timestep. We prove that our algorithm takes linear time in the number of state variables even when individual observations apply to each variable. To our knowledge, this is the first lifted (linear time) algorithm for filtering with continuous dynamic relational models. 1