Results 1  10
of
40
Multiresolution markov models for signal and image processing
 Proceedings of the IEEE
, 2002
"... This paper reviews a significant component of the rich field of statistical multiresolution (MR) modeling and processing. These MR methods have found application and permeated the literature of a widely scattered set of disciplines, and one of our principal objectives is to present a single, coheren ..."
Abstract

Cited by 125 (17 self)
 Add to MetaCart
(Show Context)
This paper reviews a significant component of the rich field of statistical multiresolution (MR) modeling and processing. These MR methods have found application and permeated the literature of a widely scattered set of disciplines, and one of our principal objectives is to present a single, coherent picture of this framework. A second goal is to describe how this topic fits into the even larger field of MR methods and concepts–in particular making ties to topics such as wavelets and multigrid methods. A third is to provide several alternate viewpoints for this body of work, as the methods and concepts we describe intersect with a number of other fields. The principle focus of our presentation is the class of MR Markov processes defined on pyramidally organized trees. The attractiveness of these models stems from both the very efficient algorithms they admit and their expressive power and broad applicability. We show how a variety of methods and models relate to this framework including models for selfsimilar and 1/f processes. We also illustrate how these methods have been used in practice. We discuss the construction of MR models on trees and show how questions that arise in this context make contact with wavelets, state space modeling of time series, system and parameter identification, and hidden
TreeBased Reparameterization Framework for Analysis of Belief Propagation and Related Algorithms
, 2001
"... We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and ext ..."
Abstract

Cited by 101 (21 self)
 Add to MetaCart
(Show Context)
We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and extensions of belief propagation. Algorithms in this class can be formulated as a sequence of reparameterization updates, each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph (i.e., a tree). The ultimate goal is to obtain an alternative but equivalent factorization using functions that represent (exact or approximate) marginal distributions on cliques of the graph. Our framework highlights an important property of BP and the entire class of reparameterization algorithms: the distribution on the full graph is not changed. The perspective of treebased updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency. We develop interpretations of these results in terms of information geometry. The invariance of the distribution, in conjunction with the fixed point characterization, enables us to derive an exact relation between the exact marginals on an arbitrary graph with cycles, and the approximations provided by belief propagation, and more broadly, any algorithm that minimizes the Bethe free energy. We also develop bounds on this approximation error, which illuminate the conditions that govern their accuracy. Finally, we show how the reparameterization perspective extends naturally to more structured approximations (e.g., Kikuchi and variants [52, 37]) that operate over higher order cliques.
WalkSums and Belief Propagation in Gaussian Graphical Models
 Journal of Machine Learning Research
, 2006
"... We present a new framework based on walks in a graph for analysis and inference in Gaussian graphical models. The key idea is to decompose the correlation between each pair of variables as a sum over all walks between those variables in the graph. The weight of each walk is given by a product of edg ..."
Abstract

Cited by 66 (13 self)
 Add to MetaCart
(Show Context)
We present a new framework based on walks in a graph for analysis and inference in Gaussian graphical models. The key idea is to decompose the correlation between each pair of variables as a sum over all walks between those variables in the graph. The weight of each walk is given by a product of edgewise partial correlation coefficients. This representation holds for a large class of Gaussian graphical models which we call walksummable. We give a precise characterization of this class of models, and relate it to other classes including diagonally dominant, attractive, nonfrustrated, and pairwisenormalizable. We provide a walksum interpretation of Gaussian belief propagation in trees and of the approximate method of loopy belief propagation in graphs with cycles. The walksum perspective leads to a better understanding of Gaussian belief propagation and to stronger results for its convergence in loopy graphs.
LogDeterminant Relaxation for Approximate Inference in Discrete Markov Random Fields
, 2006
"... Graphical models are well suited to capture the complex and nonGaussian statistical dependencies that arise in many realworld signals. A fundamental problem common to any signal processing application of a graphical model is that of computing approximate marginal probabilities over subsets of nod ..."
Abstract

Cited by 27 (3 self)
 Add to MetaCart
Graphical models are well suited to capture the complex and nonGaussian statistical dependencies that arise in many realworld signals. A fundamental problem common to any signal processing application of a graphical model is that of computing approximate marginal probabilities over subsets of nodes. This paper proposes a novel method, applicable to discretevalued Markov random fields (MRFs) on arbitrary graphs, for approximately solving this marginalization problem. The foundation of our method is a reformulation of the marginalization problem as the solution of a lowdimensional convex optimization problem over the marginal polytope. Exactly solving this problem for general graphs is intractable; for binary Markov random fields, we describe how to relax it by using a Gaussian bound on the discrete entropy and a semidefinite outer bound on the marginal polytope. This combination leads to a logdeterminant maximization problem that can be solved efficiently by interior point methods, thereby providing approximations to the exact marginals. We show how a slightly weakened logdeterminant relaxation can be solved even more efficiently by a dual reformulation. When applied to denoising problems in a coupled mixtureofGaussian model defined on a binary MRF with cycles, we find that the performance of this logdeterminant relaxation is comparable or superior to the widely used sumproduct algorithm over a range of experimental conditions.
Robust Distributed Estimation in Sensor Networks using the Embedded Polygons Algorithm
 Proc. of the 3rd Intl. Symp. on Info. Proc. in Sensor Networks
, 2004
"... We propose a new iterative distributed algorithm for linear minimum meansquarederror (LMMSE) estimation in sensor networks whose measurements follow a Gaussian hidden Markov graphical model with cycles. The embedded polygons algorithm decomposes a loopy graphical model into a number of linked em ..."
Abstract

Cited by 22 (1 self)
 Add to MetaCart
(Show Context)
We propose a new iterative distributed algorithm for linear minimum meansquarederror (LMMSE) estimation in sensor networks whose measurements follow a Gaussian hidden Markov graphical model with cycles. The embedded polygons algorithm decomposes a loopy graphical model into a number of linked embedded polygons and then applies a parallel block GaussSeidel iteration comprising local LMMSE estimation on each polygon (involving inversion of a small matrix) followed by an information exchange between neighboring nodes and polygons. The algorithm is robust to temporary communication faults such as link failures and sleeping nodes and enjoys guaranteed convergence under mild conditions. A simulation study indicates that energy consumption for iterative estimation increases substantially as more links fail or nodes sleep. Thus, somewhat surprisingly, energy conservation strategies such as lowpowered transmission and aggressive sleep schedules could actually be counterproductive.
Extended Message Passing Algorithm for Inference in Loopy Gaussian Graphical Models
, 2002
"... We consider message passing for probabilistic inference in undirected Gaussian graphical models. We show that for singly connected graphs, message passing yields an algorithm that is equivalent to the application of Gaussian elimination to the solution of a particular system of equations. This relat ..."
Abstract

Cited by 21 (0 self)
 Add to MetaCart
We consider message passing for probabilistic inference in undirected Gaussian graphical models. We show that for singly connected graphs, message passing yields an algorithm that is equivalent to the application of Gaussian elimination to the solution of a particular system of equations. This relation provides a natural way of extending message passing to arbitrary graphs with loops by first studying the operations required by Gaussian elimination. We thus obtain a finite time convergent algorithm that solves the inference problem exactly and whose complexity grows gradually with the "distance" of the graph to a tree. This algorithm can be implemented in a distributed fashion at nodes through message passing, as for example in sensor networks.
Estimation in Gaussian Graphical Models Using Tractable Subgraphs: A WalkSum Analysis
, 2008
"... Graphical models provide a powerful formalism for statistical signal processing. Due to their sophisticated modeling capabilities, they have found applications in a variety of fields such as computer vision, image processing, and distributed sensor networks. In this paper, we present a general clas ..."
Abstract

Cited by 15 (11 self)
 Add to MetaCart
Graphical models provide a powerful formalism for statistical signal processing. Due to their sophisticated modeling capabilities, they have found applications in a variety of fields such as computer vision, image processing, and distributed sensor networks. In this paper, we present a general class of algorithms for estimation in Gaussian graphical models with arbitrary structure. These algorithms involve a sequence of inference problems on tractable subgraphs over subsets of variables. This framework includes parallel iterations such as embedded trees, serial iterations such as block Gauss–Seidel, and hybrid versions of these iterations. We also discuss a method that uses local memory at each node to overcome temporary communication failures that may arise in distributed sensor network applications. We analyze these algorithms based on the recently developed walksum interpretation of Gaussian inference. We describe the walks “computed ” by the algorithms using walksum diagrams, and show that for iterations based on a very large and flexible set of sequences of subgraphs, convergence is guaranteed in walksummable models. Consequently, we are free to choose spanning trees and subsets of variables adaptively at each iteration. This leads to efficient methods for optimizing the next iteration step to achieve maximum reduction in error. Simulation results demonstrate that these nonstationary algorithms provide a significant speedup in convergence over traditional onetree and twotree iterations.
Robust distributed estimation using the embedded subgraphs algorithm
 IEEE Trans. Signal Process
, 2006
"... Abstract—We propose a new iterative, distributed approach for linear minimum meansquareerror (LMMSE) estimation in graphical models with cycles. The embedded subgraphs algorithm (ESA) decomposes a loopy graphical model into a number of linked embedded subgraphs and applies the classical parallel b ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
(Show Context)
Abstract—We propose a new iterative, distributed approach for linear minimum meansquareerror (LMMSE) estimation in graphical models with cycles. The embedded subgraphs algorithm (ESA) decomposes a loopy graphical model into a number of linked embedded subgraphs and applies the classical parallel block Jacobi iteration comprising local LMMSE estimation in each subgraph (involving inversion of a small matrix) followed by an information exchange between neighboring nodes and subgraphs. Our primary application is sensor networks, where the model encodes the correlation structure of the sensor measurements, which are assumed to be Gaussian. The resulting LMMSE estimation problem involves a large matrix inverse, which must be solved innetwork with distributed computation and minimal intersensor communication. By invoking the theory of asynchronous iterations, we prove that ESA is robust to temporary communication faults such as failing links and sleeping nodes, and enjoys guaranteed convergence under relatively mild conditions. Simulation studies demonstrate that ESA compares favorably with other recently proposed algorithms for distributed estimation. Simulations also indicate that energy consumption for iterative estimation increases substantially as more links fail or nodes sleep. Thus, somewhat surprisingly, sensor network energy conservation strategies such as lowpowered transmission and aggressive sleep schedules could actually prove counterproductive. Our results can be replicated using MATLAB code from www.dsp.rice.edu/software. Index Terms—Asynchronous iterations, distributed estimation, graphical models, matrix splitting, sensor networks, Wiener filter. I.
Loopy SAM
"... Smoothing approaches to the Simultaneous Localization and Mapping (SLAM) problem in robotics are superior to the more common filtering approaches in being exact, better equipped to deal with nonlinearities, and computing the entire robot trajectory. However, while filtering algorithms that perform ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
Smoothing approaches to the Simultaneous Localization and Mapping (SLAM) problem in robotics are superior to the more common filtering approaches in being exact, better equipped to deal with nonlinearities, and computing the entire robot trajectory. However, while filtering algorithms that perform map updates in constant time exist, no analogous smoothing method is available. We aim to rectify this situation by presenting a smoothingbased solution to SLAM using Loopy Belief Propagation (LBP) that can perform the trajectory and map updates in constant time except when a loop is closed in the environment. The SLAM problem is represented as a Gaussian Markov Random Field (GMRF) over which LBP is performed. We prove that LBP, in this case, is equivalent to GaussSeidel relaxation of a linear system. The inability to compute marginal covariances efficiently in a smoothing algorithm has previously been a stumbling block to their widespread use. LBP enables the efficient recovery of the marginal covariances, albeit approximately, of landmarks and poses. While the final covariances are overconfident, the ones obtained from a spanning tree of the GMRF are conservative, making them useful for data association. Experiments in simulation and using real data are presented. 1
Covariance estimation in decomposable Gaussian graphical models
 IEEE TRANSACTIONS ON SIGNAL PROCESSING
, 2010
"... Graphical models are a framework for representing and exploiting prior conditional independence structures within distributions using graphs. In the Gaussian case, these models are directly related to the sparsity of the inverse covariance (concentration) matrix and allow for improved covariance es ..."
Abstract

Cited by 8 (5 self)
 Add to MetaCart
(Show Context)
Graphical models are a framework for representing and exploiting prior conditional independence structures within distributions using graphs. In the Gaussian case, these models are directly related to the sparsity of the inverse covariance (concentration) matrix and allow for improved covariance estimation with lower computational complexity. We consider concentration estimation with the meansquared error (MSE) as the objective, in a special type of model known as decomposable. This model includes, for example, the well known banded structure and other cases encountered in practice. Our first contribution is the derivation and analysis of the minimum variance unbiased estimator (MVUE) in decomposable graphical models. We provide a simple closed form solution to the MVUE and compare it with the classical maximum likelihood estimator (MLE) in terms of performance and complexity. Next, we extend the celebrated Stein’s unbiased risk estimate (SURE) to graphical models. Using SURE, we prove that the MSE of the MVUE is always smaller or equal to that of the biased MLE, and that the MVUE itself is dominated by other approaches. In addition, we propose the use of SURE as a constructive mechanism for deriving new covariance estimators. Similarly to the classical MLE, all of our proposed estimators have simple closed form solutions but result in a significant reduction in MSE.