Abstract

We present a tree-based reparameterization framework that provides a new conceptual view of a large class of iterative algorithms for computing approximate marginals in graphs with cycles. It includes belief propagation (BP), which can be reformulated as a very local form of reparameterization. More generally, we consider algorithms that perform exact computations over spanning trees of the full graph. On the practical side, we nd that such tree reparameterization (TRP) algorithms typically converge more quickly than BP with lower cost per iteration; moreover, TRP often converges on problems for which BP fails. The reparameterization perspective also provides theoretical insight into approximate estimation, including a new probabilistic characterization of xed points; and an invariance intrinsic to TRP/BP. These two properties in conjunction enable us to analyze and bound the approximation error that arises in applying these techniques. Our results also have natural extensions to approximations (e.g., Kikuchi) that involve clustering nodes. 1

Citations