Results 1  10
of
29
Messagepassing for graphstructured linear programs: Proximal methods and rounding schemes
, 2008
"... The problem of computing a maximum a posteriori (MAP) configuration is a central computational challenge associated with Markov random fields. A line of work has focused on “treebased ” linear programming (LP) relaxations for the MAP problem. This paper develops a family of superlinearly convergen ..."
Abstract

Cited by 62 (0 self)
 Add to MetaCart
(Show Context)
The problem of computing a maximum a posteriori (MAP) configuration is a central computational challenge associated with Markov random fields. A line of work has focused on “treebased ” linear programming (LP) relaxations for the MAP problem. This paper develops a family of superlinearly convergent algorithms for solving these LPs, based on proximal minimization schemes using Bregman divergences. As with standard messagepassing on graphs, the algorithms are distributed and exploit the underlying graphical structure, and so scale well to large problems. Our algorithms have a doubleloop character, with the outer loop corresponding to the proximal sequence, and an inner loop of cyclic Bregman divergences used to compute each proximal update. Different choices of the Bregman divergence lead to conceptually related but distinct LPsolving algorithms. We establish convergence guarantees for our algorithms, and illustrate their performance via some simulations. We also develop two classes of graphstructured rounding schemes, randomized and deterministic, for obtaining integral configurations from the LP solutions. Our deterministic rounding schemes use a “reparameterization ” property of our algorithms so that when the LP solution is integral, the MAP solution can be obtained even before the LPsolver converges to the optimum. We also propose a graphstructured randomized rounding scheme that applies to iterative LP solving algorithms in general. We analyze the performance of our rounding schemes, giving bounds on the number of iterations required, when the LP is integral, for the rounding schemes to obtain the MAP solution. These bounds are expressed in terms of the strength of the potential functions, and the energy gap, which measures how well the integral MAP solution is separated from other integral configurations. We also report simulations comparing these rounding schemes. 1
Quantifying Statistical Interdependence by Message Passing on Graphs  PART II: MultiDimensional Point Processes
, 2009
"... Stochastic event synchrony is a technique to quantify the similarity of pairs of signals. First, “events” are extracted from the two given time series. Next, one tries to align events from one time series with events from the other. The better the alignment, the more similar the two time series are ..."
Abstract

Cited by 20 (12 self)
 Add to MetaCart
Stochastic event synchrony is a technique to quantify the similarity of pairs of signals. First, “events” are extracted from the two given time series. Next, one tries to align events from one time series with events from the other. The better the alignment, the more similar the two time series are considered to be. In Part I, onedimensional events are considered, this paper (Paper II) concerns multidimensional events. Although the basic idea is similar, the extension to multidimensional point processes involves a significantly harder combinatorial problem, and therefore, it is nontrivial. Also in the multidimensional, the problem of jointly computing the pairwise alignment and SES parameters is cast as a statistical inference problem. This problem is solved by coordinate descent, more specifically, by alternating the following two steps: (i) one estimates the SES parameters from a given pairwise alignment; (ii) with the resulting estimates, one refines the pairwise alignment. The SES parameters are computed by maximum a posteriori (MAP) estimation (Step 1), in
Message Passing for Maximum Weight Independent Set
"... Abstract—In this paper, we investigate the use of messagepassing algorithms for the problem of finding the maxweight independent set (MWIS) in a graph. First, we study the performance of the classical loopy maxproduct belief propagation. We show that each fixedpoint estimate of max product can be ..."
Abstract

Cited by 18 (2 self)
 Add to MetaCart
(Show Context)
Abstract—In this paper, we investigate the use of messagepassing algorithms for the problem of finding the maxweight independent set (MWIS) in a graph. First, we study the performance of the classical loopy maxproduct belief propagation. We show that each fixedpoint estimate of max product can be mapped in a natural way to an extreme point of the linear programming (LP) polytope associated with the MWIS problem. However, this extreme point may not be the one that maximizes the value of node weights; the particular extreme point at final convergence depends on the initialization of max product. We then show that if max product is started from the natural initialization of uninformative messages, it always solves the correct LP, if it converges. This result is obtained via a direct analysis of the iterative algorithm, and cannot be obtained by looking only at fixed points. The tightness of the LP relaxation is thus necessary for maxproduct optimality, but it is not sufficient. Motivated by this observation, we show that a simple modification of max product becomes gradient descent on (a smoothed version of) the dual of the LP, and converges to the dual optimum. We also develop a messagepassing algorithm that recovers the primal MWIS solution from the output of the descent algorithm. We show that the MWIS estimate obtained using these two algorithms in conjunction is correct when the graph is bipartite and the MWIS is unique. Finally, we show that any problem of maximum a posteriori (MAP) estimation for probability distributions over finite domains can be reduced to an MWIS problem. We believe this reduction will yield new insights and algorithms for MAP estimation. Index Terms—Belief propagation, combinatorial optimization, distributed algorithms, independent set, iterative algorithms, linear programming (LP), optimization.
Belief Propagation for Mincost Network Flow: Convergence & Correctness
"... We formulate a Belief Propagation (BP) algorithm in the context of the capacitated minimumcost network flow problem (MCF). Unlike most of the instances of BP studied in the past, the messages of BP in the context of this problem are piecewiselinear functions. We prove that BP converges to the opti ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
(Show Context)
We formulate a Belief Propagation (BP) algorithm in the context of the capacitated minimumcost network flow problem (MCF). Unlike most of the instances of BP studied in the past, the messages of BP in the context of this problem are piecewiselinear functions. We prove that BP converges to the optimal solution in pseudopolynomial time, provided that the optimal solution is unique and the problem input is integral. Moreover, we present a simple modification of the BP algorithm which gives a fully polynomialtime randomized approximation scheme (FPRAS) for MCF. Thisisthe first instance where BP is proved to have fullypolynomial running time. 1
On the exactness of the cavity method for weighted bmatchings on arbitrary graphs and its relation to linear programs
 Journal of Statistical Mechanics: Theory and Experiment
"... iv ..."
(Show Context)
Fast bMatching via Sufficient Selection Belief Propagation
"... This article describes scalability enhancements to a previously established belief propagation algorithm that solves bipartite maximum weight bmatching. The previous algorithm required O(V  + E) space and O(V E) time, whereas we apply improvements to reduce the space to O(V ) and thetimet ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
This article describes scalability enhancements to a previously established belief propagation algorithm that solves bipartite maximum weight bmatching. The previous algorithm required O(V  + E) space and O(V E) time, whereas we apply improvements to reduce the space to O(V ) and thetimetoO(V  2.5) in the expected case (though worst case time is still O(V E)). The space improvement is most significant in cases where edge weights are determined by a function of node descriptors, such as a distance or kernel function. In practice, we demonstrate maximum weight bmatchings to be solvable on graphs with hundreds of millions of edges in only a few hours of compute time on a modern personal computer without parallelization, whereas neither the memory nor the time requirement of previously known algorithms would have allowed graphs of this scale. 1
Exactness of Belief Propagation for Some Graphical Models with Loops
, 801
"... Abstract. It is well known that an arbitrary graphical model of statistical inference defined on a tree, i.e. on a graph without loops, is solved exactly and efficiently by an iterative Belief Propagation (BP) algorithm convergent to unique minimum of the socalled Bethe free energy functional. For ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
(Show Context)
Abstract. It is well known that an arbitrary graphical model of statistical inference defined on a tree, i.e. on a graph without loops, is solved exactly and efficiently by an iterative Belief Propagation (BP) algorithm convergent to unique minimum of the socalled Bethe free energy functional. For a general graphical model on a loopy graph the functional may show multiple minima, the iterative BP algorithm may converge to one of the minima or may not converge at all, and the global minimum of the Bethe free energy functional is not guaranteed to correspond to the optimal MaximumLikelihood (ML) solution in the zerotemperature limit. However, there are exceptions to this general rule, discussed in [12] and [2] in two different contexts, where zerotemperature version of the BP algorithm finds ML solution for special models on graphs with loops. These two models share a key feature: their ML solutions can be found by an efficient Linear Programming (LP) algorithm with a TotallyUniModular (TUM) matrix of constraints. Generalizing the two models we consider a class of graphical models reducible in the zero temperature limit to LP with TUM constraints. Assuming that a gedanken algorithm, gBP, finding the global minimum of the Bethe free energy is available we show that in the limit of zero temperature gBP outputs the ML solution. Our consideration is based on equivalence established between gapless Linear Programming (LP) relaxation of the graphical model in the T → 0 limit and respective LP version of the BetheFree energy minimization.
Bipartite Graph Structures for Efficient Balancing of Heterogeneous Loads
, 2012
"... This paper considers large scale distributed content service platforms, such as peertopeer videoondemand systems. Such systems feature two basic resources, namely storage and bandwidth. Their efficiency critically depends on two factors: (i) content replication within servers, and (ii) how incom ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
(Show Context)
This paper considers large scale distributed content service platforms, such as peertopeer videoondemand systems. Such systems feature two basic resources, namely storage and bandwidth. Their efficiency critically depends on two factors: (i) content replication within servers, and (ii) how incoming service requests are matched to servers holding requested content. To inform the corresponding design choices, we make the following contributions. We first show that, for underloaded systems, socalled proportional content placement with a simple greedy strategy for matching requests to servers ensures full system efficiency provided storage size grows logarithmically with the system size. However, for constant storage size, this strategy undergoes a phase transition with severe loss of efficiency as system load approaches criticality. To better understand the role of the matching strategy in this performance degradation, we characterize the asymptotic system efficiency under an optimal matching policy. Our analysis shows that –in contrast to greedy matching– optimal matching incurs an inefficiency that is exponentially small in the server storage size, even at critical system loads. It further allows a characterization of content replication policies that minimize the inefficiency. These optimal policies, which differ markedly from proportional placement, have a simple structure which makes them implementable in practice. On the methodological side, our analysis of matching performance uses the theory of local weak limits of random graphs, and highlights a novel characterization of matching numbers in bipartite graphs, which may both be of independent interest.
3MessagePassing Algorithms for Sparse Network Alignment
"... Network alignment generalizes and unifies several approaches for forming a matching or alignment between the vertices of two graphs. We study a mathematical programming framework for network alignment problem and a sparse variation of it where only a small number of matches between the vertices of ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
Network alignment generalizes and unifies several approaches for forming a matching or alignment between the vertices of two graphs. We study a mathematical programming framework for network alignment problem and a sparse variation of it where only a small number of matches between the vertices of the two graphs are possible. We propose a new message passing algorithm that allows us to compute, very efficiently, approximate solutions to the sparse network alignment problems with graph sizes as large as hundreds of thousands of vertices. We also provide extensive simulations comparing our algorithms with two of the best solvers for network alignment problems on two synthetic matching problems, two bioinformatics problems, and three large ontology alignment problems including a multilingual problem with a known labeled alignment.
Messagepassing for Maximum Weight Independent Set
 In Advances in Neural Information Processing Systems (NIPS
, 2007
"... ar ..."
(Show Context)