Results 1  10
of
152
A New Class of Upper Bounds on the Log Partition Function
 In Uncertainty in Artificial Intelligence
, 2002
"... Bounds on the log partition function are important in a variety of contexts, including approximate inference, model fitting, decision theory, and large deviations analysis [11, 5, 4]. We introduce a new class of upper bounds on the log partition function, based on convex combinations of distribution ..."
Abstract

Cited by 220 (33 self)
 Add to MetaCart
Bounds on the log partition function are important in a variety of contexts, including approximate inference, model fitting, decision theory, and large deviations analysis [11, 5, 4]. We introduce a new class of upper bounds on the log partition function, based on convex combinations of distributions in the exponential domain, that is applicable to an arbitrary undirected graphical model. In the special case of convex combinations of treestructured distributions, we obtain a family of variational problems, similar to the Bethe free energy, but distinguished by the following desirable properties: (i) they are convex, and have a unique global minimum; and (ii) the global minimum gives an upper bound on the log partition function. The global minimum is defined by stationary conditions very similar to those defining xed points of belief propagation (BP) or treebased reparameterization [see 13, 14]. As with BP fixed points, the elements of the minimizing argument can be used as approximations to the marginals of the original model. The analysis described here can be extended to structures of higher treewidth (e.g., hypertrees), thereby making connections with more advanced approximations (e.g., Kikuchi and variants [15, 10]).
MAP estimation via agreement on trees: Messagepassing and linear programming
, 2002
"... We develop and analyze methods for computing provably optimal maximum a posteriori (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles. By decomposing the original distribution into a convex combination of treestructured distributions, we obtain an upper bound ..."
Abstract

Cited by 198 (10 self)
 Add to MetaCart
(Show Context)
We develop and analyze methods for computing provably optimal maximum a posteriori (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles. By decomposing the original distribution into a convex combination of treestructured distributions, we obtain an upper bound on the optimal value of the original problem (i.e., the log probability of the MAP assignment) in terms of the combined optimal values of the tree problems. We prove that this upper bound is tight if and only if all the tree distributions share an optimal configuration in common. An important implication is that any such shared configuration must also be a MAP configuration for the original distribution. Next we develop two approaches to attempting to obtain tight upper bounds: (a) a treerelaxed linear program (LP), which is derived from the Lagrangian dual of the upper bounds; and (b) a treereweighted maxproduct messagepassing algorithm that is related to but distinct from the maxproduct algorithm. In this way, we establish a connection between a certain LP relaxation of the modefinding problem, and a reweighted form of the maxproduct (minsum) messagepassing algorithm.
MAP estimation via agreement on (hyper)trees: Messagepassing and linear programming approaches
 IEEE Transactions on Information Theory
, 2002
"... We develop an approach for computing provably exact maximum a posteriori (MAP) configurations for a subclass of problems on graphs with cycles. By decomposing the original problem into a convex combination of treestructured problems, we obtain an upper bound on the optimal value of the original ..."
Abstract

Cited by 144 (10 self)
 Add to MetaCart
(Show Context)
We develop an approach for computing provably exact maximum a posteriori (MAP) configurations for a subclass of problems on graphs with cycles. By decomposing the original problem into a convex combination of treestructured problems, we obtain an upper bound on the optimal value of the original problem (i.e., the log probability of the MAP assignment) in terms of the combined optimal values of the tree problems. We prove that this upper bound is met with equality if and only if the tree problems share an optimal configuration in common. An important implication is that any such shared configuration must also be a MAP configuration for the original problem. Next we present and analyze two methods for attempting to obtain tight upper bounds: (a) a treereweighted messagepassing algorithm that is related to but distinct from the maxproduct (minsum) algorithm; and (b) a treerelaxed linear program (LP), which is derived from the Lagrangian dual of the upper bounds. Finally, we discuss the conditions that govern when the relaxation is tight, in which case the MAP configuration can be obtained. The analysis described here generalizes naturally to convex combinations of hypertreestructured distributions.
Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal
 In Proc. of ACM Symposium on Theory of computing (STOC
, 2007
"... ABSTRACT In the MINIMUM BOUNDED DEGREE SPANNING TREE problem,we are given an undirected graph with a degree upper bound Bv oneach vertex v, and the task is to find a spanning tree of minimumcost which satisfies all the degree bounds. Let OPT be the costof an optimal solution to this problem. In this ..."
Abstract

Cited by 64 (8 self)
 Add to MetaCart
ABSTRACT In the MINIMUM BOUNDED DEGREE SPANNING TREE problem,we are given an undirected graph with a degree upper bound Bv oneach vertex v, and the task is to find a spanning tree of minimumcost which satisfies all the degree bounds. Let OPT be the costof an optimal solution to this problem. In this paper, we present a polynomial time algorithm which returns a spanning tree T ofcost at most OPT and dT (v) ^ Bv + 1 for all v, where dT (v)denotes the degree of v in T. This generalizes a result of Furerand Raghavachari [8] to weighted graphs, and settles a 15yearold conjecture of Goemans [10] affirmatively. The algorithm generalizes when each vertex v has a degree lower bound Av and a degreeupper bound Bv, and returns a spanning tree with cost at most OPTand Av \Gamma 1 ^ dT (v) ^ Bv + 1 for all v. This is essentially thebest possible. The main technique used is an extension of the iterative rounding method introduced by Jain [12] for the design ofapproximation algorithms.
The convex hull of two core capacitated network design problems
 MATHEMATICAL PROGRAMMING
, 1993
"... The network loading problem (NLP) is a specialized capacitated network design problem in which prescribed pointtopoint demand between various pairs of nodes of a network must be met by installing (loading) a capacitated facility. We can load any number of units of the facility on each of the arcs ..."
Abstract

Cited by 59 (1 self)
 Add to MetaCart
The network loading problem (NLP) is a specialized capacitated network design problem in which prescribed pointtopoint demand between various pairs of nodes of a network must be met by installing (loading) a capacitated facility. We can load any number of units of the facility on each of the arcs at a specified arc dependent cost. The problem is to determine the number of facilities to be loaded on the arcs that will satisfy the given demand at minimum cost. This paper studies two core subproblems of the NLP. The first problem, motivated by a Lagrangian relaxation approach for solving the problem, considers a multiple commodity, single arc capacitated network design problem. The second problem is a three node network; this specialized network arises in larger networks if we aggregate nodes. In both cases, we develop families of facets and completely characterize the convex hull of feasible solutions to the integer programming formulation of the problems. These results in turn strengthen the formulation of the NLP.
Algorithm theories and design tactics
 Science of Computer Programming
, 1990
"... Algorithm theories represent the structure common to a class of algorithms, such as divideandconquer or backtrack. An algorithm theory for a class A provides the basis for design tactics – specialized methods for designing Aalgorithms from formal problem specifications. We illustrate this approac ..."
Abstract

Cited by 50 (16 self)
 Add to MetaCart
(Show Context)
Algorithm theories represent the structure common to a class of algorithms, such as divideandconquer or backtrack. An algorithm theory for a class A provides the basis for design tactics – specialized methods for designing Aalgorithms from formal problem specifications. We illustrate this approach with recent work on the theory of global search algorithms and briefly mention several others. Several design tactics have been implemented in the KIDS/CYPRESS system and have been used to semiautomatically derive many algorithms. 1.
Solving small TSPs with constraints
 PROCEEDINGS OF THE 14TH INTERNATIONAL CONFERENCE ON LOGIC PROGRAMMING
, 1997
"... This paper presents a set of techniques that makes constraint programming a technique of choice for solving small (up to 30 nodes) traveling salesman problems. These techniques include a propagation scheme to avoid intermediate cycles (a global constraint), a branching scheme and a redundant constra ..."
Abstract

Cited by 49 (0 self)
 Add to MetaCart
This paper presents a set of techniques that makes constraint programming a technique of choice for solving small (up to 30 nodes) traveling salesman problems. These techniques include a propagation scheme to avoid intermediate cycles (a global constraint), a branching scheme and a redundant constraint that can be used as a bounding method. The resulting improvement is that we can solve problems twice larger than those solved previously with constraint programming tools. We evaluate the use of Lagrangean Relaxation to narrow the gap between constraint programming and other Operations Research techniques and we show that improved constraint propagation has now a place in the array of techniques that should be used to solve a traveling salesman problem.
Approximating Submodular Functions Everywhere
"... Submodular functions are a key concept in combinatorial optimization. Algorithms that involve submodular functions usually assume that they are given by a (value) oracle. Many interesting problems involving submodular functions can be solved using only polynomially many queries to the oracle, e.g., ..."
Abstract

Cited by 46 (4 self)
 Add to MetaCart
Submodular functions are a key concept in combinatorial optimization. Algorithms that involve submodular functions usually assume that they are given by a (value) oracle. Many interesting problems involving submodular functions can be solved using only polynomially many queries to the oracle, e.g., exact minimization or approximate maximization. In this paper, we consider the problem of approximating a nonnegative, monotone, submodular function f on a ground set of size n everywhere, after only poly(n) oracle queries. Our main result is a deterministic algorithm that makes poly(n) oracle queries and derives a function ˆ f such that, for every set S, ˆ f(S) approximates f(S) within a factor α(n), where α(n) = √ n + 1 for rank functions of matroids and α(n) = O ( √ n log n) for general monotone submodular functions. Our result is based on approximately finding a maximum volume inscribed ellipsoid in a symmetrized polymatroid, and the analysis involves various properties of submodular functions and polymatroids. Our algorithm is tight up to logarithmic factors. Indeed, we show that no algorithm can achieve a factor better than Ω ( √ n / log n), even for rank functions of a matroid.
Minimum Bounded Degree Spanning Trees
, 2006
"... We consider the minimum cost spanning tree problem under the restriction that all degrees must be at most a given value k. We show that we can efficiently find a spanning tree of maximum degree at most k + 2 whose cost is at most the cost of the optimum spanning tree of maximum degree at most k. Thi ..."
Abstract

Cited by 46 (0 self)
 Add to MetaCart
We consider the minimum cost spanning tree problem under the restriction that all degrees must be at most a given value k. We show that we can efficiently find a spanning tree of maximum degree at most k + 2 whose cost is at most the cost of the optimum spanning tree of maximum degree at most k. This is almost best possible. The approach uses a sequence of simple algebraic, polyhedral and combinatorial arguments. It illustrates many techniques and ideas in combinatorial optimization as it involves polyhedral characterizations, uncrossing, matroid intersection, and graph orientations (or packing of spanning trees). The result generalizes to the setting where every vertex has both upper and lower bounds and gives then a spanning tree which violates the bounds by at most two units and whose cost is at most the cost of the optimum tree. It also gives a better understanding of the subtour relaxation for both the symmetric and asymmetric traveling salesman problems. The generalization to ledgeconnected subgraphs is briefly discussed.