## Statistical mechanics of combinatorial search (1994)

Venue: | In Proc. of the Workshop on Physics and Computation (PhysComp94 |

Citations: | 19 - 5 self |

### BibTeX

@INPROCEEDINGS{Hogg94statisticalmechanics,

author = {Tad Hogg},

title = {Statistical mechanics of combinatorial search},

booktitle = {In Proc. of the Workshop on Physics and Computation (PhysComp94},

year = {1994},

pages = {196--202},

publisher = {IEEE Press}

}

### Years of Citing Articles

### OpenURL

### Abstract

The statistical mechanics of combinatorial search problems is described using the example of the well-known NP-complete graph coloring problem. We focus on a recently identified phase transition from under- to overconstrained problems, near which are concentrated many hard to solve search problems. Thus, a readily computed measure of problem structure predicts the difficulty of solving the problem, on average. However, this prediction is associated with a large variance and depends on the somewhat arbitrary choice of the problem ensemble. Thus these results are of limited direct use for individual instances. To help address this limitation, additional parameters, describing problem structure as well as heuristic effectiveness, are introduced. This also highlights the distinction between the statistical mechanics of combinatorial search problems, with their exponentially large search spaces, and physical systems, whose interactions are often governed by a simple euclidean metric. Chapter 1

### Citations

10886 |
Computers and Intractability: A Guide to the Theory of NP-Completeness
- Garey, Johnson
- 1979
(Show Context)
Citation Context ... by a simple euclidean metric.Chapter 1 Introduction Combinatorial search is among the hardest of common computational problems: the solution time can grow exponentially with the size of the problem =-=[14]-=-. Examples arise in scheduling, circuit layout design and automated proofs, to name a few areas. For instance, in scheduling classes, we attempt to find an assignment of rooms, times, students and fac... |

3517 | Optimization by simulated annealing
- KIRKPATRICK, GELATT, et al.
- 1983
(Show Context)
Citation Context ...me number of conflicts will not help. This then requires either restarting the search or allowing occasional increases in the number of conflicts as is done with the simulated annealing search method =-=[30]-=-. From this discussion, we can see that the repair strategy is likely to be useful when there are few local minima and small plateaus so the heuristic guidance of locally reducing the number of remain... |

1791 | Random Graphs
- Bollobás
- 1985
(Show Context)
Citation Context ...f the graph is a constraint, requiring that the two variables linked by that edge have different assigned values. For simplicity, we primarily consider the ensemble of problems given by random graphs =-=[3]-=-, i.e., taking each graph with a specified number of nodes and edges to be equally likely. Our experiments primarily used a depth-first backtracking search based on the Brelaz heuristic [27]. This heu... |

1481 |
The sciences of the artificial
- Simon
- 1996
(Show Context)
Citation Context ...nts with some locality also. This is because such artifacts are usually nearly decomposable in having strong interactions among a few tightly coupled components and weaker interactions among the rest =-=[52]-=-. Unlike the Euclidean structures of physical systems, the modular construction of many artifacts produces instead hierarchical spaces with an ultrametric topology, i.e., in which interaction strength... |

686 |
Heuristics: intelligent search strategies for computer problem solving
- Pearl
- 1984
(Show Context)
Citation Context ...s to consider grows rapidly, the actual number that must be searched before finding a solution can vary greatly depending on the order in which the possibilities are examined. In practice, heuristics =-=[43]-=- are often used to select the next combination, or state, to consider during a search among the possibilities. Typically, during a search the heuristic evaluates a small number of potential changes to... |

680 | A New Method for Solving Hard Satisfiability Problems
- Selman, Levesque, et al.
- 1992
(Show Context)
Citation Context ...ent conflicts with a constraint and set its new value to the one that minimizes the number of conflicts [38], or select the variable to change that reduces the remaining conflicts as much as possible =-=[50]-=-. This repair method can get stuck in “plateau” states, in which no additional single assignment change reduces the number of conflicts. A common way to address this possibility is to allow for change... |

577 | Where the really hard problems are
- Cheeseman, Kanefsky, et al.
- 1991
(Show Context)
Citation Context ...or of combinatorial search problems [19]. These include phase transitions due to pruning in heuristic search [26], models of associative memory [29, 51], automatic planning [5], optimization problems =-=[6, 64]-=- and various cases of pattern matching [58, 23, 17, 33]. Here we will focus on the recent studies that relate search difficulty to the structure of constraint satisfaction problems [6, 9, 15, 32, 39, ... |

509 |
Optimization by simulated annealing: an experimental evaluation; part 1, graph partitioning
- Johnson, Aragon, et al.
- 1989
(Show Context)
Citation Context ...en assignment of colors is a solution, additional solutions can be obtained simply by permuting the colors. As a wellknown NP-complete problem [14], graph coloring has received considerable attention =-=[38, 27, 50]-=-. That is, graph coloring is among the class of hardest combinatorial search problems and so provides a particularly challenging example. Specifically, this means that a general, rapid search method f... |

496 |
Graphs and Hypergraphs
- Berge
- 1973
(Show Context)
Citation Context ...onentially fast. Whether such rapid methods exist in general is an important open question in computer science theory. Many important artificial intelligence problems, such as planning and scheduling =-=[63, 2]-=-, can be mapped onto the graph coloring problem. To see how this can be done, consider the problem of scheduling n classes in b time slots. We can represent the classes as nodes in a graph. Each edge ... |

448 |
An assumption-based TMS
- Kleer
- 1986
(Show Context)
Citation Context ...sion of the problem as a guide to finding the detailed solution. There is also the possibility to avoid unnecessary repetition of parts of the search by recording those combinations found not to work =-=[10]-=-. However, with all these heuristics it is important to realize the extra use of computational resources (both time and memory) may counteract the savings in search steps. Furthermore, in some cases t... |

345 | Hybrid algorithms for the constraint satisfaction problem
- Prosser
- 1993
(Show Context)
Citation Context ... present the specific example of graph coloring used to illustrate the behaviors discussed in the remainder of the paper. 3.1 Search Methods While there are a large variety of search methods for CSPs =-=[48]-=-, they can be conveniently viewed as one of the following strategies (which also readily generalize to other combinatorial search problems). First and simplest is to examine all possibilities in turn ... |

283 |
and easy distributions of sat problems
- Hard
- 1992
(Show Context)
Citation Context ...on problems [6, 64] and various cases of pattern matching [58, 23, 17, 33]. Here we will focus on the recent studies that relate search difficulty to the structure of constraint satisfaction problems =-=[6, 9, 15, 32, 39, 47, 59, 60]-=-, with information on these results also available via the World Wide Web [21]. Specifically, for large problems, a few parameters characterizing the structure of the search problem determine the diff... |

253 |
Constraint satisfaction
- Mackworth
- 1992
(Show Context)
Citation Context ...nt progress has been made toward this goal through the use of a statistical mechanics approach. In this paper, we summarize these results for one type of combinatorial search, constraint satisfaction =-=[35]-=-, using the particular problem of graph coloring as an example. Specifically, the next section describes the general reasons to expect useful insights from an approach based on statistical mechanics. ... |

242 |
Forward reasoning and dependency-directed backtracking in a system for computer-aided circuit analysis
- Stallman, Sussman
- 1977
(Show Context)
Citation Context ... aborted immediately rather than continuing with a fruitless search. This decomposition of the problem into independent pieces can also be viewed as a limited form of dependency-directed backtracking =-=[55]-=-, and also suggests that these clustered graphs may benefit from more sophisticated decomposition methods [13]. cost 8 10 6 10 4 10 2 10 1 2 3 4 5 connectivity cost 6 10 5 10 4 10 3 10 2 10 1 10 1 2 3... |

215 | Domain-independent extensions to GSAT: Solving large structured satisfiability problems
- Selman, Kautz
- 1993
(Show Context)
Citation Context ...lution after a prespecified number of steps. Usually this new initial condition is chosen randomly, but it can also be based to some extent on the remaining conflicts from the previous search attempt =-=[49]-=-. Note that, unlike the previous search strategies, this method will never terminate if there is no solution for a problem, i.e., it is not a systematic or complete search method. Thus another bound i... |

201 | Experimental results on the cross-over point in satisfiability problems
- Crawford, Auton
- 1993
(Show Context)
Citation Context ...on problems [6, 64] and various cases of pattern matching [58, 23, 17, 33]. Here we will focus on the recent studies that relate search difficulty to the structure of constraint satisfaction problems =-=[6, 9, 15, 32, 39, 47, 59, 60]-=-, with information on these results also available via the World Wide Web [21]. Specifically, for large problems, a few parameters characterizing the structure of the search problem determine the diff... |

150 | Critical behavior in the satisfiability of random boolean expressions
- Kirkpatrick, Selman
- 1994
(Show Context)
Citation Context ...es, the peak in search cost becomes sharper and the fraction of soluble cases drops more abruptly. While this behavior with increasing size has primarily been studied empirically, finite-size scaling =-=[31]-=- and approximate “mean-field” theories [62] have also been applied. However, a definitive theory of this transition, similar to the rigorous thresholds known to exist for other properties of random gr... |

113 |
An empirical study of phase transitions in binary constraint satisfaction problems
- Prosser
- 1996
(Show Context)
Citation Context ...on problems [6, 64] and various cases of pattern matching [58, 23, 17, 33]. Here we will focus on the recent studies that relate search difficulty to the structure of constraint satisfaction problems =-=[6, 9, 15, 32, 39, 47, 59, 60]-=-, with information on these results also available via the World Wide Web [21]. Specifically, for large problems, a few parameters characterizing the structure of the search problem determine the diff... |

88 | The hardest constraint problems: A double phase transition
- Williams, Hogg
- 1994
(Show Context)
Citation Context ...d problems and explains why hard cases are likely to be found in the transition region. In more quantitative terms, it predicts an observed transition from linear to exponentially growing search cost =-=[25]-=- and determines roughly the parameter values at which these transitions take place. In addition, the peak in the search cost also appears [60] for methods, such as heuristic repair [38] or simulated a... |

78 | Easy problems are sometimes hard
- Gent, Walsh
- 1994
(Show Context)
Citation Context |

72 | Exploiting the deep structure of constraint problems
- Williams, Hogg
- 1994
(Show Context)
Citation Context ...constitute the set of minimized nogoods for the problem. Often in describing a set of constraints, it is sufficient to specify the size and number of such nogoods rather than their detailed structure =-=[62]-=-. In this context, the search problem is then to find a solution, i.e., an assignment of a value to each variable such that all the constraints are satisfied, or else show that no such assignment exis... |

68 |
Graph Evolution: An Introduction to the Theory of Random Graphs, John-Wiley and
- Palmer
- 1985
(Show Context)
Citation Context ...sets Sr = ∑ |El1 ∩ . . . ∩ Elr | (7.1) where the sum is over all r-subsets of {1, . . . , e}, and we define S0 = bn , the total number of possible colorings. Then the principle of inclusion-exclusion =-=[42]-=- gives the number of colorings that are not in any of the Ek, i.e., the number of solutions: e∑ N = (−1) i Si (7.2) i=0 Each edge eliminates bn−1 colorings, so S1 = ∑e l=1 bn−1 = ebn−1 . The colorings... |

66 | A parallel graph coloring heuristic
- Jones, Plassmann
- 1993
(Show Context)
Citation Context ...ese cases, there are a number of plausible structural classes as well as some cases from particular applications such as class scheduling [34] or graph colorings that arise in some numerical programs =-=[28]-=-. In this section we examine several such graph ensembles. This shows first, that the transition behavior and existence of an extended distribution is robust in that it appears in these ensembles as w... |

58 |
Combinatorial Algorithms for Computers and Calculators
- Nijenhuis, Wilf
- 1978
(Show Context)
Citation Context ...e To gain an understanding of how the number of edges determines the search cost, we first consider a small example. Specifically, consider a series of connected graphs constructed from a random tree =-=[41]-=- with 10 nodes (and 9 edges). Additional edges were added randomly to give a series of related graphs. Each graph was then searched using a simple, nonheuristic backtrack search in which nodes were co... |

46 | Evidence for a Satisfiability Threshold for Random 3CNF Formulas
- Larrabee, Tsuji
- 1992
(Show Context)
Citation Context |

45 |
Domain filtering can degrade intelligent backtracking
- Prosser
- 1993
(Show Context)
Citation Context ...ve the problem with relatively little search. Note that this is not a simple choice in that there can be cases in which preprocessing actually interferes with the more sophisticated search strategies =-=[46]-=-. Thus, in general, we can expect that within a class of problems, each search method will have its own set of hard cases, some of which 32will be relatively easy for other methods while others will ... |

42 |
Solving the Really Hard Problems with Cooperative Search
- Hogg, Williams
- 1993
(Show Context)
Citation Context ...n individual search methods become ineffective that the overhead 17associated with more sophisticated methods may be worth incurring. This has led to studies of cooperative search for graph coloring =-=[24]-=-. More generally, because this phenomenon relates likely search difficulty to easily computed structural parameters, it can be used as a heuristic when selecting the next subproblem to work on [8]. 18... |

41 | Using deep structure to locate hard problems
- Williams, Hogg
- 1992
(Show Context)
Citation Context |

35 | The Hazards of Fancy Backtracking
- Baker
- 1994
(Show Context)
Citation Context ...lize the extra use of computational resources (both time and memory) may counteract the savings in search steps. Furthermore, in some cases they can result in an increase in the required search steps =-=[45, 1]-=- when the nature of the problem strongly violates the assumptions of the heuristics. 73.2 Graph Coloring As a specific example of a constraint satisfaction problem, in this paper we use the well-stud... |

31 |
Phase transitions in artificial intelligence systems
- Huberman, Hogg
- 1987
(Show Context)
Citation Context ...ical mechanics has been applied in a variety of situations to understand the generic behavior of combinatorial search problems [19]. These include phase transitions due to pruning in heuristic search =-=[26]-=-, models of associative memory [29, 51], automatic planning [5], optimization problems [6, 64] and various cases of pattern matching [58, 23, 17, 33]. Here we will focus on the recent studies that rel... |

29 |
The solution of some random NP-hard problems in polynomial expected time
- Dyer, Frieze
- 1989
(Show Context)
Citation Context ...ility of n = O( ) 1 ln n ) This suggests that dense graphs are easy to color, or determine that no coloring exists, which is also the case when restricting attention to graphs that do have a coloring =-=[57, 12]-=-. To summarize, this theory shows how the variation in search cost as edges are added is due to a competition between increased pruning and a reduction in the number of solutions. Additional edges red... |

29 | The phase transition in constraint satisfaction problems: A closer look at the mushy region
- Smith
- 1993
(Show Context)
Citation Context ...ning the nature of the partial solutions, without requiring a detailed analysis of the individual search methods. 154.2 Theory This behavior can be quantified with the following approximate argument =-=[60, 54]-=-. Let b be the number of available colors. A given edge in the graph eliminates b of the b2 possible ways to color the nodes it connects. So a choice of colors will satisfy the constraint with probabi... |

28 |
Models of Network Structure
- Burt
- 1980
(Show Context)
Citation Context ...y few components of the system [36, 37, 20]. Finally, many systems have a mixture of intentional design as well as unplanned incremental evolution as in large software systems and human organizations =-=[44, 18, 4]-=-. These observations suggest that many real-world CSPs will involve more clustering among the constraints than typically found in the random ensemble. It is thus of interest to see whether the phase t... |

25 |
Phase transitions in sequence matches and nucleic acid structure
- Waterman, Gordon, et al.
- 1987
(Show Context)
Citation Context ...e include phase transitions due to pruning in heuristic search [26], models of associative memory [29, 51], automatic planning [5], optimization problems [6, 64] and various cases of pattern matching =-=[58, 23, 17, 33]-=-. Here we will focus on the recent studies that relate search difficulty to the structure of constraint satisfaction problems [6, 9, 15, 32, 39, 47, 59, 60], with information on these results also ava... |

23 | The hardest random SAT problems, in
- Gent, Walsh
- 1994
(Show Context)
Citation Context ...ard precisely because the first few search choices often produce a hard, insoluble subproblem which must then be searched completely before backtracking far enough back to correct these early choices =-=[16, 53]-=-. 5 1 2 3 4 7 6 Figure 8.1: A difficult graph structure for the Brelaz heuristic to 3–color. This example has 11 nodes. The numbers show the order in which the Brelaz heuristic colors the nodes (excep... |

22 |
Heuristic Sampling: A Method for Predicting the Performance of Tree Searching Programs
- Chen
- 1992
(Show Context)
Citation Context ...ductive nodes, can be simply quantified as the probability p that an unproductive partial coloring will be recognized as such in the search. In practice, such a parameter can be estimated by sampling =-=[7]-=-. A partial coloring of size k is considered only if the smaller colorings preceding it in the search order did not eliminate it, i.e., with probability (1 − p) k−1 if the pruning ability is independe... |

22 | Experiments with parallel graph coloring heuristics and applications of graph coloring
- Lewandowski, Condon
(Show Context)
Citation Context ...e detailed nature of problems that might generally arise in these cases, there are a number of plausible structural classes as well as some cases from particular applications such as class scheduling =-=[34]-=- or graph colorings that arise in some numerical programs [28]. In this section we examine several such graph ensembles. This shows first, that the transition behavior and existence of an extended dis... |

19 |
Observation of phase transitions in spreading activation networks
- Shrager, Hogg, et al.
- 1987
(Show Context)
Citation Context ...a variety of situations to understand the generic behavior of combinatorial search problems [19]. These include phase transitions due to pruning in heuristic search [26], models of associative memory =-=[29, 51]-=-, automatic planning [5], optimization problems [6, 64] and various cases of pattern matching [58, 23, 17, 33]. Here we will focus on the recent studies that relate search difficulty to the structure ... |

17 |
Using inferred disjunctive constraints to decompose constraint satisfaction problems
- Freuder, Hubbe
- 1993
(Show Context)
Citation Context ...dependent pieces can also be viewed as a limited form of dependency-directed backtracking [55], and also suggests that these clustered graphs may benefit from more sophisticated decomposition methods =-=[13]-=-. cost 8 10 6 10 4 10 2 10 1 2 3 4 5 connectivity cost 6 10 5 10 4 10 3 10 2 10 1 10 1 2 3 4 5 connectivity Figure 6.3: Cost percentiles for clustered 100–node graphs using component-based search. The... |

17 |
Extending Deep Structure
- Williams, Hogg
- 1993
(Show Context)
Citation Context |

12 |
On the density of solutions in equilibrium points for the queens problem
- Morris
- 1992
(Show Context)
Citation Context ...plateau states and local minima within the search space is thus an important characteristic for the success of this method, and has been investigated in the context of some simple constraint problems =-=[40, 62]-=-. Further improvements in all these search methods are often possible by identifying and removing trivial aspects of the problem before searching, checking for whether the problem can be decomposed in... |

12 | Epsilon-transformation: Exploiting phase transitions to solve combinatorial optimization problems - initial results
- Zhang, Pemberton
- 1994
(Show Context)
Citation Context ...sical systems. These identify situations in which major gains are possible from small improvements in the local heuristic evaluations. This work is also 3beginning to lead to improved search methods =-=[8, 65]-=- as well as a refined understanding of the structure of the search problems [22]. 4Chapter 3 Constraint Satisfaction Problems In this paper we focus on a particular kind of combinatorial search: the ... |

11 |
The stability of ecosystems
- Hogg, Huberman, et al.
- 1989
(Show Context)
Citation Context ...ne often observes a clustering of interactions, and this is also suggested theoretically since for stability reasons interactions in large systems must involve relatively few components of the system =-=[36, 37, 20]-=-. Finally, many systems have a mixture of intentional design as well as unplanned incremental evolution as in large software systems and human organizations [44, 18, 4]. These observations suggest tha... |

11 |
a Large Complex System be Stable
- Will
- 1972
(Show Context)
Citation Context ...ne often observes a clustering of interactions, and this is also suggested theoretically since for stability reasons interactions in large systems must involve relatively few components of the system =-=[36, 37, 20]-=-. Finally, many systems have a mixture of intentional design as well as unplanned incremental evolution as in large software systems and human organizations [44, 18, 4]. These observations suggest tha... |

10 |
On the generation of random graphs with given properties and known distributions
- Tinhofer
- 1979
(Show Context)
Citation Context ... type of problem that remains after decomposition into independent parts. It is conceptually straightforward to generate graphs uniformly from the set of connected graphs with a given number of edges =-=[56]-=-, but this is computationally demanding. A simpler approach, as used here, is to first construct a random tree [41] on the n nodes of the graph, which uses n − 1 edges, and then add the remaining edge... |

10 |
An average-case analysis of branch-and-bound with applications: Summary of results
- Zhang, Korf
- 1992
(Show Context)
Citation Context ...or of combinatorial search problems [19]. These include phase transitions due to pruning in heuristic search [26], models of associative memory [29, 51], automatic planning [5], optimization problems =-=[6, 64]-=- and various cases of pattern matching [58, 23, 17, 33]. Here we will focus on the recent studies that relate search difficulty to the structure of constraint satisfaction problems [6, 9, 15, 32, 39, ... |

8 |
An average case analysis of planning
- Bylander
- 1993
(Show Context)
Citation Context ...derstand the generic behavior of combinatorial search problems [19]. These include phase transitions due to pruning in heuristic search [26], models of associative memory [29, 51], automatic planning =-=[5]-=-, optimization problems [6, 64] and various cases of pattern matching [58, 23, 17, 33]. Here we will focus on the recent studies that relate search difficulty to the structure of constraint satisfacti... |

8 |
search of exceptionally difficult constraint satisfaction problems
- In
- 1994
(Show Context)
Citation Context ...o the variance in the search cost. Specifically, with a fixed number of solutions, additional clustering means that poor initial choices can lead to searching very many unproductive partial colorings =-=[53]-=-. Thus even if the number of solutions were accurately known, there could still be significant variation in search cost due to differences in how tightly clustered the solutions are. It thus remains a... |

8 | A Technique for Coloring a Graph Applicable to Large-Scale Optimization Problems - Wood - 1969 |

7 |
Exploiting problem structure in genetic algorithms
- Clearwater, Hogg
- 1994
(Show Context)
Citation Context ...sical systems. These identify situations in which major gains are possible from small improvements in the local heuristic evaluations. This work is also 3beginning to lead to improved search methods =-=[8, 65]-=- as well as a refined understanding of the structure of the search problems [22]. 4Chapter 3 Constraint Satisfaction Problems In this paper we focus on a particular kind of combinatorial search: the ... |