Determining whether a propositional theory is satisfiable is a prototypical example of an NP-complete problem. Further, a large number of problems that occur in knowledge-representation, learning, planning, and other ares of AI are essentially satisfiability problems. This paper reports on the most extensive set of experiments to date on the location and nature of the cross-over point in satisfiability problems. These experiments generally confirm previous results with two notable exceptions. First, we have found that neither of the functions previously proposed accurately models the location of the cross-over point. Second, we have found no evidence of any hard problems in the underconstrained region. In fact the hardest problems found in the underconstrained region were many times easier than the easiest unsatisfiable problems found in the neighborhood of the cross-over point. We offer explanations for these apparent contradictions of previous results. This work has been supported ...

We propose a definition of `constrainedness' that unifies two of the most common but informal uses of the term. These are that branching heuristics in search algorithms often try to make the most "constrained" choice, and that hard search problems tend to be "critically constrained". Our definition of constrainedness generalizes a number of parameters used to study phase transition behaviour in a wide variety of problem domains. As well as predicting the location of phase transitions in solubility, constrainedness provides insight into why problems at phase transitions tend to be hard to solve. Such problems are on a constrainedness "knife-edge", and we must search deep into the problem before they look more or less soluble. Heuristics that try to get off this knife-edge as quickly as possible by, for example, minimizing the constrainedness are often very effective. We show that heuristics from a wide variety of problem domains can be seen as minimizing the constrainedness (or proxies ...

We present a detailed experimental investigation of the easy-hard-easy phase transition for randomly generated instances of satisfiability problems. Problems in the hard part of the phase transition have been extensively used for benchmarking satisfiability algorithms. This study demonstrates that problem classes and regions of the phase transition previously thought to be easy can sometimes be orders of magnitude more difficult than the worst problems in problem classes and regions of the phase transition considered hard. These difficult problems are either hard unsatisfiable problems or are satisfiable problems which give a hard unsatisfiable subproblem following a wrong split. Whilst these hard unsatisfiable problems may have short proofs, these appear to be difficult to find, and other proofs are long and hard. This paper is a revised version of Research Paper 642, available from the department of Artificial Intelligence, Edinburgh. This version is to appear in the journal Artific...

We wish to bring to the attention of the OR community the phenomenon of phase transitions in randomly generated problems. These are of considerable practical use for benchmarking algorithms. They also offer insight into problem hardness and algorithm performance. Whilst phase transition experiments are frequently performed by AI researchers, such experiments do not appear to be in common use in the OR community. To illustrate the value of such experiments, we examine a typical OR problem, the traveling salesman problem. We report in detail many features of the phase transition in this problem, and show how some of these features are also seen in real problems. Acknowledgements The second author is supported by a HCM Postdoctoral Fellowship. We thank Iain Buchanan for comments on a draft of this paper, and Alan Bundy, and the members of the Mathematical Reasoning Group in Edinburgh for their constructive comments and many CPU cycles donated to these and other experiments from SERC grant GR/H/23610. We also thank the MRG group at Trento and the Department of Computer Science at the University of Strathclyde for additional CPU cycles. Finally, we thank Robert Craig for providing us with his code. 1

: We describe a detailed experimental investigation of the phase transition for several different classes of randomly generated satisfiability problems. We observe a remarkable consistency of features in the phase transition despite the presence in some of the problem classes of clauses of mixed lengths. For instance, each of the problem classes considered has a sharp transition from satisfiable to unsatisfiable problems at a critical value. In addition, there is a common easyhard -easy pattern in the difficulty of the problems, with the hardest problems being associated with the phase transition. However, the difficulty of problems of mixed clause lengths is much more variable than that of fixed clause length. Indeed, whilst the median difficulty of random problems of mixed clause lengths can be orders of magnitude easier than that of equivalently sized problems of fixed clause length, the hardest problems of mixed clause lengths can be orders of magnitude harder than the hardest equi...

The Constraint Satisfaction Problem is a type of combinatorial search problem of much interest in Artificial Intelligence and Operations Research. The simplest algorithm for solving such a problem is chronological backtracking, but this method suffers from a malady known as "thrashing," in which essentially the same subproblems end up being solved repeatedly. Intelligent backtracking algorithms, such as backjumping and dependency-directed backtracking, were designed to address this difficulty, but the exact utility and range of applicability of these techniques have not been fully explored. This dissertation describes an experimental and theoretical investigation into the power of these intelligent backtracking algorithms. We compare the empirical performance of several such algorithms on a range of problem distributions. We show that the more sophisticated algorithms are especially useful on those problems with a small number of constraints that happen to be difficult for chronologica...

Many types of problem exhibit a phase transition as a problem parameter is varied, from a region where most problems are easy and soluble to a region where most problems are easy but insoluble. In the intervening phase transition region, the median problem difficulty is greatest. However, occasional exceptionally hard problems (ehps) can be found in the easy and soluble region: these problems can be much harder than any problem occurring in the phase transition. We show that in binary constraint satisfaction problems ehps are much more likely to occur when the constraints are sparse than in dense problems. In ehps, the search algorithm encounters a large insoluble subproblem at an early stage; the exceptional difficulty is due to the cost of searching the subproblem to prove insolubility. This cost can be dramatically reduced by using conflict-directed backjumping (CBJ) rather than a chronological backtracker. However, when used with forward checking and the fail-first heuristic, it is...

Many problem ensembles exhibit a phase transition that is associated with a large peak in the average cost of solving the problem instances. However, this peak is not necessarily due to a lack of solutions: indeed the average number of solutions is typically exponentially large. Here, we study this situation within the context of the satisfiability transition in Random 3SAT. We find that a significant subclass of instances emerges as we cross the phase transition. These instances are characterized by having about 85--95% of their variables occurring in unary prime implicates (UPIs), with their remaining variables being subject to few constraints. In such instances the models are not randomly distributed but all lie in a cluster that is exponentially large, but still admits a simple description. Studying the effect of UPIs on the local search algorithm Wsat shows that these "single-cluster" instances are harder to solve, and we relate their appearance at the phase transition to the peak...

The local search algorithm WSat is one of the most successful algorithms for solving the satisfiability (SAT) problem. It is notably e#ective at solving hard Random 3-SAT instances near the so-called `satisfiability threshold', but still shows a peak in search cost near the threshold and large variations in cost over di#erent instances. We make a number of significant contributions to the analysis of WSat on high-cost random instances, using the recently-introduced concept of the backbone of a SAT instance. The backbone is the set of literals which are entailed by an instance. We find that the number of solutions predicts the cost well for small-backbone instances but is much less relevant for the large-backbone instances which appear near the threshold and dominate in the overconstrained region. We show a very strong correlation between search cost and the Hamming distance to the nearest solution early in WSat's search. This pattern leads us to introduce a measure of the ba...

Abstract. While CNF propositional satisfiability (SAT) is a sub-class of the more general constraint satisfaction problem (CSP), conventional wisdom has it that some well-known CSP look-back techniques-- including backjumping and learning-- are of little use for SAT. We enhance the Tableau SAT algorithm of Crawford and Auton with look-back techniques and evaluate its performance on problems specifically designed to challenge it. The Random 3-SAT problem space has commonly been used to benchmark SAT algorithms because consistently difficult instances can be found near a region known as the phase transition. We modify Random 3-SAT in two ways which make instances even harder. First, we evaluate problems with structural regularities and find that CSP look-back techniques offer little advantage. Second, we evaluate problems in which a hard unsatisfiable instance of medium size is embedded in a larger instance, and we find the look-back enhancements to be indispensable. Without them, most instances are “exceptionally hard ”-orders of magnitude harder than typical Random 3-SAT instances with the same surface characteristics.