## Generating Hard Satisfiability Problems (1996)

### Cached

### Download Links

- [ftp.research.att.com]
- [www.cs.utoronto.ca]
- DBLP

### Other Repositories/Bibliography

Venue: | Artificial Intelligence |

Citations: | 98 - 2 self |

### BibTeX

@ARTICLE{Selman96generatinghard,

author = {Bart Selman and David Mitchell and Hector Levesque},

title = {Generating Hard Satisfiability Problems},

journal = {Artificial Intelligence},

year = {1996},

volume = {81},

pages = {17--29}

}

### Years of Citing Articles

### OpenURL

### Abstract

We report results from large-scale experiments in satisfiability testing. As has been observed by others, testing the satisfiability of random formulas often appears surprisingly easy. Here we show that by using the right distribution of instances, and appropriate parameter values, it is possible to generate random formulas that are hard, that is, for which satisfiability testing is quite difficult. Our results provide a benchmark for the evaluation of satisfiability-testing procedures. In Artificial Intelligence, 81 (19996) 17--29. 1 Introduction Many computational tasks of interest to AI, to the extent that they can be precisely characterized at all, can be shown to be NP-hard in their most general form. However, there is fundamental disagreement, at least within the AI community, about the implications of this. It is claimed on the one hand that since the performance of algorithms designed to solve NP-hard tasks degrades rapidly with small increases in input size, something ...

### Citations

10921 | Computers and Intractability: A Guide to the Theory of NP-Completeness - Garey, Johnson |

1788 | Random Graphs
- Bollobás
- 1985
(Show Context)
Citation Context ...ertainly will not have any larger cliques. In other words, the probability of a random graph having a clique of size k is represented by a threshold function with the threshold at 2logn (Matula 1972; =-=Bollobas 1985-=-). It appears much harder to show formally that there exists a threshold phenomenon for SAT, even though our experimental results certainly do suggest so. A recent breakthrough was the establishment o... |

1075 |
A computing procedure for quantification theory
- Davis, Putnam
- 1960
(Show Context)
Citation Context ... it is not useful in the evaluation of satisfiability-testing procedures. We then briefly review related work, and summarize our results. 2 The Davis-Putnam Procedure The Davis-Putnam (DP) procedure (=-=Davis and Putnam 1960) is sketched in Fig-=-ure 1. It takes as input a set of clauses S over a set of variables V; and returns either "satisfiable" or "unsatisfiable." (A clause is a disjunction of literals. A set of clauses... |

774 | The Complexity of Theorem-Proving Procedures - COOK - 1971 |

680 | A New Method for Solving Hard Satisfiability Problems
- Selman, Levesque, et al.
- 1992
(Show Context)
Citation Context ...almost any reasonable method, it is the hard cases ultimately that separate the winners from the losers. Thus, our data is presented as challenging test material for developers of SAT procedures (see =-=Selman et al., 1992-=-, for example). The SAT procedure we used for our tests is the Davis-Putnam procedure, which we describe below. We believe this was a good choice for two reasons: First, it is basically a variant of r... |

529 |
A machine program for theorem-proving
- Davis, Logemann, et al.
- 1962
(Show Context)
Citation Context ...sed methods for propositional satisfiability testing is the Davis-Putnam procedure [11]. Our procedure, which we refer to as DP, is the splitting variant of the Davis-Putnam procedure as described in =-=[12]-=-, but without the pure literal rule. (The pure literal rule is: if a literal p occurs in a formula, but its negation does not, then p can be immediately assigned the value true.) DP is sketched in Fig... |

264 | The intractability of resolution - Haken - 1985 |

218 | and easy distributions of SAT problems - Mitchell, Selman, et al. - 1992 |

214 |
Many hard examples for resolution
- Chvátal, Szemerédi
- 1988
(Show Context)
Citation Context ... ratio of variables to clauses of less than 1.7 almost all formulas are satisfiable. Moreover, for a ratio of 5.2 or higher, it has been shown that almost all formulas are unsatisfiable (Dubois 1991; =-=Chvatal and Szemeredi 1988-=-). Note that these ratios are still quite far removed from our experimentally determined threshold at 4.3. We now return to the discussion of our experimental results. To get some further insight into... |

201 | Experimental results on the cross-over point in satisfiability problems
- Crawford, Auton
- 1993
(Show Context)
Citation Context ...any variables in order to satisfy all clauses, so it generates a relatively long branches for satisfying assignments. 2 Recent, more detailed experiments confirm our observations about the 50% point (=-=Crawford and Auton 1993-=-; Larrabee and Tsuji 1993). 8 0 5000 10000 15000 20000 25000 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 Number of DP calls Ratio of clauses-to-variables Satisfiable 3 3 3 3 3 3 3 3 3 3 3 3 33 333 Unsatisfiable + +... |

165 |
Mick gets some (the odds are on his side
- Chvátal, Reed
- 1992
(Show Context)
Citation Context ...mally that there exists a threshold phenomenon for SAT, even though our experimental results certainly do suggest so. A recent breakthrough was the establishment of a threshold for 2SAT (Goerdt 1991; =-=Chvatal and Reed 1992-=-; de la Vega 1992). The threshold lies at a ratio of 1, i.e., for a n variable formula, if one generates less then n binary clauses the formula is almost certainly satisfiable, whereas when generating... |

104 |
A threshold for unsatisfiability
- Goerdt
- 1992
(Show Context)
Citation Context ...r to show formally that there exists a threshold phenomenon for SAT, even though our experimental results certainly do suggest so. A recent breakthrough was the establishment of a threshold for 2SAT (=-=Goerdt 1991-=-; Chvatal and Reed 1992; de la Vega 1992). The threshold lies at a ratio of 1, i.e., for a n variable formula, if one generates less then n binary clauses the formula is almost certainly satisfiable, ... |

97 | Tail bounds for occupancy and the satisfiability threshold conjecture. Random Structures and Algorithms
- Kamath, Motwani, et al.
- 1995
(Show Context)
Citation Context ...the probability that an instance of Random 3-SAT is satisfiable approaches 0 whenever the ratio of clauses to variables is less than 3.003 [15], and approaches 1 when this ratio is greater than 4.758 =-=[23]-=-. As can be seen, our empirical results are in agreement with, though much more fine-grained than, the best theoretical bounds. To get some further insight into the search performed by DP, we now cons... |

93 | Probabilistic analysis of a generalization of the unit clause literal selection heuristic for the k-satisfiability problem
- Chao, Franco
- 1990
(Show Context)
Citation Context ...omly generated assignments would, with probability approaching 1, find a satisfying assignment in a constant number of guesses. Further analytic results for the constant-density model can be found in =-=[5,28,31]-=-. Franco and Paull [14] also investigated the performance of DP on random formulas with a fixed clause length of 3, and suggested that it might be more useful for generating random instances, an hypot... |

89 | On the satisfiability and maximum satisfiability of random 3-CNF formulas - Broder, Frieze, et al. - 1993 |

78 | Report on a sat competition - Buro, Buning - 1992 |

74 | A rearrangement search strategy for determining propositional satisfiability
- Zabih
- 1988
(Show Context)
Citation Context ...tive performance on satisfiable and unsatisfiable instances will shift the location of peak hardness. While the performance of DP can be improved by using clever variable selection heuristics, (e.g., =-=[32,4]-=-), it seems unlikely that such heuristics will qualitatively alter the easy-hard-easy pattern. The formulas in the hard area appear to be the most challenging for the strategies we have tested, and we... |

64 | Algorithms for Testing the Satisfiability of Propositional Formulae - Gallo, Urbani - 1989 |

51 |
Resolution vs. cutting plane solution of inference problems: Some computational experience
- Hooker
- 1988
(Show Context)
Citation Context ...n a way that produced almost no hard cases. Nevertheless, papers continue to appear purporting to empirically demonstrate 2 the efficacy of some new procedure, but using just this distribution (e.g., =-=Hooker, 1988-=-; Kamath et al. 1990), or presenting data suggesting that very large satisfiability problems --- with thousands of propositional variables --- can be solved. How are we to evaluate these empirical res... |

46 |
Branch-and-cut solution of inference problems in propositional logic
- Hooker, Fedjki
- 1990
(Show Context)
Citation Context ...m P-SAT with the same numbers for random 3-SAT. In this case, we have set the probability P to give an average clause length of 3. Although we see a slight easy-hard-easy pattern (previously noted by =-=Hooker and Fedjki, 1989-=-), the hard area is not nearly as pronounced as that for random 3-SAT formulas of similar size, and in absolute terms the random P-SAT formulas are much easier. Note that we are using a logarithmic sc... |

46 | Evidence for a Satisfiability Threshold for Random 3CNF Formulas
- Larrabee, Tsuji
- 1992
(Show Context)
Citation Context ... satisfy all clauses, so it generates a relatively long branches for satisfying assignments. 2 Recent, more detailed experiments confirm our observations about the 50% point (Crawford and Auton 1993; =-=Larrabee and Tsuji 1993-=-). 8 0 5000 10000 15000 20000 25000 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 Number of DP calls Ratio of clauses-to-variables Satisfiable 3 3 3 3 3 3 3 3 3 3 3 3 33 333 Unsatisfiable + + + + + + + + + + + + ++++... |

43 | Probabilistic analysis of the Davis Putnam procedure for solving the satisfiability problem - Franco, Paull - 1983 |

42 | Computational experience with an interior point algorithm on the satisfiability problem
- Kamath, Karmarkar, et al.
- 1990
(Show Context)
Citation Context ...roduced almost no hard cases. Nevertheless, papers continue to appear purporting to empirically demonstrate 2 the efficacy of some new procedure, but using just this distribution (e.g., Hooker, 1988; =-=Kamath et al. 1990-=-), or presenting data suggesting that very large satisfiability problems --- with thousands of propositional variables --- can be solved. How are we to evaluate these empirical results, given the dang... |

41 | Using deep structure to locate hard problems
- Williams, Hogg
- 1992
(Show Context)
Citation Context ...signing values to many variables, resulting in a deep search tree. Similar under- and over-constrained areas have been found for random instances of other NP-complete problems (Cheeseman et al. 1991; =-=Williams and Hogg 1992-=-). The curves in Figure 2 are for all formulas of a given size, that is they are composites of satisfiable and unsatisfiable subsets. In Figure 3 the median number of calls for 50-variable formulas is... |

34 |
On the complexity of regular resolution and the davisputnam procedure
- Galil
- 1977
(Show Context)
Citation Context ...rocedure we used for our tests is the Davis-Putnam procedure, which we describe below. We believe this was a good choice for two reasons: First, it is basically a variant of resolution (Vellino 1989; =-=Galil 1977-=-), the most widely used general reasoning method in AI; second, almost all empirical work on SAT testing has used one or another refinement of this method, which facilitates comparison. We suspect tha... |

29 |
Counting the number of solutions for instances of satisfiability
- Dubois
- 1991
(Show Context)
Citation Context ...tances with a ratio of variables to clauses of less than 1.7 almost all formulas are satisfiable. Moreover, for a ratio of 5.2 or higher, it has been shown that almost all formulas are unsatisfiable (=-=Dubois 1991-=-; Chvatal and Szemeredi 1988). Note that these ratios are still quite far removed from our experimentally determined threshold at 4.3. We now return to the discussion of our experimental results. To g... |

29 |
On the complexity of the satisfiability problem
- Goldberg
- 1979
(Show Context)
Citation Context ...argely on empirical results. A number of papers (some discussed below) have claimed that the difficulty of SAT on randomly generated problems is not so daunting. For example, an often--quoted result (=-=Goldberg 1979) suggests-=- that SAT can be readily solved "on average" in O(n 2 ) time. This does not settle the question of how well the methods will work in practice, but at first blush it does appear to be more re... |

19 | Some pitfalls for experimenters with random
- Mitchell, Levesque
- 1996
(Show Context)
Citation Context ...om 3-SAT formulas, as a function of the ratio of clauses--to--variables. picture, we limit our attention to formulas with K = 3, that is, random 3-SAT. (Some data on other values of K can be found in =-=[27]-=-.) For a given N and M , an instance of random 3-SAT is produced by randomly generating M clauses of length 3. Each clause is produced by randomly choosing 3 distinct variables from the set of N avail... |

12 | Probabilistic Performance of A Heuristic for the Satisfiability Problem - Franco, Ho - 1988 |

10 |
A Survey of Average Time Analyses of Satisfiability Algorithms
- PURDOM
- 1990
(Show Context)
Citation Context ...omly generated assignments would, with probability approaching 1, find a satisfying assignment in a constant number of guesses. Further analytic results for the constant-density model can be found in =-=[5,28,31]-=-. Franco and Paull [14] also investigated the performance of DP on random formulas with a fixed clause length of 3, and suggested that it might be more useful for generating random instances, an hypot... |

9 | The complexity of automated reasoning
- Vellino
- 1989
(Show Context)
Citation Context ...le). The SAT procedure we used for our tests is the Davis-Putnam procedure, which we describe below. We believe this was a good choice for two reasons: First, it is basically a variant of resolution (=-=Vellino 1989-=-; Galil 1977), the most widely used general reasoning method in AI; second, almost all empirical work on SAT testing has used one or another refinement of this method, which facilitates comparison. We... |

9 |
Solving the satisfiability problem by using randomized approach
- Wu, Tang
- 1992
(Show Context)
Citation Context ...omly generated assignments would, with probability approaching 1, find a satisfying assignment in a constant number of guesses. Further analytic results for the constant-density model can be found in =-=[5,28,31]-=-. Franco and Paull [14] also investigated the performance of DP on random formulas with a fixed clause length of 3, and suggested that it might be more useful for generating random instances, an hypot... |

8 |
The employee party problem
- Matula
- 1972
(Show Context)
Citation Context ... but almost certainly will not have any larger cliques. In other words, the probability of a random graph having a clique of size k is represented by a threshold function with the threshold at 2logn (=-=Matula 1972-=-; Bollobas 1985). It appears much harder to show formally that there exists a threshold phenomenon for SAT, even though our experimental results certainly do suggest so. A recent breakthrough was the ... |

5 | On random 2-sat - Vega - 1992 |

5 | Polynomial average-time satisfiability problems - Brown - 1987 |

3 |
Analysis of three simple heuristics on a random instance of k-SAT
- Frieze, Suen
- 1992
(Show Context)
Citation Context ...ng the theoretical bounds on the transition. As N gets large the probability that an instance of Random 3-SAT is satisfiable approaches 0 whenever the ratio of clauses to variables is less than 3.003 =-=[15]-=-, and approaches 1 when this ratio is greater than 4.758 [23]. As can be seen, our empirical results are in agreement with, though much more fine-grained than, the best theoretical bounds. To get some... |

2 | Probabilistic analysis of algorithms for NP-complete problems. United States Air Force Annual Scientific Report - Franco - 1986 |

2 | Average time analysis of simplified Davis-Putnam procedures - W, Brown - 1982 |