## Ten challenges redux: Recent progress in propositional reasoning and search (2003)

### Cached

### Download Links

Venue: | In Proceedings of CP ’03 |

Citations: | 15 - 1 self |

### BibTeX

@INPROCEEDINGS{Kautz03tenchallenges,

author = {Henry Kautz and Bart Selman},

title = {Ten challenges redux: Recent progress in propositional reasoning and search},

booktitle = {In Proceedings of CP ’03},

year = {2003},

pages = {1--18}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract. In 1997 we presented ten challenges for research on satisfiability testing [1]. In this paper we review recent progress towards each of these challenges, including our own work on the power of clause learning and randomized restart policies. 1

### Citations

1380 |
Symbolic Model Checking
- McMillan
- 1993
(Show Context)
Citation Context ...or identifying conjunctions of nested equivalencies subformulas using linear programming. 4 Much work in verification has involved non-clausal representations, in particular Boolean Decision Diagrams =-=[31, 32]-=-; but the large body of work on BDD’s will not be further discussed here.sproof of the same size, and tree-like proofs must sometimes be exponentially larger than the DAG-like proofs generated by gene... |

1219 | Chaff: Engineering an Efficient SAT Solver
- Moskewicz, Madigan, et al.
- 2001
(Show Context)
Citation Context ...o the original formula. Rel-sat and SATO were suprisingly powerful, and even able to solve open problems in finite mathematics. Clause learning was further developed for the solvers GRASP [46], Chaff =-=[47, 48]-=- and BerkMin [49], and is currently a key technique in backtracking SAT solvers for applications such as verification. Marquis-Silva [50] observed that clause learning can be viewed as adding resolven... |

1131 |
A computation procedure for quantification theory
- Davis, Putman
- 1960
(Show Context)
Citation Context ...ystematic methods for propositional reasoning on clausal formulas were based on creating a resolution proof tree. 4 This includes the depth-first search Davis-Putnam-LovelandLogemann procedure (DPLL) =-=[33, 34]-=-, where the proof tree can be recovered from the trace of the algorithm’s execution, but is not explicitly represented in a data structure (the algorithm only maintains a single branch of the proof tr... |

780 | Symbolic Model Checking without BDDs
- Biere, Cimatti, et al.
- 1999
(Show Context)
Citation Context ...ormance of Boolean satisfiability (SAT) solvers. Despite the worst-case exponential run time of all known algorithms, SAT solvers are now in routine use for applications such as hardware verification =-=[2]-=- that involve solving hard structured problems with up to a million variables [3, 4]. Each year the International Conference on Theory and Applications of Satisfiability Testing hosts a SAT competitio... |

776 |
A machine program for theorem proving
- Davis, Logemann, et al.
- 1962
(Show Context)
Citation Context ...and the other highly structured. CHALLENGE 1: Prove that a hard 700 variable random 3-SAT formula is unsatisfiable. When we formulated in this challenge in 1997, complete SAT procedures based on DPLL =-=[9]-=- could handle around 300 to 400 variable hard random 3SAT problems. Progress in recent years had slowed and it was not clear DPLL could be much improved upon for random 3-SAT. In particular, the there... |

709 | D.: A new method for solving hard satisfiability problems
- Selman, Levesque, et al.
- 1992
(Show Context)
Citation Context ...en lessseffective than local search style methods. Note that this is not always the case. For example, in runs on verification benchmarks, Velev [3] showed how the performance of DLM [72] and Walksat =-=[24, 73]-=- is comparable to many of the best DPLL style methods. A series of papers, such as [74–79] among others, has also led to a much improved understanding of local search methods for SAT. Hirsch [80] intr... |

705 | Diagnosing multiple faults - Kleer, Williams - 1987 |

654 | Symbolic model checking: 1020 states and beyond
- Burch, Clarke, et al.
- 1990
(Show Context)
Citation Context ...or identifying conjunctions of nested equivalencies subformulas using linear programming. 4 Much work in verification has involved non-clausal representations, in particular Boolean Decision Diagrams =-=[31, 32]-=-; but the large body of work on BDD’s will not be further discussed here.sproof of the same size, and tree-like proofs must sometimes be exponentially larger than the DAG-like proofs generated by gene... |

550 | Pushing the Envelope: Planning, Propositional Logic, and Stochastic Search
- Kautz, Selman
- 1996
(Show Context)
Citation Context ...more unifying, domain-independent principles. CHALLENGE 9: Find encodings of real-world domains which are robust in the sense that “near models” are actually “near solutions”. In our work on planning =-=[104]-=-, we noticed that assignments that satisfy all but a few of the clauses encoding our planning problems often represented action sequences that were very different from valid plans. This means that the... |

394 | GRASP: A new search algorithm for satisfiability
- Silva, Sakallah
- 1996
(Show Context)
Citation Context ...ause added to the original formula. Rel-sat and SATO were suprisingly powerful, and even able to solve open problems in finite mathematics. Clause learning was further developed for the solvers GRASP =-=[46]-=-, Chaff [47, 48] and BerkMin [49], and is currently a key technique in backtracking SAT solvers for applications such as verification. Marquis-Silva [50] observed that clause learning can be viewed as... |

347 | The Relative Efficiency of Propositional Proof systems
- Cook, Reckhow
- 1979
(Show Context)
Citation Context ...However, there are known fundamental limitations on the size of the shortest resolution proofs that can be obtained in this manner, even with ideal branching strategies. The study of proof complexity =-=[35]-=- compares inference systems in terms of the sizes of the shortest proofs they sanction. For example, two proof systems are linearly related if there is a linear function f(n) such that for any proof o... |

332 | Boosting combinatorial search through randomization
- Gomes, Selman, et al.
- 1998
(Show Context)
Citation Context ...d upon more than once in any path from the root to a leaf. It is easy to see that all tree-like proofs are regular but not vice-versa. They further showed that combining clause learning with restarts =-=[52, 53]-=- (where learned clauses are saved between restarts) is equivalent to general resolution. However, the questions of whether clause learning is strictly stronger than regular resolution — that is, wheth... |

309 | Efficient conflict driven learning in boolean satisfiability solver
- Zhang, Malik, et al.
(Show Context)
Citation Context ...o the original formula. Rel-sat and SATO were suprisingly powerful, and even able to solve open problems in finite mathematics. Clause learning was further developed for the solvers GRASP [46], Chaff =-=[47, 48]-=- and BerkMin [49], and is currently a key technique in backtracking SAT solvers for applications such as verification. Marquis-Silva [50] observed that clause learning can be viewed as adding resolven... |

294 |
and easy distributions of sat problems
- Hard
- 1992
(Show Context)
Citation Context ...he range [11, 100] across the problem distribution. The median run time of any particular instance is 2b, so we expect that median run times of the sampled instances would fall uniformly in the range =-=[22, 200]-=-. A binary clustering by the median run times of the samples should give one cluster where the instance medians are in the range [22, 110] (equivalently, b ∈ [11, 55]) and another cluster where the in... |

285 | Local Search Strategies for Satisfiability Testing
- Selman, Cohen, et al.
(Show Context)
Citation Context ...e solution of hard random 3-SAT phase transition instances of up to a million variables in about 2 hours of CPU time. For comparsion, the previously most effective procedure for random 3-SAT, WalkSAT =-=[24]-=-, can handle instances with around 100,000 variables within this timeframe. The exact scaling properties of survey propagation — and WalksSAT for that matter — are still unknown. In conclusion, even t... |

275 | Blackbox: Unifying satbased and graph-based planning
- Kautz, Selman
- 1999
(Show Context)
Citation Context ...s from cryptography [108]. Structured problem generators have also been created by linking a random generator for some particular domain to a SAT translator. For example, the Blackbox planning system =-=[109]-=- can be used to convert STRIPS planning problems into CNF formulas. The Blackbox distribution included a simple generator for random logistics planning problems, making it easy to generate random SAT ... |

270 | The intractibility of resolution - Haken - 1985 |

259 | Forward Reasoning and Dependency-Directed Backtracking in a System for Computer-Aided Circuit Analysis - Stallman, Sussman - 1977 |

256 | Diagnostic Reasoning Based on Structure and Behavior - Davis - 1985 |

218 | Many Hard Examples for Resolution - Chv'atal, Szemer'edi - 1988 |

208 |
The breakout method for escaping from local minima
- Morris
- 1993
(Show Context)
Citation Context ...arly hard for local search methods because such algorithms tend to become trapped at a near-solution such that a small subset of clauses is never satisfied simultaneously. Clause re-weighting schemes =-=[28, 29]-=- try to smooth out the search space by giving higher weight to clauses that are often unsatisfied. A clause weighting scheme based on Langrange multipliers [30] was able to solve the 16-bit versions o... |

186 |
L.: Determining computational complexity from characteristic ’phase transitions. Nature 400
- Monasson, Zecchina, et al.
- 1999
(Show Context)
Citation Context ...on structure of random instances. In particular, it can be shown that a relatively large set of backbone variables suddenly emerges when one passes though the phase transition point for k-SAT (k ≥ 3) =-=[12]-=-. Using a backbone-guided search heuristic, Dubois and Dequen can solve a 700 variable unsatisfiable, hard random 3-SAT instance in around 25 days of CPU time, thereby approaching practical feasibilit... |

168 | Symmetry-breaking predicates for search problems
- Crawford, Ginsberg, et al.
- 1996
(Show Context)
Citation Context ...d a renaming (permutation) ψ of the variables in a formula that resulted in the same set of clauses, which justified a new rule of inference: from any clause (a ∨ b ∨ ...), infer (ψ(a) ∨ ψ(b) ∨ ...). =-=[59]-=- introduced a different way of using symmetries, by strengthening the formula through the addition of clauses that ruled out all but one of the symmetric cases. The drawback of this approach appeared ... |

166 | Sharp Thresholds of Graph Properties, and the k-sat Problem - Friedgut - 1999 |

158 | Encoding plans in propositional logic
- Kautz, McAllester, et al.
- 1996
(Show Context)
Citation Context ...it-propagation. By exploiting some key ideas from CSPs, such as m-loosenes [99], one can in fact optimize the SAT encodings [100]. Examples of other work in the area are on encoding planning problems =-=[101, 102]-=- and quasi-group completion problems (a multi-coloring task) [103]. This work shows clearly that encodings have a significant impact on the practical solvability of the underlying problems. Some gener... |

156 | Bounded model checking using satisfiability solving
- Clarke, Biere, et al.
- 2001
(Show Context)
Citation Context ...es of the sampled instances would fall uniformly in the range [22, 200]. A binary clustering by the median run times of the samples should give one cluster where the instance medians are in the range =-=[22, 110]-=- (equivalently, b ∈ [11, 55]) and another cluster where the instance medians are in the range [111, 200] (equivalently, b ∈ [56, 100]). Each cluster, or sub-ensemble, yields an ensemble run time distr... |

154 | Heavy-tailed phenomena in satisfiability and constraint satisfaction problems - Gomes, Selman, et al. - 2000 |

147 |
Analytic and algorithmic solution of random satisfiability problems. Science 297:812–815
- Mézard, Parisi, et al.
- 2002
(Show Context)
Citation Context ...ts in the area have been obtained by using tools from statistical physics. This work has recently culminated in a new algorithm for solving satisfiable k-SAT instances near the phase transition point =-=[23]-=-. The method is called survey propagation and involves, in a sense, a sophisticated probabilistic analysis of the problem instance under consideration. An efficient implementation enables the solution... |

129 | Automatic SAT-compilation of planning problems
- Ernst, D, et al.
- 1997
(Show Context)
Citation Context ...it-propagation. By exploiting some key ideas from CSPs, such as m-loosenes [99], one can in fact optimize the SAT encodings [100]. Examples of other work in the area are on encoding planning problems =-=[101, 102]-=- and quasi-group completion problems (a multi-coloring task) [103]. This work shows clearly that encodings have a significant impact on the practical solvability of the underlying problems. Some gener... |

126 | Optimal speedup of Las Vegas algorithms
- Luby, Sinclair, et al.
- 1993
(Show Context)
Citation Context ...p to a million variables, from, e.g., verification applications, can be solved with current state-of-the-art solvers. An important related issue is how to decide on a good restart policy. Luby et al. =-=[92]-=- described restart policies for general randomized algorithms for two scenarios where runtime itself is the only observable: (i) when each run is a random sample from a known distribution, one can cal... |

120 | D.G.: Finding hard instances of the satisfiability problem: A survey - Cook, Mitchell - 1997 |

105 |
An efficient propositional prover
- SATO
- 1997
(Show Context)
Citation Context ...n tree-like resolution can be made practical for satisfiability testing. Two new satisfiability testing algorithms were introduced in 1997, the same year as our challenge paper: rel-sat [41] and SATO =-=[42]-=-. Both were versions of DPLL augmented with “conflict clause learning”, a technique that grew out of research in AI on explanation-based approaches to speed-up learning [43–45]. The idea in clause lea... |

105 | Search in a small world - Walsh |

101 | An Empirical Study of Greedy Local Search for Satisfiability Testing
- Selman, Kautz
(Show Context)
Citation Context ...arly hard for local search methods because such algorithms tend to become trapped at a near-solution such that a small subset of clauses is never satisfied simultaneously. Clause re-weighting schemes =-=[28, 29]-=- try to smooth out the search space by giving higher weight to clauses that are often unsatisfied. A clause weighting scheme based on Langrange multipliers [30] was able to solve the 16-bit versions o... |

91 | Backdoors to typical case complexity
- Williams, Gomes, et al.
- 2003
(Show Context)
Citation Context ...ts. Work on strong backdoor sets, which are small sets of variables that, together with a polytime propagation method, can demonstrate unsatisfiability may lead to some new opportunities in this area =-=[70]-=-. CHALLENGE 6: Improve stochastic local search on structured problems by efficiently handling variable dependencies. DPLL procedures handle variable dependency quite effectively through unit propagati... |

91 | On the run-time behaviour of stochastic local search algorithms for SAT - Hoos - 1999 |

89 | Effective use of Boolean satisfiability procedures in the formal verification of superscalar and VLIW microprocessors
- Velev, Bryant
(Show Context)
Citation Context ...al run time of all known algorithms, SAT solvers are now in routine use for applications such as hardware verification [2] that involve solving hard structured problems with up to a million variables =-=[3, 4]-=-. Each year the International Conference on Theory and Applications of Satisfiability Testing hosts a SAT competition that highlights a new group of “world’s fastest” SAT solvers, and presents detaile... |

87 | Ten challenges in propositional reasoning and search
- Selman, Kautz, et al.
- 1997
(Show Context)
Citation Context ...z@cs.washington.edu 2 Department of Computer Science Cornell University Ithaca, NY 14853 USA selman@cs.cornell.edu Abstract. In 1997 we presented ten challenges for research on satisfiability testing =-=[1]-=-. In this paper we review recent progress towards each of these challenges, including our own work on the power of clause learning and randomized restart policies. 1 Introduction The past few years ha... |

86 | Integrating equivalency reasoning into Davis-Putnam
- Li
- 2000
(Show Context)
Citation Context ... challenge was solved in 1998 by preprocessing the formula to detect chains of literals that are equivalent considering binary clauses alone, and then applying DPLL after simplification [26]. 3 Later =-=[27]-=- showed similar performance by performance equivalency detection at every node in the search tree. Parity problems are particularly hard for local search methods because such algorithms tend to become... |

86 | Sakallah, Generic ILP versus specialized 0-1 ILP: an update - Aloul, Ramani, et al. |

84 | Generating satisfiable problem instances
- Achlioptas, Gomes, et al.
- 2000
(Show Context)
Citation Context ...nes [99], one can in fact optimize the SAT encodings [100]. Examples of other work in the area are on encoding planning problems [101, 102] and quasi-group completion problems (a multi-coloring task) =-=[103]-=-. This work shows clearly that encodings have a significant impact on the practical solvability of the underlying problems. Some general lessons have been obtained, but there is still a need for more ... |

82 | T.: Easy problems are sometimes hard - Gent, Walsh - 1994 |

75 | Problem structure in the presence of perturbations
- Gomes, Selman
(Show Context)
Citation Context ...to real-world instances. The final challenge is in response to the concern that the random k-SAT formulas that dominated benchmarks in 1997 might begin to drive research in the wrong direction [106]. =-=[107]-=- introduced a generation model based on the quasigroup (or Latin square) completion problem (QCP). The task is to determine if a partially colored square can be completed so that no color is repeated ... |

71 |
2003, ‘BerkMin: A fast and robust SAT solver
- Goldberg, Novikov
(Show Context)
Citation Context ...a. Rel-sat and SATO were suprisingly powerful, and even able to solve open problems in finite mathematics. Clause learning was further developed for the solvers GRASP [46], Chaff [47, 48] and BerkMin =-=[49]-=-, and is currently a key technique in backtracking SAT solvers for applications such as verification. Marquis-Silva [50] observed that clause learning can be viewed as adding resolvents to a tree-like... |

68 | A backbone-search heuristic for efficient solving of hard 3-SAT formulae
- Dubois, Dequen
- 2001
(Show Context)
Citation Context ...-SAT. In particular, the there was the possibility that the best DPLL methods were obtaining search trees that were close to minimal in terms of the number of backtrack points [10]. Dubois and Dequen =-=[11]-=-, however, showed that there was still room for improvement. They introduced a new branching heuristic that exploits so-called “backbone” variables in a SAT problem. A backbone variable of a formula i... |

67 | Effective Preprocessing with Hyper-Resolution and Equality Reduction - Bacchus, Winter |

66 | A Bayesian approach to tackling hard computational problems
- Horvitz
- 2001
(Show Context)
Citation Context ...ate a fixed optimal cutoff; (ii) when there is no knowledge of the distribution, a universal schedule mixing short and longer cutoffs comes within a log factor of the minimal run time. Horvitz et al. =-=[93]-=- showed that it is possible to do better than Luby’s fixed optimal policy by making observations of a variety of features related to the nature and progress of problem solving during an early portion ... |

65 | UnitWalk: A new SAT solver that uses local search guided by unit clause elimination
- Hirsch, Kojevnikov
(Show Context)
Citation Context ... [24, 73] is comparable to many of the best DPLL style methods. A series of papers, such as [74–79] among others, has also led to a much improved understanding of local search methods for SAT. Hirsch =-=[80]-=- introduces a local search procedure, UnitWalk, where variable dependencies are propagated explicitly as part of the search process. The propagation strategy is closed related to the one studied in [8... |

65 | Satisfiability coding lemma
- Paturi, Pudlák, et al.
- 1999
(Show Context)
Citation Context ...0] introduces a local search procedure, UnitWalk, where variable dependencies are propagated explicitly as part of the search process. The propagation strategy is closed related to the one studied in =-=[81]-=-. UnitWalk is quite effective on certain classes of structured problems but there is still room for improvement. Comparisons with WalkSat shows that neither strategy dominates. This led to QingTing [8... |

64 | Stochastic Local Search – Methods, Models, Applications - Hoos - 1998 |