Results 1  10
of
78
SATzilla: Portfoliobased Algorithm Selection for SAT
"... It has been widely observed that there is no single “dominant ” SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a perinst ..."
Abstract

Cited by 145 (22 self)
 Add to MetaCart
(Show Context)
It has been widely observed that there is no single “dominant ” SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a perinstance basis. Building on previous work, we describe SATzilla, an automated approach for constructing perinstance algorithm portfolios for SAT that use socalled empirical hardness models to choose among their constituent solvers. This approach takes as input a distribution of problem instances and a set of component solvers, and constructs a portfolio optimizing a given objective function (such as mean runtime, percent of instances solved, or score in a competition). The excellent performance of our SATzilla portfolios has been independently verified in the 2007 SAT Competition, where our SATzilla07 solvers won three gold, one silver and one bronze medal. In this article, we go well beyond SATzilla07 by making the portfolio construction scalable and completely automated, and improving it by integrating local search solvers as candidate solvers, by predicting performance score instead of runtime, and by using hierarchical hardness models that take into account different types of SAT instances. We demonstrate the effectiveness of these new techniques in extensive experimental results on data sets including instances from the most recent SAT competition. 1.
Sequential ModelBased Optimization for General Algorithm Configuration (extended version)
"... Abstract. Stateoftheart algorithms for hard computational problems often expose many parameters that can be modified to improve empirical performance. However, manually exploring the resulting combinatorial space of parameter settings is tedious and tends to lead to unsatisfactory outcomes. Recen ..."
Abstract

Cited by 107 (27 self)
 Add to MetaCart
(Show Context)
Abstract. Stateoftheart algorithms for hard computational problems often expose many parameters that can be modified to improve empirical performance. However, manually exploring the resulting combinatorial space of parameter settings is tedious and tends to lead to unsatisfactory outcomes. Recently, automated approaches for solving this algorithm configuration problem have led to substantial improvements in the state of the art for solving various problems. One promising approach constructs explicit regression models to describe the dependence of target algorithm performance on parameter settings; however, this approach has so far been limited to the optimization of few numerical algorithm parameters on single instances. In this paper, we extend this paradigm for the first time to general algorithm configuration problems, allowing many categorical parameters and optimization for sets of instances. We experimentally validate our new algorithm configuration procedure by optimizing a local search and a tree search solver for the propositional satisfiability problem (SAT), as well as the commercial mixed integer programming (MIP) solver CPLEX. In these experiments, our procedure yielded stateoftheart performance, and in many cases outperformed the previous best configuration approach. 1
Run the GAMUT: A comprehensive approach to evaluating gametheoretic algorithms
 In AAMAS04
, 2004
"... We present GAMUT 1, a suite of game generators designed for testing gametheoretic algorithms. We explain why such a generator is necessary, offer a way of visualizing relationships between the sets of games supported by GAMUT, and give an overview of GAMUT’s architecture. We highlight the importanc ..."
Abstract

Cited by 90 (8 self)
 Add to MetaCart
(Show Context)
We present GAMUT 1, a suite of game generators designed for testing gametheoretic algorithms. We explain why such a generator is necessary, offer a way of visualizing relationships between the sets of games supported by GAMUT, and give an overview of GAMUT’s architecture. We highlight the importance of using comprehensive test data by benchmarking existing algorithms. We show surprisingly large variation in algorithm performance across different sets of games for two widelystudied problems: computing Nash equilibria and multiagent learning in repeated games. 2 1.
Using Casebased Reasoning in an Algorithm Portfolio for Constraint Solving
 IRISH CONFERENCE ON ARTIFICIAL INTELLIGENCE AND COGNITIVE SCIENCE
, 2008
"... It has been shown in areas such as satisfiability testing and integer linear programming that a carefully chosen combination of solvers can outperform the best individual solver for a given set of problems. This selection process is usually performed using a machine learning technique based on feat ..."
Abstract

Cited by 64 (1 self)
 Add to MetaCart
(Show Context)
It has been shown in areas such as satisfiability testing and integer linear programming that a carefully chosen combination of solvers can outperform the best individual solver for a given set of problems. This selection process is usually performed using a machine learning technique based on feature data extracted from constraint satisfaction problems. In this paper we present CPHYDRA, an algorithm portfolio for constraint satisfaction that uses casebased reasoning to determine how to solve an unseen problem instance by exploiting a case base of problem solving experience. We demonstrate the superiority of our portfolio over each of its constituent solvers using challenging benchmark problem instances from the most recent CSP Solver Competition.
Performance prediction and automated tuning of randomized and parametric algorithms: An initial investigation
, 2006
"... Machine learning can be utilized to build models that predict the runtime of search algorithms for hard combinatorial problems. Such empirical hardness models have previously been studied for complete, deterministic search algorithms. In this work, we demonstrate that such models can also make surp ..."
Abstract

Cited by 61 (23 self)
 Add to MetaCart
(Show Context)
Machine learning can be utilized to build models that predict the runtime of search algorithms for hard combinatorial problems. Such empirical hardness models have previously been studied for complete, deterministic search algorithms. In this work, we demonstrate that such models can also make surprisingly accurate runtime predictions for incomplete, randomized search methods, such as stochastic local search algorithms. We also show for the first time how information about an algorithm’s parameter settings can be incorporated into a model, and how such models can be used to automatically adjust the algorithm’s parameters on a perinstance basis in order to optimize its performance. Empirical results for Novelty + and SAPS on random and structured SAT instances show good predictive performance and significant speedups using our automatically determined parameter settings when compared to the default and best fixed parameter settings.
Understanding Random SAT: Beyond the ClausestoVariables Ratio
 In Proc. of CP04
"... It is well known that the ratio of the number of clauses to the number of variables in a random kSAT instance is highly correlated with the instance's empirical hardness. We consider the problem of identifying such features of random SAT instances automatically with machine learning. We des ..."
Abstract

Cited by 56 (19 self)
 Add to MetaCart
(Show Context)
It is well known that the ratio of the number of clauses to the number of variables in a random kSAT instance is highly correlated with the instance's empirical hardness. We consider the problem of identifying such features of random SAT instances automatically with machine learning. We describe and analyze models for three SAT solverskcnfs, oksolver and satzand for two different distributions of instances: uniform random 3SAT with varying ratio of clausestovariables, and uniform random 3SAT with fixed ratio of clausestovariables.
A Portfolio Approach to Algorithm Selection
 In IJCAI03
, 2003
"... this paper describes a technique for using rejection sampling to automatically generate such instances. In Figures 4 and 5 we show how our techniques are able to automatically skew two of the easiest CATS instance distributions towards harder regions. In fact, for these two distributions we generate ..."
Abstract

Cited by 48 (10 self)
 Add to MetaCart
(Show Context)
this paper describes a technique for using rejection sampling to automatically generate such instances. In Figures 4 and 5 we show how our techniques are able to automatically skew two of the easiest CATS instance distributions towards harder regions. In fact, for these two distributions we generated instances that were (respectively) 100 and 50 times harder than anything we had previously seen! Moreover, the average runtime for the new distributions was greater than the observed maximum running time on the original distribution
Satzilla07: The design and analysis of an algorithm portfolio for SAT
 In Thirteenth Internatioal Conference on Principles and Practice of Constraint Programming (CP’07
, 2007
"... Abstract. It has been widely observed that there is no “dominant” SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a perin ..."
Abstract

Cited by 45 (6 self)
 Add to MetaCart
(Show Context)
Abstract. It has been widely observed that there is no “dominant” SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a perinstance basis. Building on previous work, we describe a perinstance solver portfolio for SAT, SATzilla07, which uses socalled empirical hardness models to choose among its constituent solvers. We leverage new modelbuilding techniques such as censored sampling and hierarchical hardness models, and demonstrate the effectiveness of our techniques by building a portfolio of stateoftheart SAT solvers and evaluating it on several widelystudied SAT data sets. Overall, we show that our portfolio significantly outperforms its constituent algorithms on every data set. Our approach has also proven itself to be effective in practice: in the 2007 SAT competition, SATzilla07 won three gold medals, one silver, and one bronze; it is available online at
Propositional Satisfiability and Constraint Programming: a Comparative Survey
 ACM Computing Surveys
, 2006
"... Propositional Satisfiability (SAT) and Constraint Programming (CP) have developed as two relatively independent threads of research, crossfertilising occasionally. These two approaches to problem solving have a lot in common, as evidenced by similar ideas underlying the branch and prune algorithms ..."
Abstract

Cited by 38 (4 self)
 Add to MetaCart
Propositional Satisfiability (SAT) and Constraint Programming (CP) have developed as two relatively independent threads of research, crossfertilising occasionally. These two approaches to problem solving have a lot in common, as evidenced by similar ideas underlying the branch and prune algorithms that are most successful at solving both kinds of problems. They also exhibit differences in the way they are used to state and solve problems, since SAT’s approach is in general a blackbox approach, while CP aims at being tunable and programmable. This survey overviews the two areas in a comparative way, emphasising the similarities and differences between the two and the points where we feel that one technology can benefit from ideas or experience acquired
Boosting as a Metaphor for Algorithm Design
"... Hard computational problems are often solvable by multiple algorithms that each perform well on different problem instances. We describe techniques for building an algorithm portfolio that can outperform its constituent algorithms, just as the aggregate classifiers learned by boosting outperform ..."
Abstract

Cited by 34 (9 self)
 Add to MetaCart
Hard computational problems are often solvable by multiple algorithms that each perform well on different problem instances. We describe techniques for building an algorithm portfolio that can outperform its constituent algorithms, just as the aggregate classifiers learned by boosting outperform the classifiers of which they are composed. We also provide a method for generating test distributions to focus future algorithm design work on problems that are hard for an existing portfolio. We demonstrate the effectiveness of our techniques on the combinatorial auction winner determination problem, showing that our portfolio outperforms the stateoftheart algorithm by a factor of three.