Results 1  10
of
57
ParamILS: An automatic algorithm configuration framework
, 2009
"... The identification of performanceoptimizing parameter settings is an important part of the development and application of algorithms. We describe an automatic framework for this algorithm configuration problem. More formally, we provide methods for optimizing a target algorithm’s performance on a g ..."
Abstract

Cited by 96 (29 self)
 Add to MetaCart
(Show Context)
The identification of performanceoptimizing parameter settings is an important part of the development and application of algorithms. We describe an automatic framework for this algorithm configuration problem. More formally, we provide methods for optimizing a target algorithm’s performance on a given class of problem instances by varying a set of ordinal and/or categorical parameters. We review a family of localsearchbased algorithm configuration procedures and present novel techniques for accelerating them by adaptively limiting the time spent for evaluating individual configurations. We describe the results of a comprehensive experimental evaluation of our methods, based on the configuration of prominent complete and incomplete algorithms for SAT. We also present what is, to our knowledge, the first published work on automatically configuring the CPLEX mixed integer programming solver. All the algorithms we considered had default parameter settings that were manually identified with considerable effort. Nevertheless, using our automated algorithm configuration procedures, we achieved substantial and consistent performance improvements.
SATzilla: Portfoliobased Algorithm Selection for SAT
"... It has been widely observed that there is no single “dominant ” SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a perinst ..."
Abstract

Cited by 94 (16 self)
 Add to MetaCart
(Show Context)
It has been widely observed that there is no single “dominant ” SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a perinstance basis. Building on previous work, we describe SATzilla, an automated approach for constructing perinstance algorithm portfolios for SAT that use socalled empirical hardness models to choose among their constituent solvers. This approach takes as input a distribution of problem instances and a set of component solvers, and constructs a portfolio optimizing a given objective function (such as mean runtime, percent of instances solved, or score in a competition). The excellent performance of our SATzilla portfolios has been independently verified in the 2007 SAT Competition, where our SATzilla07 solvers won three gold, one silver and one bronze medal. In this article, we go well beyond SATzilla07 by making the portfolio construction scalable and completely automated, and improving it by integrating local search solvers as candidate solvers, by predicting performance score instead of runtime, and by using hierarchical hardness models that take into account different types of SAT instances. We demonstrate the effectiveness of these new techniques in extensive experimental results on data sets including instances from the most recent SAT competition. 1.
Automatic Algorithm Configuration based on Local Search
 IN AAAI ’07: PROC. OF THE TWENTYSECOND CONFERENCE ON ARTIFICAL INTELLIGENCE
, 2007
"... The determination of appropriate values for free algorithm parameters is a challenging and tedious task in the design of effective algorithms for hard problems. Such parameters include categorical choices (e.g., neighborhood structure in local search or variable/value ordering heuristics in tree sea ..."
Abstract

Cited by 60 (28 self)
 Add to MetaCart
The determination of appropriate values for free algorithm parameters is a challenging and tedious task in the design of effective algorithms for hard problems. Such parameters include categorical choices (e.g., neighborhood structure in local search or variable/value ordering heuristics in tree search), as well as numerical parameters (e.g., noise or restart timing). In practice, tuning of these parameters is largely carried out manually by applying rules of thumb and crude heuristics, while more principled approaches are only rarely used. In this paper, we present a local search approach for algorithm configuration and prove its convergence to the globally optimal parameter configuration. Our approach is very versatile: it can, e.g., be used for minimising runtime in decision problems or for maximising solution quality in optimisation problems. It further applies to arbitrary algorithms, including heuristic tree search and local search algorithms, with no limitation on the number of parameters. Experiments in four algorithm configuration scenarios demonstrate that our automatically determined parameter settings always outperform the algorithm defaults, sometimes by several orders of magnitude. Our approach also shows better performance and greater flexibility than the recent CALIBRA system. Our ParamILS code, along with instructions on how to use it for tuning your own algorithms, is available online at
Boosting Verification by Automatic Tuning of Decision Procedures
 SEVENTH INTERNATIONAL CONFERENCE ON FORMAL METHODS IN COMPUTERAIDED DESIGN
, 2007
"... Parameterized heuristics abound in computer aided design and verification, and manual tuning of the respective parameters is difficult and timeconsuming. Very recent results from the artificial intelligence (AI) community suggest that this tuning process can be automated, and that doing so can lead ..."
Abstract

Cited by 46 (30 self)
 Add to MetaCart
(Show Context)
Parameterized heuristics abound in computer aided design and verification, and manual tuning of the respective parameters is difficult and timeconsuming. Very recent results from the artificial intelligence (AI) community suggest that this tuning process can be automated, and that doing so can lead to significant performance improvements; furthermore, automated parameter optimization can provide valuable guidance during the development of heuristic algorithms. In this paper, we study how such an AI approach can improve a stateoftheart SAT solver for large, realworld bounded modelchecking and software verification instances. The resulting, automaticallyderived parameter settings yielded runtimes on average 4.5 times faster on bounded model checking instances and 500 times faster on software verification problems than extensive handtuning of the decision procedure. Furthermore, the availability of automatic tuning influenced the design of the solver, and the automaticallyderived parameter settings provided a deeper insight into the properties of problem instances.
Satzilla07: The design and analysis of an algorithm portfolio for SAT
 In Thirteenth Internatioal Conference on Principles and Practice of Constraint Programming (CP’07
, 2007
"... Abstract. It has been widely observed that there is no “dominant” SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a perin ..."
Abstract

Cited by 36 (5 self)
 Add to MetaCart
(Show Context)
Abstract. It has been widely observed that there is no “dominant” SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a perinstance basis. Building on previous work, we describe a perinstance solver portfolio for SAT, SATzilla07, which uses socalled empirical hardness models to choose among its constituent solvers. We leverage new modelbuilding techniques such as censored sampling and hierarchical hardness models, and demonstrate the effectiveness of our techniques by building a portfolio of stateoftheart SAT solvers and evaluating it on several widelystudied SAT data sets. Overall, we show that our portfolio significantly outperforms its constituent algorithms on every data set. Our approach has also proven itself to be effective in practice: in the 2007 SAT competition, SATzilla07 won three gold medals, one silver, and one bronze; it is available online at
Propositional Satisfiability and Constraint Programming: a Comparative Survey
 ACM Computing Surveys
, 2006
"... Propositional Satisfiability (SAT) and Constraint Programming (CP) have developed as two relatively independent threads of research, crossfertilising occasionally. These two approaches to problem solving have a lot in common, as evidenced by similar ideas underlying the branch and prune algorithms ..."
Abstract

Cited by 34 (4 self)
 Add to MetaCart
Propositional Satisfiability (SAT) and Constraint Programming (CP) have developed as two relatively independent threads of research, crossfertilising occasionally. These two approaches to problem solving have a lot in common, as evidenced by similar ideas underlying the branch and prune algorithms that are most successful at solving both kinds of problems. They also exhibit differences in the way they are used to state and solve problems, since SAT’s approach is in general a blackbox approach, while CP aims at being tunable and programmable. This survey overviews the two areas in a comparative way, emphasising the similarities and differences between the two and the points where we feel that one technology can benefit from ideas or experience acquired
Predicting Learnt Clauses Quality in Modern SAT Solvers
, 2009
"... Beside impressive progresses made by SAT solvers over the last ten years, only few works tried to understand why Conflict Directed Clause Learning algorithms (CDCL) are so strong and efficient on most industrial applications. We report in this work a key observation of CDCL solvers behavior on this ..."
Abstract

Cited by 31 (6 self)
 Add to MetaCart
Beside impressive progresses made by SAT solvers over the last ten years, only few works tried to understand why Conflict Directed Clause Learning algorithms (CDCL) are so strong and efficient on most industrial applications. We report in this work a key observation of CDCL solvers behavior on this family of benchmarks and explain it by an unsuspected side effect of their particular Clause Learning scheme. This new paradigm allows us to solve an important, still open, question: How to designing a fast, static, accurate, and predictive measure of new learnt clauses pertinence. Our paper is followed by empirical evidences that show how our new learning scheme improves stateofthe art results by an order of magnitude on both SAT and UNSAT industrial problems.
Exploring Hyperheuristic Methodologies with Genetic Programming
"... Hyperheuristics represent a novel search methodology that is motivated by the goal of automating the process of selecting or combining simpler heuristics in order to solve hard computational search problems. An extension of the original hyperheuristic idea is to generate new heuristics which are n ..."
Abstract

Cited by 22 (11 self)
 Add to MetaCart
(Show Context)
Hyperheuristics represent a novel search methodology that is motivated by the goal of automating the process of selecting or combining simpler heuristics in order to solve hard computational search problems. An extension of the original hyperheuristic idea is to generate new heuristics which are not currently known. These approaches operate on a search space of heuristics rather than directly on a search space of solutions to the underlying problem which is the case with most metaheuristics implementations. In the majority of hyperheuristic studies so far, a framework is provided with a set of human designed heuristics, taken from the literature, and with good measures of performance in practice. A less well studied approach aims to generate new heuristics from a set of potential heuristic components. The purpose of this chapter is to discuss this class of hyperheuristics, in which Genetic Programming is the most widely used methodology. A detailed discussion is presented including the steps needed to apply this technique, some representative case studies, a literature review of related work, and a discussion of relevant issues. Our aim is to convey the exciting potential of this innovative approach for automating the heuristic design process
Learning dynamic algorithm portfolios
 ANN MATH ARTIF INTELL (2006) 47:295–328
, 2006
"... Algorithm selection can be performed using a model of runtime distribution, learned during a preliminary training phase. There is a tradeoff between the performance of modelbased algorithm selection, and the cost of learning the model. In this paper, we treat this tradeoff in the context of bandi ..."
Abstract

Cited by 18 (1 self)
 Add to MetaCart
(Show Context)
Algorithm selection can be performed using a model of runtime distribution, learned during a preliminary training phase. There is a tradeoff between the performance of modelbased algorithm selection, and the cost of learning the model. In this paper, we treat this tradeoff in the context of bandit problems. We propose a fully dynamic and online algorithm selection technique, with no separate training phase: all candidate algorithms are run in parallel, while a model incrementally learns their runtime distributions. A redundant set of time allocators uses the partially trained model to propose machine time shares for the algorithms. A bandit problem solver mixes the modelbased shares with a uniform share, gradually increasing the impact of the best time allocators as the model improves. We present experiments with a set of SAT solvers on a mixed SATUNSAT benchmark; and with a set of solvers for the Auction Winner Determination problem.
ISAC  InstanceSpecific Algorithm Configuration
"... We present a new method for instancespecific algorithm configuration (ISAC). It is based on the integration of the algorithm configuration system GGA and the recently proposed stochastic offline programming paradigm. ISAC is provided a solver with categorical, ordinal, and/or continuous parameter ..."
Abstract

Cited by 17 (2 self)
 Add to MetaCart
(Show Context)
We present a new method for instancespecific algorithm configuration (ISAC). It is based on the integration of the algorithm configuration system GGA and the recently proposed stochastic offline programming paradigm. ISAC is provided a solver with categorical, ordinal, and/or continuous parameters, a training benchmark set of input instances for that solver, and an algorithm that computes a feature vector that characterizes any given instance. ISAC then provides high quality parameter settings for any new input instance. Experiments on a variety of different constrained optimization and constraint satisfaction solvers show that automatic algorithm configuration vastly outperforms manual tuning. Moreover, we show that instancespecific tuning frequently leads to significant speedups over instanceoblivious configurations.