Adaptive operator selection with dynamic multi-armed bandits (2008)

by Luis DaCosta, Alvaro Fialho, Marc Schoenauer, Michele Sebag
Venue:In Proceedings of the 10th annual conference on Genetic and evolutionary computation, GECCO '08
Add To MetaCart
Results 1 - 10 of 26

M.: Extreme value based adaptive operator selection

by Álvaro Fialho, Luís Da Costa, Marc Schoenauer, Michèle Sebag - Proc. Intl. Conference on Parallel Solving from Nature, LNCS , 2008
"... Abstract. Credit Assignment is an important ingredient of several proposals that have been made for Adaptive Operator Selection. Instead of the average fitness improvement of newborn offspring, this paper proposes to use some empirical order statistics of those improvements, arguing that rare but hi ..."
Abstract - Cited by 26 (11 self) - Add to MetaCart
Abstract. Credit Assignment is an important ingredient of several proposals that have been made for Adaptive Operator Selection. Instead of the average fitness improvement of newborn offspring, this paper proposes to use some empirical order statistics of those improvements, arguing that rare but highly beneficial jumps matter as much or more than frequent but small improvements. An extreme value based Credit Assignment is thus proposed, rewarding each operator with the best fitness improvement observed in a sliding window for this operator. This mechanism, combined with existing Adaptive Operator Selection rules, is investigated in an EC-like setting. First results show that the proposed method allows both the Adaptive Pursuit and the Dynamic Multi-Armed Bandit selection rules to actually track the best operators along evolution. 1
(Show Context)

Analysis of Adaptive Operator Selection Techniques on the Royal Road and Long K-Path Problems

by Álvaro Fialho, Marc Schoenauer, Michèle Sebag , 2009
"... One of the choices that most affect the performance of Evolutionary Algorithms is the selection of the variation operators that are efficient to solve the problem at hand. This work presents an empirical analysis of different Adaptive Operator Selection (AOS) methods, i.e., techniques that automatic ..."
Abstract - Cited by 17 (10 self) - Add to MetaCart
One of the choices that most affect the performance of Evolutionary Algorithms is the selection of the variation operators that are efficient to solve the problem at hand. This work presents an empirical analysis of different Adaptive Operator Selection (AOS) methods, i.e., techniques that automatically select the operator to be applied among the available ones, while searching for the solution. Four previously published operator selection rules are combined to four different credit assignment mechanisms. These 16 AOS combinations are analyzed and compared in the light of two wellknown benchmark problems in Evolutionary Computation, the Royal Road and the Long K-Path.
(Show Context)

Analyzing Bandit-based Adaptive Operator Selection Mechanisms

by Álvaro Fialho, Luis Da Costa, Marc Schoenauer, Michèle Sebag - N/P, AMAI – SPECIAL ISSUE ON LION , 2010
"... Several techniques have been proposed to tackle the Adaptive Operator Selection (AOS) issue in Evolutionary Algorithms. Some recent proposals are based on the Multi-Armed Bandit (MAB) paradigm: each operator is viewed as one arm of a MAB problem, and the rewards are mainly based on the fitness imp ..."
Abstract - Cited by 17 (1 self) - Add to MetaCart
Several techniques have been proposed to tackle the Adaptive Operator Selection (AOS) issue in Evolutionary Algorithms. Some recent proposals are based on the Multi-Armed Bandit (MAB) paradigm: each operator is viewed as one arm of a MAB problem, and the rewards are mainly based on the fitness improvement brought by the corresponding operator to the individual it is applied to. However, the AOS problem is dynamic, whereas standard MAB algorithms are known to optimally solve the exploitation versus exploration trade-off in static settings. An original dynamic variant of the standard MAB Upper Confidence Bound algorithm is proposed here, using a sliding time window to compute both its exploitation and exploration terms. In order to perform sound comparisons between AOS algorithms, artificial scenarios have been proposed in the literature. They are extended here toward smoother transitions between different reward settings. The resulting original testbed also includes a real evolutionary algorithm that is applied to the well-known Royal Road problem. It is used here to perform a thorough analysis of the behavior of AOS algorithms, to assess their sensitivity with respect to their own hyper-parameters, and to propose a sound
(Show Context)

Dynamic multi-armed bandits and extreme value-based rewards for adaptive operator selection in evolutionary algorithms

by Álvaro Fialho, Luis Da Costa, Marc Schoenauer, Michèle Sebag , 2009
"... Abstract. The performance of many efficient algorithms critically depends on the tuning of their parameters, which on turn depends on the problem at hand. For example, the performance of Evolutionary Algorithms critically depends on the judicious setting of the operator rates. The Adaptive Operator ..."
Abstract - Cited by 15 (9 self) - Add to MetaCart
Abstract. The performance of many efficient algorithms critically depends on the tuning of their parameters, which on turn depends on the problem at hand. For example, the performance of Evolutionary Algorithms critically depends on the judicious setting of the operator rates. The Adaptive Operator Selection (AOS) heuristic that is proposed here rewards each operator based on the extreme value of the fitness improvement lately incurred by this operator, and uses a Multi-Armed Bandit (MAB) selection process based on those rewards to choose which operator to apply next. This Extreme-based Multi-Armed Bandit approach is experimentally validated against the Average-based MAB method, and is shown to outperform previously published methods, whether using a classical Average-based rewarding technique or the same Extreme-based mechanism. The validation test suite includes the easy One-Max problem and a family of hard problems known as “Long k-paths”. 1
(Show Context)

" Toward Comparison-based Adaptive Operator Selection

by Álvaro Fialho, Marc Schoenauer, Michèle Sebag - GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE (GECCO) , 2010
"... Adaptive Operator Selection (AOS) turns the impacts of the applications of variation operators into Operator Selection through a Credit Assignment mechanism. However, most Credit Assignment schemes make direct use of the fitness gain between parent and offspring. A first issue is that the Operator S ..."
Abstract - Cited by 13 (9 self) - Add to MetaCart
Adaptive Operator Selection (AOS) turns the impacts of the applications of variation operators into Operator Selection through a Credit Assignment mechanism. However, most Credit Assignment schemes make direct use of the fitness gain between parent and offspring. A first issue is that the Operator Selection technique that uses such kind of Credit Assignment is likely to be highly dependent on the a priori unknown bounds of the fitness function. Additionally, these bounds are likely to change along evolution, as fitness gains tend to get smaller as convergence occurs. Furthermore, and maybe more importantly, a fitness-based credit assignment forbid any invariance by monotonous transformation of the fitness, what is a usual source of robustness for comparisonbased Evolutionary Algorithms. In this context, this paper proposes two new Credit Assignment mechanisms, one inspired by the Area Under the Curve paradigm, and the other close to the Sum of Ranks. Using fitness improvement as raw reward, and directly coupled to a Multi-Armed Bandit Operator Selection Rule, the resulting AOS obtain very good performances on both the OneMax problem and some artificial scenarios, while demonstrating their robustness with respect to hyper-parameter and fitness transformations. Furthermore, using fitness ranks as raw reward results in a fully comparison-based AOS with reasonable performances.
(Show Context)

Extreme compass and dynamic multi-armed bandits for adaptive operator selection

by Jorge Maturana, Álvaro Fialho, Frédéric Saubion, Marc Schoenauer, Michèle Sebag - In Proc. IEEE Congress on Evol. Comp , 2009
"... Abstract — The goal of Adaptive Operator Selection is the on-line control of the choice of variation operators within Evolutionary Algorithms. The control process is based on two main components, the credit assignment, that defines the reward that will be used to evaluate the quality of an operator ..."
Abstract - Cited by 11 (5 self) - Add to MetaCart
Abstract — The goal of Adaptive Operator Selection is the on-line control of the choice of variation operators within Evolutionary Algorithms. The control process is based on two main components, the credit assignment, that defines the reward that will be used to evaluate the quality of an operator after it has been applied, and the operator selection mechanism, that selects one operator based on some operators qualities. Two previously developed Adaptive Operator Selection methods are combined here: Compass evaluates the performance of operators by considering not only the fitness improvements from parent to offspring, but also the way they modify the diversity of the population, and their execution time; Dynamic Multi-Armed Bandit proposes a selection strategy based on the well-known UCB algorithm, achieving a compromise between exploitation and exploration, while nevertheless quickly adapting to changes. Tests with the proposed method, called ExCoDyMAB, are carried out using several hard instances of the Satisfiability problem (SAT). Results show the good synergetic effect of combining both approaches. I.

Population-Based Algorithm Portfolios for Numerical Optimization

by Fei Peng, Ke Tang, Guoliang Chen, Xin Yao - IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION
"... In this paper, we consider the scenario that a population-based algorithm is applied to a numerical optimization problem and a solution needs to be presented within a given time budget. Although a wide range of population-based algorithms, such as evolutionary algorithms, particle swarm optimizers, ..."
Abstract - Cited by 9 (0 self) - Add to MetaCart
In this paper, we consider the scenario that a population-based algorithm is applied to a numerical optimization problem and a solution needs to be presented within a given time budget. Although a wide range of population-based algorithms, such as evolutionary algorithms, particle swarm optimizers, and differential evolution, have been developed and studied under this scenario, the performance of an algorithm may vary significantly from problem to problem. This implies that there is an inherent risk associated with the selection of algorithms. We propose that, instead of choosing an existing algorithm and investing the entire time budget in it, it would be less risky to distribute the time among multiple different algorithms. A new approach named population-based algorithm portfolio (PAP), which takes multiple algorithms as its constituent algorithms, is proposed based upon this idea. PAP runs each

Comparison-based adaptive strategy selection with bandits in differential evolution

by Raymond Ros, Marc Schoenauer, Raymond Ros, Marc Schoenauer, Comparison-based Adaptive, Raymond Ros, Marc Schoenauer - in PPSN’10 , 2010
"... HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte p ..."
Abstract - Cited by 6 (1 self) - Add to MetaCart
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et a ̀ la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
(Show Context)

A Self-Learning Particle Swarm Optimizer for Global Optimization Problems

by Changhe Li, Shengxiang Yang , Trung Thanh Nguyen , 2011
"... Particle swarm optimization (PSO) has been shown as an effective tool for solving global optimization problems. So far, most PSO algorithms use a single learning pattern for all particles, which means all particles in a swarm use the same strategy. This monotonic learning pattern may cause the lack ..."
Abstract - Cited by 6 (0 self) - Add to MetaCart
Particle swarm optimization (PSO) has been shown as an effective tool for solving global optimization problems. So far, most PSO algorithms use a single learning pattern for all particles, which means all particles in a swarm use the same strategy. This monotonic learning pattern may cause the lack of intelligence for a particular particle, which makes it unable to deal with different complex situations. This paper presents a novel algorithm, called self-learning particle swarm optimizer (SLPSO), for global optimization problems. In SLPSO, each particle has a set of four strategies to cope with different situations in the search space. The cooperation of the four strategies is implemented by an adaptive learning framework at the individual level, which can enable a particle to choose the optimal strategy according to its own local fitness landscape. The experimental study on a set of 45 test functions and two realworld problems show that SLPSO has a superior performance in comparison with several other peer algorithms.

Programming by Optimisation

by Holger H. Hoos , 2010
"... When creating software, developers often explore various ways of achieving certain tasks. Traditionally, these alternatives are eliminated or abandoned early in the process, based on the idea that the flexibility afforded by them would be difficult or impossible to exploit effectively by developers ..."
Abstract - Cited by 3 (3 self) - Add to MetaCart
When creating software, developers often explore various ways of achieving certain tasks. Traditionally, these alternatives are eliminated or abandoned early in the process, based on the idea that the flexibility afforded by them would be difficult or impossible to exploit effectively by developers or users. Here, we challenge this view and advocate an approach that encourages developers to not only avoid premature commitment to certain design choices, but to actively develop promising alternatives for parts of the design. In this paradigm, dubbed Programming by Optimisation (PbO), developers specify a potentially large design space of programs that accomplish a given tasks, from which then versions of the program optimised for various application contexts are obtained automatically; per-instance selectors and parallel portfolios of programs can be obtained from the same design space (i.e., from exactly the same sources). We describe a simple, generic programming language extension that supports the specification of such design spaces and discuss ways in which specific programs that perform well in a given use context can be obtained from these specifications by using relatively simple source code transformations and powerful design optimisation methods. Using PbO, human experts can focus on the creative task of thinking about possible mechanisms for solving given problems or subproblems, while the tedious and boring task of determining what works best in a given use context is performed automatically, substituting human labour by computation. We believe that PbO provides an attractive way of creating software whose performance can be effectively adapted to a wide range of use contexts; it also enables the principled empirical investigation of the impact of design choices on performance, of the interaction between design choices and of the suitability of design choices in specific use contexts. 1
(Show Context)