Results 1  10
of
87
A Survey of Optimization by Building and Using Probabilistic Models
 COMPUTATIONAL OPTIMIZATION AND APPLICATIONS
, 1999
"... This paper summarizes the research on populationbased probabilistic search algorithms based on modeling promising solutions by estimating their probability distribution and using the constructed model to guide the further exploration of the search space. It settles the algorithms in the field of ge ..."
Abstract

Cited by 278 (82 self)
 Add to MetaCart
This paper summarizes the research on populationbased probabilistic search algorithms based on modeling promising solutions by estimating their probability distribution and using the constructed model to guide the further exploration of the search space. It settles the algorithms in the field of genetic and evolutionary computation where they have been originated. All methods are classified into a few classes according to the complexity of the class of models they use. Algorithms from each of these classes are briefly described and their strengths and weaknesses are discussed.
Hierarchical Bayesian Optimization Algorithm = Bayesian Optimization Algorithm + Niching + Local Structures
, 2001
"... The paper describes the hierarchical Bayesian optimization algorithm which combines the Bayesian optimization algorithm, local structures in Bayesian networks, and a powerful niching technique. The proposed algorithm is able to solve hierarchical traps and other difficult problems very efficiently. ..."
Abstract

Cited by 255 (63 self)
 Add to MetaCart
The paper describes the hierarchical Bayesian optimization algorithm which combines the Bayesian optimization algorithm, local structures in Bayesian networks, and a powerful niching technique. The proposed algorithm is able to solve hierarchical traps and other difficult problems very efficiently.
The Equation for the Response to Selection and Its Use for Prediction
, 1997
"... The Breeder Genetic Algorithm (BGA) was designed according to the theories and methods used in the science of livestock breeding. The prediction of a breeding experiment is based on the response to selection (RS) equation. This equation relates the change in a population 's fitness to the standard d ..."
Abstract

Cited by 103 (15 self)
 Add to MetaCart
The Breeder Genetic Algorithm (BGA) was designed according to the theories and methods used in the science of livestock breeding. The prediction of a breeding experiment is based on the response to selection (RS) equation. This equation relates the change in a population 's fitness to the standard deviation of its fitness, as well as to the parameters selection intensity and realized heritability. In this paper the exact RS equation is derived for proportionate selection given an infinite population in linkage equilibrium. In linkage equilibrium the genotype frequencies are the product of the univariate marginal frequencies. The equation contains Fisher's fundamental theorem of natural selection as an approximation. The theorem shows that the response is approximately equal to the quotient of a quantity called additive genetic variance, VA , and the average fitness. We compare Mendelian twoparent recombination with genepool recombination, which belongs to a special class of genetic ...
The Bivariate Marginal Distribution Algorithm
, 1999
"... The paper deals with the Bivariate Marginal Distribution Algorithm (BMDA). BMDA is an extension of the Univariate Marginal Distribution Algorithm (UMDA). It uses the pair gene dependencies in order to improve algorithms that use simple univariate marginal distributions. BMDA is a special case of the ..."
Abstract

Cited by 90 (19 self)
 Add to MetaCart
The paper deals with the Bivariate Marginal Distribution Algorithm (BMDA). BMDA is an extension of the Univariate Marginal Distribution Algorithm (UMDA). It uses the pair gene dependencies in order to improve algorithms that use simple univariate marginal distributions. BMDA is a special case of the Factorization Distribution Algorithm, but without any problem specic knowledge in the initial stage. The dependencies are being discovered during the optimization process itself. In this paper BMDA is described in detail. BMDA is compared to dierent algorithms including the simple genetic algorithm with dierent crossover methods and UMDA. For some tness functions the relation between problem size and the number of tness evaluations until convergence is shown. 1. Introduction Genetic algorithms work with populations of strings of xed length. In this paper binary strings will be considered. From current population better strings are selected at the expense of worse ones. New strings ar...
Schemata, Distributions and Graphical Models in Evolutionary Optimization
 Journal of Heuristics
, 1999
"... In this paper the optimization of additively decomposed discrete functions is investigated. For these functions genetic algorithms have exhibited a poor performance. First the schema theory of genetic algorithms is reformulated in probability theory terms. A schema denes the structure of a marginal ..."
Abstract

Cited by 89 (8 self)
 Add to MetaCart
In this paper the optimization of additively decomposed discrete functions is investigated. For these functions genetic algorithms have exhibited a poor performance. First the schema theory of genetic algorithms is reformulated in probability theory terms. A schema denes the structure of a marginal distribution. Then the conceptual algorithm BEDA is introduced. BEDA uses a Boltzmann distribution to generate search points. From BEDA a new algorithm, FDA, is derived. FDA uses a factorization of the distribution. The factorization captures the structure of the given function. The factorization problem is closely connected to the theory of conditional independence graphs. For the test functions considered, the performance of FDA in number of generations till convergence is similar to that of a genetic algorithm for the OneMax function. This result is theoretically explained.
Linkage Problem, Distribution Estimation, and Bayesian Networks
, 2000
"... This paper proposes an algorithm that uses an estimation of the joint distribution of promising solutions in order to generate new candidate solutions. The algorithm is settled into the context of genetic and evolutionary computation and the algorithms based on the estimation of distributions. Th ..."
Abstract

Cited by 88 (18 self)
 Add to MetaCart
This paper proposes an algorithm that uses an estimation of the joint distribution of promising solutions in order to generate new candidate solutions. The algorithm is settled into the context of genetic and evolutionary computation and the algorithms based on the estimation of distributions. The proposed algorithm is called the Bayesian Optimization Algorithm (BOA). To estimate the distribution of promising solutions, the techniques for modeling multivariate data by Bayesian networks are used. TheBOA identifies, reproduces, and mixes building blocks up to a specified order. It is independent of the ordering of the variables in strings representing the solutions. Moreover, prior information about the problem can be incorporated into the algorithm, but it is not essential. First experiments were done with additively decomposable problems with both nonoverlapping as well as overlapping building blocks. The proposed algorithm is able to solve all but one of the tested problems in linear or close to linear time with respect to the problem size. Except for the maximal order of interactions to be covered, the algorithm does not use any prior knowledge about the problem. The BOA represents a step toward alleviating the problem of identifying and mixing building blocks correctly to obtain good solutions for problems with very limited domain information.
Escaping Hierarchical Traps with Competent Genetic Algorithms
 Proceedings of the Genetic and Evolutionary Computation Conference (GECCO2001
, 2001
"... To solve hierarchical problems, one must be able to learn the linkage, represent partial solutions efficiently, and assure effective niching. We propose the hierarchical ... ..."
Abstract

Cited by 85 (46 self)
 Add to MetaCart
To solve hierarchical problems, one must be able to learn the linkage, represent partial solutions efficiently, and assure effective niching. We propose the hierarchical ...
FDA  A scalable evolutionary algorithm for the optimization of additively decomposed functions
, 1999
"... FDA  the Factorized Distribution Algorithm  is an evolutionary algorithm which combines mutation and recombination by using a distribution instead. The distribution is estimated from a set of selected points. In general a discrete distribution defined for n binary variables has 2 n parameters. T ..."
Abstract

Cited by 62 (7 self)
 Add to MetaCart
FDA  the Factorized Distribution Algorithm  is an evolutionary algorithm which combines mutation and recombination by using a distribution instead. The distribution is estimated from a set of selected points. In general a discrete distribution defined for n binary variables has 2 n parameters. Therefore it is too expensive to compute. For additively decomposed discrete functions (ADFs) there exist algorithms which factor the distribution into conditional and marginal distributions. This factorization is used by FDA. The scaling of FDA is investigated theoretically and numerically. The scaling depends on the ADF structure and the specific assignment of function values. Difficult functions on a chain or a tree structure are solved in about O(n p n) operations. More standard genetic algorithms are not able to optimize these functions. FDA is not restricted to exact factorizations. It also works for approximate factorizations as is shown for a circle and a grid structure. By using results from Bayes networks, FDA is extended to LFDA. LFDA computes an approximate factorization using only the data, not the ADF structure. The scaling of LFDA is compared to the scaling of FDA. Keywords Genetic algorithms, Boltzmann distribution, simulated annealing, Bayes network, learning of Bayes networks, convergence, factorization of distributions. 1
Extending PopulationBased Incremental Learning to Continuous Search Spaces
, 1998
"... . An alternative to Darwinianlike artificial evolution is offered by PopulationBased Incremental Learning (PBIL): this algorithm memorizes the best past individuals and uses this memory as a distribution, to generate the next population from scratch. This paper extends PBIL from boolean to con ..."
Abstract

Cited by 59 (3 self)
 Add to MetaCart
. An alternative to Darwinianlike artificial evolution is offered by PopulationBased Incremental Learning (PBIL): this algorithm memorizes the best past individuals and uses this memory as a distribution, to generate the next population from scratch. This paper extends PBIL from boolean to continuous search spaces. A Gaussian model is used for the distribution of the population. The center of this model is constructed as in boolean PBIL. Several ways of defining and adjusting the variance of the model are investigated. The approach is validated on several largesized problems. 1 Introduction Evolutionary algorithms (EAs) [13, 6, 5] are mostly used to find the optima of some fitness function F defined on a search space\Omega . F :\Omega ! IR From a machine learning (ML) perspective [9], evolution is similar to learning by query: Learning by query starts with a void hypothesis and gradually refines the current hypothesis through asking questions to some oracle. In ML, ...
Using Learning for Approximation in Stochastic Processes
 In Proceedings of the International Conference on Machine Learning (ICML
, 1998
"... To monitor or control a stochastic dynamic system, we need to reason about its current state. Exact inference for this task requires that we maintain a complete joint probability distribution over the possible states, an impossible requirement for most processes. Stochastic simulation algorithms pro ..."
Abstract

Cited by 55 (3 self)
 Add to MetaCart
To monitor or control a stochastic dynamic system, we need to reason about its current state. Exact inference for this task requires that we maintain a complete joint probability distribution over the possible states, an impossible requirement for most processes. Stochastic simulation algorithms provide an alternative solution by approximating the distribution at time t via a (relatively small) set of samples. The time t samples are used as the basis for generating the samples at time t + 1. However, since only existing samples are used as the basis for the next sampling phase, new parts of the space are never explored. We propose an approach whereby we try to generalize from the time t samples to unsampled regions of the state space. Thus, these samples are used as data for learning a distribution over the states at time t, which is then used to generate the time t+1 samples. We examine different representations for a distribution, including density trees, Bayesian networks, and tree...