## Algorithms (2006)

### BibTeX

@MISC{Yu06algorithms,

author = {Tian-li Yu and Kumara Sastry and David E. Goldberg and Martin Pelikan and Tian-li Yu and Kumara Sastry and David E. Goldberg and Martin Pelikan},

title = {Algorithms},

year = {2006}

}

### OpenURL

### Abstract

This paper presents a population-sizing model for the entropy-based model building in genetic algorithms. Specifically, the population size required for building an accurate model is investigated. The effect of the selection pressure on population sizing is also incorporated. The proposed model indicates that the population size required for building an accurate model scales as Θ(mlog m), where m is the number of substructures and proportional to the problem size. Experiments are conducted to verify the derivations, and the results agree with the proposed model. 1

### Citations

7562 |
A mathematical theory of communication
- Shannon
- 1948
(Show Context)
Citation Context ... failure. 3 Distribution of the Entropy Measurement Many different metrics have been proposed to detect the linkage for model building. One of the most commonly-used measurement is Shannon’s entropy (=-=Shannon, 1948-=-) . Typical examples for such entropy-based model-building GAs include FDA (Mühlenbien, 1999), eCGA (Harik, 1999), EBNA (Etxeberria & Larrañaga, 1999), BOA (Pelikan, Goldberg, & Cantú-Paz, 1999), and ... |

2615 |
Stegun I., Handbook of mathematical functions
- Abramowitz
- 1970
(Show Context)
Citation Context ...where Φ is the cumulati‘ve stand normal distribution. Define z = E[Z] √ V ar[Z] = d 2 k−1√ 2m · σBB + O(m −1.5 ). (11) For large m, z is small, and Φ(z) can be approximated by 1 z 2 + √ − O(z 2π 3 ) (=-=Abramowitz & Stegun, 1970-=-). Φ(z) = 1 2 + d 2 k√ πm · σBB ± O(m −1.5 ). (12) Define p0 and p1 as the proportions of H0 and H1 before selection (p0 = p1 = 1 2 ), and p′ 0 and 4p ′ 1 are that after selection. p ′ 0 = p 2 0 + 2p... |

932 | JH: Adaptation in Natural and Artificial Systems Ann Arbor - Holland - 1975 |

279 | Hierarchical Bayesian Optimization Algorithm. Toward a New Generation of Evolutionary Algorithms
- Pelikan
- 2005
(Show Context)
Citation Context ...urement is Shannon’s entropy (Shannon, 1948) . Typical examples for such entropy-based model-building GAs include FDA (Mühlenbien, 1999), eCGA (Harik, 1999), EBNA (Etxeberria & Larrañaga, 1999), BOA (=-=Pelikan, Goldberg, & Cantú-Paz, 1999-=-), and DSMGA (Yu & Goldberg, 2006), and the work of Wright et al. (Wright, Poli, Stephens, Landgon, & Pulavarty, 2004). The loss of the entropy by linking two decision variables together is essentiall... |

250 | Genetic Algorithms, Noise and the Sizing of Populations
- Goldberg, Kalyanmoy, et al.
- 1992
(Show Context)
Citation Context ...population of size larger than required leads to wastage of computational resources. Therefore, facetwise models, such as initial-supply (Goldberg, Sastry, & Latoza, 2001) and decision-making models (=-=Goldberg, Deb, & Clark, 1992-=-a; Harik, Cantú-Paz, Goldberg, & Miller, 1997), have been developed to model different bounds on population sizing required for genetic algorithm (GA) success. The issue of population sizing is equall... |

220 | The Gambler’s ruin problem, genetic algorithms, and the sizing of populations
- Harik, Cant˘u-Paz, et al.
- 1999
(Show Context)
Citation Context ...required leads to wastage of computational resources. Therefore, facetwise models, such as initial-supply (Goldberg, Sastry, & Latoza, 2001) and decision-making models (Goldberg, Deb, & Clark, 1992a; =-=Harik, Cantú-Paz, Goldberg, & Miller, 1997-=-), have been developed to model different bounds on population sizing required for genetic algorithm (GA) success. The issue of population sizing is equally critical, if not more, in model-building GA... |

201 | Linkage learning via probabilistic modeling in the ECGA
- Harik
- 1999
(Show Context)
Citation Context ...e for model building. One of the most commonly-used measurement is Shannon’s entropy (Shannon, 1948) . Typical examples for such entropy-based model-building GAs include FDA (Mühlenbien, 1999), eCGA (=-=Harik, 1999-=-), EBNA (Etxeberria & Larrañaga, 1999), BOA (Pelikan, Goldberg, & Cantú-Paz, 1999), and DSMGA (Yu & Goldberg, 2006), and the work of Wright et al. (Wright, Poli, Stephens, Landgon, & Pulavarty, 2004).... |

160 |
Sizing Populations for Serial and Parallel Genetic Algorithms
- Goldberg
- 1989
(Show Context)
Citation Context ...e presence of at least on copy of all raw schemata is modeled. Holland (1975) estimated the number of BBs that receive at least a specified number of trials using Poisson distribution. A later study (=-=Goldberg, 1989-=-) calculated the same quantity more exactly using binomial distribution and studied their effects on population sizing in serial and parallel computation. Reeves (1993) proposed a population sizing mo... |

134 |
Simple genetic algorithms and the minimal, deceptive problem
- Goldberg
- 1987
(Show Context)
Citation Context ... 2E[I] n E[ ˆ X1,n] ≥ △2 m 2 ln2 V ar[ ˆ X1,n] ≥ △2m . (21) n ln2 Now we empirically verify the derived means and variances of ˆ X0,n and ˆ X1,n. The test is done by applying a GA on the (m, k)-trap (=-=Goldberg, 1987-=-) with k = 4 and d = 0.25. After the binary tournament selection, we compute the mutual information of the pairs of independent genes and the pairs of dependent genes. All results are averaged over 10... |

75 | A mathematical analysis of tournament selection - Blickle, Thiele - 1995 |

67 |
Estimation of distribution algorithms
- Larranaga, Lozano
- 2002
(Show Context)
Citation Context ...n sizing required for genetic algorithm (GA) success. The issue of population sizing is equally critical, if not more, in model-building GAs, such as the estimation of distribution algorithms (EDAs) (=-=Larrañaga & Lozano, 2002-=-), which build linkage models for the given problems and utilize the knowledge gained from linkage models to efficiently recombine new solution candidates. For model-building GAs, the population shoul... |

51 |
Global optimization using Bayesian networks
- Etxeberria, Larrañaga
- 1999
(Show Context)
Citation Context .... One of the most commonly-used measurement is Shannon’s entropy (Shannon, 1948) . Typical examples for such entropy-based model-building GAs include FDA (Mühlenbien, 1999), eCGA (Harik, 1999), EBNA (=-=Etxeberria & Larrañaga, 1999-=-), BOA (Pelikan, Goldberg, & Cantú-Paz, 1999), and DSMGA (Yu & Goldberg, 2006), and the work of Wright et al. (Wright, Poli, Stephens, Landgon, & Pulavarty, 2004). The loss of the entropy by linking t... |

46 |
Scalability of the Bayesian optimization algorithm
- Pelikan, Sastry, et al.
(Show Context)
Citation Context ...n analyzed for estimation of distribution algorithms (EDAs) in general, and Bayesian optimization algorithm and extended compact genetic algorithm in particular (Pelikan, Goldberg, & Cantú-Paz, 2000; =-=Pelikan, Sastry, & Goldberg, 2003-=-; Sastry & Goldberg, 2000; Sastry & Goldberg, 2004). The population-sizing model which incorporates the effect of model-building and its accuracy on the population sizing of the GA, and predicts the p... |

40 | Bayesian optimization algorithm, population sizing, and time to convergence
- Pelikan, Goldberg, et al.
- 2000
(Show Context)
Citation Context ...making on the population size have been analyzed for estimation of distribution algorithms (EDAs) in general, and Bayesian optimization algorithm and extended compact genetic algorithm in particular (=-=Pelikan, Goldberg, & Cantú-Paz, 2000-=-; Pelikan, Sastry, & Goldberg, 2003; Sastry & Goldberg, 2000; Sastry & Goldberg, 2004). The population-sizing model which incorporates the effect of model-building and its accuracy on the population s... |

40 | Using genetic algorithm with small population - Reeves - 1993 |

31 | Designing competent mutation operators via probabilistic model building of neighborhoods
- Sastry, Goldberg
- 2004
(Show Context)
Citation Context ...roblem size of the order between 1.05 and 2.1. Θ(m 1.05 ) ≤ n ≤ Θ(m 2.1 ). (1) These bounds also apply to many other model-building GAs, and empirical results show that n roughly scales as Θ(m 1.4 ) (=-=Sastry & Goldberg, 2004-=-). However, we need a more refined model to 1explain the empirical results and better understand population sizing. In addition, empirical results also indicate that the selection pressure affects th... |

27 | On the supply of building blocks - Goldberg, Sastry, et al. - 2001 |

21 | Identifying linkage groups by nonlinearity/non-monotonicity detection
- Munetomo, Goldberg
- 1999
(Show Context)
Citation Context ...r the model builder. Finally, although the proposed model is based on the entropy measurement, we believe that a similar procedure should be applied to some other measurements including nonlinearity (=-=Munetomo & Goldberg, 1999-=-) and simultaneity (Aporntewan & Chongstitvatana, 2003). Acknowledgements This work was sponsored by the Air Force Office of Scientific Research, Air Force Materiel Command, USAF, under grant FA9550-0... |

14 | Distribution of mutual information from complete and incomplete data
- Hutter, Zaffalon
(Show Context)
Citation Context ... that E[ ˆ X0,n=∞] = E[X0] = 0 and E[ ˆ X1,n=∞] = E[X1]. For a finite number of samples, the mean and variance of the mutual information of a pair of independent random variables is known as follows (=-=Hutter & Zaffalon, 2004-=-). E[ ˆ X0,n] = 1 1 + O( 2n n2) V ar[ ˆ X0,n] = 1 1 + O( 2n2 n3). (5) To calculate E[ ˆ X1,n], first we need to estimate E[X1]. The derivation is based on Holland’s Royal road function, which servers ... |

14 | The posterior probability of Bayes nets with strong dependences - Kleiter - 1999 |

10 | Generalized convergence models for tournament- and (µ, λ)-selection - Bäck - 1995 |

9 | An estimation of distribution algorithm based on maximum entropy
- Wright, Poli, et al.
- 2004
(Show Context)
Citation Context ...As include FDA (Mühlenbien, 1999), eCGA (Harik, 1999), EBNA (Etxeberria & Larrañaga, 1999), BOA (Pelikan, Goldberg, & Cantú-Paz, 1999), and DSMGA (Yu & Goldberg, 2006), and the work of Wright et al. (=-=Wright, Poli, Stephens, Landgon, & Pulavarty, 2004-=-). The loss of the entropy by linking two decision variables together is essentially the mutual information: I(X; Y ) = H(X) + H(Y ) − H(X; Y ) (Cover & Thomas, 1991) . In entropy-based model building... |

6 | Building-block identification by simultaneity matrix
- Aporntewan, Chongstitvatana
- 2003
(Show Context)
Citation Context ...oposed model is based on the entropy measurement, we believe that a similar procedure should be applied to some other measurements including nonlinearity (Munetomo & Goldberg, 1999) and simultaneity (=-=Aporntewan & Chongstitvatana, 2003-=-). Acknowledgements This work was sponsored by the Air Force Office of Scientific Research, Air Force Materiel Command, USAF, under grant FA9550-06-1-0096, the National Science Foundation under NSF CA... |

6 |
An Introduction to Probability Theory, Volume 2
- Feller
- 1971
(Show Context)
Citation Context ...ata (6) H0 = 0∗ ∗ ∗ · · · ∗ } {{ } k−1 H1 = 1∗} ∗ ∗ {{· · · ∗} . (7) k−1 Let F0 and F1 be the random variables of the fitness values of H0 and H1, respectively. According to the central limit theory (=-=Feller, 1966-=-) , the distributions of F0 and F1 are close to a normal distribution. The variances of F0 and F1 are the same, which is defined as σ2 F . By treating other (m − 1) BBs as external noises, we can expr... |

6 |
On extended compact genetic algorithm. Late Breaking Paper
- Sastry, Goldberg
- 2000
(Show Context)
Citation Context ...bution algorithms (EDAs) in general, and Bayesian optimization algorithm and extended compact genetic algorithm in particular (Pelikan, Goldberg, & Cantú-Paz, 2000; Pelikan, Sastry, & Goldberg, 2003; =-=Sastry & Goldberg, 2000-=-; Sastry & Goldberg, 2004). The population-sizing model which incorporates the effect of model-building and its accuracy on the population sizing of the GA, and predicts the population size required t... |

3 |
Elements of information theory (pp
- Cover, Thomas
- 1991
(Show Context)
Citation Context ...al. (Wright, Poli, Stephens, Landgon, & Pulavarty, 2004). The loss of the entropy by linking two decision variables together is essentially the mutual information: I(X; Y ) = H(X) + H(Y ) − H(X; Y ) (=-=Cover & Thomas, 1991-=-) . In entropy-based model building, the measurement I(X; Y ) needs to be significant enough for accurately detecting the existence of the linkage between X and Y . In this paper, we focus on model bu... |

2 |
Fda: A scaleable evolutionary algorithm for the optimisation of additively decomposable problems
- Mühlenbien
- 1999
(Show Context)
Citation Context ...osed to detect the linkage for model building. One of the most commonly-used measurement is Shannon’s entropy (Shannon, 1948) . Typical examples for such entropy-based model-building GAs include FDA (=-=Mühlenbien, 1999-=-), eCGA (Harik, 1999), EBNA (Etxeberria & Larrañaga, 1999), BOA (Pelikan, Goldberg, & Cantú-Paz, 1999), and DSMGA (Yu & Goldberg, 2006), and the work of Wright et al. (Wright, Poli, Stephens, Landgon,... |