Recently, there has been increased interest in evolutionary computation applied to changing optimization problems. This paper surveys a number of approaches that extend the evolutionary algorithm with implicit or explicit memory, suggests a new benchmark problem and examines under which circumstances a memory may be helpful. From these observations we derive a new way to explore the benefits of a memory while minimizing its negative side effects.

We discuss a case-based method of initializing genetic algorithms that are used to guide search in changing environments. This is incorporated in an anytime learning system. Anytime learning is a general approach to continuous learning in a changing environment. A genetic algorithm with a case-based component provides an appropriate search mechanism for anytime learning. When the genetic algorithm is restarted, strategies which were previously learned under similar environmental conditions are included in the initial population of the genetic algorithm. We have evaluated the system by comparing performance with and without the case-based component, and case-based initialization of the population results in a significantly improved performance. INTRODUCTION We discuss a case-based method of initializing genetic algorithms in changing environments. This work is part of an ongoing investigation of machine learning techniques for sequential decision problems. The SAMUEL learning system e...

If the optimization problem is dynamic, the goal is no longer to nd the extrema, but to track their progression through the space as closely as possible. This paper surveys a number of techniques that have been published in the literature in order to make evolutionary algorithms suitable for changing optimization problems.

Abstract—In recent years, interest in studying evolutionary algorithms (EAs) for dynamic optimization problems (DOPs) has grown due to its importance in real-world applications. Several approaches, such as the memory and multiple population schemes, have been developed for EAs to address dynamic problems. This paper investigates the application of the memory scheme for population-based incremental learning (PBIL) algorithms, a class of EAs, for DOPs. A PBIL-specific associative memory scheme, which stores best solutions as well as corresponding environmental information in the memory, is investigated to improve its adaptability in dynamic environments. In this paper, the interactions between the memory scheme and random immigrants, multipopulation, and restart schemes for PBILs in dynamic environments are investigated. In order to better test the performance of memory schemes for PBILs and other EAs in dynamic environments, this paper also proposes a dynamic environment generator that can systematically generate dynamic environments of different difficulty with respect to memory schemes. Using this generator, a series of dynamic environments are generated and experiments are carried out to compare the performance of investigated algorithms. The experimental results show that the proposed memory scheme is efficient for PBILs in dynamic environments and also indicate that different interactions exist between the memory scheme and random immigrants, multipopulation schemes for PBILs in different dynamic environments. Index Terms—Associative memory scheme, dynamic optimization problems (DOPs), immune system-based genetic algorithm (ISGA), memory-enhanced genetic algorithm, multipopulation scheme, population-based incremental learning (PBIL), random immigrants.

We combine genetic algorithms and case-based reasoning principles to find optimally directed solutions to open shop scheduling and open shop re-scheduling problems. Appropriate solutions to open shop scheduling problems are injected into the genetic algorithm's population to speed up and augment genetic search on a related open shop re-scheduling problem. Preliminary results indicate that the combined genetic algorithm -- case-based reasoning system quickly finds better solutions than the genetic algorithm alone. Keywords: Genetic Algorithms, Case-Based Reasoning, Open Shop Scheduling. 1 Introduction Genetic algorithms (GAs) are stochastic, parallel search algorithms based on the mechanics of natural selection, the process of evolution [3, 2]. GAs were designed to efficiently search large, non-linear search spaces where expert knowledge is lacking or difficult to encode and where traditional optimization techniques fail. They are flexible and robust, exhibiting the adaptiveness and g...

Evolutionary learning methods have been found to be useful in several areas in the development of intelligent robots. In the approach described here, evolutionary algorithms are used to explore alternative robot behaviors within a simulation model as a way of reducing the overall knowledge engineering effort. This paper presents some initial results of applying the SAMUEL genetic learning system to a collision avoidance and navigation task for mobile robots. 1 INTRODUCTION This is a progress report on our efforts to design intelligent robots for complex environments. The sort of applications we have in mind include sentry robots, autonomous delivery vehicles, undersea surveillance vehicles, and automated warehouse robots. In particular, we are investigating issues relating to machine learning, using multiple mobile robots to perform tasks such as playing hide-and-seek, tag, or competing to find hidden objects. Given the wide range of tasks in the area of robotics and learning, it may...

ID A318 Abstract This paper presents a new approach to genetic algorithm based machine learning. We use genetic algorithms augmented with a case-based memory of past problem solving attempts to obtain better performance over time on sets of similar problems. Rather than starting anew on each problem, we periodically inject a genetic algorithm's population with appropriate intermediate solutions to similar, previously solved problems. Using simple syntactic similarity measures, our experimental results on a configuration design problem, and on a path planning problem demonstrate the robustness of our approach. These results show that our system learns to take less time to provide a solution to a new problem as it gains experience from solving other similar problems -- exactly what we want from a learning system. Content Areas: Machine Learning and Discovery, Techniques or Algorithms, evolutionary computations, Machine Learning and Discovery, Techniques or Algorithms, Empirical evaluat...

For many years, researchers in the field of mobile robotics have been investigating the use of genetic and evolutionary computation (GEC) to aid the development of mobile robot controllers. Alongside the fundamental choices of the GEC mechanism and its operators, which apply to both simulated and physical evolutionary robotics, other issues have emerged which are specific to the application of GEC to physical mobile robotics. This paper presents a survey of recent methods in GEC-developed mobile robot controllers, focusing on those methods that include a physical robot at some point in the learning loop. It simultaneously relates each of these methods to a framework of two orthogonal issues: the use of a simulated and/or a physical robot, and the use of finite, training phase evolution prior to a task and/or lifelong adaptation by evolution during a task. A list of evaluation criteria are presented and each of the surveyed methods are compared to them. Analyses of the framework and evaluation criteria suggest several possibilities; however, there appear to be particular advantages in combining simulated, training phase evolution (TPE) with lifelong adaptation by evolution (LAE) on a physical robot.

Abstract — In this paper we present new methods for the automated learning of heuristics in knowledge-lean applications and for finding heuristics that can be generalized to unlearned domains. These applications lack domain knowledge for credit assignment; hence, operators for composing new heuristics are generally modelfree, domain independent, and syntactic in nature. The operators we have used are genetics-based; examples of which include mutation and cross-over. Learning is based on a generate-and-test paradigm that maintains a pool of competing heuristics, tests them to a limited extent, creates new ones from those that perform well in the past, and prunes poor ones from the pool. We hav e studied three important issues in learning better heuristics: (a) anomalies in performance evaluation, (b) rational scheduling of limited computational resources in testing candidate heuristics in single-objective as well as multiobjective learning, and (c) finding heuristics that can be generalized to unlearned domains. We show experimental results in learning better heuristics for (a) process placement for distributed-memory multicomputers, (b) node decomposition in a branch-and-bound search, (c) generation of test patterns in VLSI circuit testing, and (d) VLSI cell placement and routing. Index Terms — Branch-and-bound search, generalization, genetics-based learning, heuristics, knowledge-lean

Real-world problems are often nonstationary and can cause cyclic repetitive patterns in the search landscape. For this class of problems we introduce a new GA with dynamic explicit memory which showed superior performance compared to a classic GA and a previously introduced memorybased GA for two dynamic benchmark problems.