There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications. 1. Introduction Kaelbling, Littman, and Moore (1996) and more recently Sutton and Barto (1998) provide informative surveys of the field of reinforcement learning (RL). They characterize two classes of methods for reinforcement learning: methods that search the space of value fu...

Abstract not found

. Fitness landscapes are a valuable concept in evolutionary biology, combinatorial optimization, and the physics of disordered systems. A fitness landscape is a mapping from a configuration space that is equipped with some notion of adjacency, nearness, distance or accessibility, into the real numbers. Landscape theory has emerged as an attempt to devise suitable mathematical structures for describing the "static" properties of landscapes as well as their influence on the dynamics of adaptation. This chapter gives a brief overview on recent developments in this area, focusing on "geometrical" properties of landscapes. 1 Introduction The concept of a fitness landscape originated in theoretical biology more than seventy years ago [1]. It can be thought of as a kind of "potential function" underlying the dynamics of evolutionary optimization. Implicit in this idea is both a fitness function f that assigns a fitness value to every possible genotype (or organism), and the arrangement of t...

Abstract not found

Abstract not found

Any learning process can be viewed as a self-modification of the leaxnefs current knowledge tArough an. interaction with some information source. Such knowledge modification is guided by the learner's deshe to achieve a certain outcome, and can engage any kind of inference. The type of inference involved depends on he input information, the current (background) knowledge and the learneFs task ax hand. Based on such a view of learning, several fundamental concepts are analized and clarified, in paxticular, analytic and synthetic learning, derivm:ional and hypothetical explanation, constnictive induction, abduction, abstraction and deductive generalization. It is shown that inductive generalization and abduction can be viewed as two basic forms of general induction, and that abstraction and deductive generalization axe two related forms of constructive deduction. Using this conceptual framework, a methodology for multistrategy task-adaptive learning (MTL) is outlined, in which learning strategies axe combined dynamically, depending on the current learning situation. Speccally, an MTL learner anaLizes a "wiad" relationship among the input information, the background knowledge and the learning task, and on that basis determines which strategy, or. a combination thereof, is most appropriate at a given learning step. To implement the MTL methodology, a new knowledge representation is proposed, based on the parametric association rules (PARs). Basic ideas of MTL are illustrated by means of the well-known "cup" example, through which is shown how an MTL learner can employ, depending the above mad relationship, emprical learning, constructive inductive generalization, abduction, explanation-based learning and absuaction.

A new mathematical representation is proposed for the configuration space structure induced by recombination which we called "P-structure". It consists of a mapping of pairs of objects to the power set of all objects in the search space. The mapping assigns to each pair of parental "genotypes" the set of all recombinant genotypes obtainable from the parental ones. It is shown that this construction allows a Fourier-decomposition of fitness landscapes into a superposition of "elementary landscapes". This decomposition is analogous to the Fourier decomposition of fitness landscapes on mutation spaces. The elementary landscapes are obtained as eigenfunctions of a Laplacian operator defined for P-structures. For binary string recombination the elementary landscapes are exactly the p-spin functions (Walsh functions), i.e. the same as the elementary landscapes of the string point mutation spaces (i.e. the hypercube). This supports the notion of a strong homomorphisms between string mutation ...

Fitness landscapes have proven to be a valuable concept in evolutionary biology, combinatorial optimization, and the physics of disordered systems. A fitness landscape is a mapping from a configuration space into the real numbers. The configuration space is equipped with some notion of adjacency, nearness, distance or accessibility. Landscape theory has emerged as an attempt to devise suitable mathematical structures for describing the "static" properties of landscapes as well as their influence on the dynamics of adaptation. In this review we focus on the connections of landscape theory with algebraic combinatorics and random graph theory, where exact results are available.

Abstract. Information retrieval is the selection of documents that are potentially relevant to a user’s information need. Given the vast volume of data stored in modern information retrieval systems, searching the document database requires vast computational resources. To meet these computational demands, various researchers have developed parallel information retrieval systems. As efficient exploitation of parallelism demands fast access to the documents, data organization and placement significantly affect the total processing time. We describe and evaluate a data placement strategy for distributed memory, distributed 1/0 multicomputers. Initially, a formal description of the Multiprocessor Document Allocation Problem (MDAP) and a proof that MDAP is NP Complete are presented. A document allocation

INTRODUCTION Evolutionary change is caused by the spontaneously generated genetic variation and its subsequent fixation by drift and/or selection. Consequently, the main focus of evolutionary theory has been to understand the genetic structure and dynamics of populations, see e.g. [101]. In recent years, however, alternative approaches have gained increasing prominence in evolutionary theory. This development has been stimulated to some extent by the application of evolutionary models to designing evolutionary algorithms such as Genetic Al- Evolutionary Dynamics edited by J.P. Crutchfield and P. Schuster, 1999 2 Spectral Landscape Theory gorithms, Evolution Strategies, and Genetic Programming, as well as by the theory of Complex Adaptive Systems [69, 79, 38]. The generic structure of an evolutionary model is x 0 = S (x; w) ffi T (x;<F