Assume a coevolutionary algorithm capable of storing and utilizing all phenotypes discovered during its operation, for as long as it operates on a problem; that is, assume an algorithm with a monotonically increasing knowledge of the search space. We ask: If such an algorithm were to periodically report, over the course of its operation, the best solution found so far, would the quality of the solution reported by the algorithm improve monotonically over time? To answer this question, we construct a simple preference relation to reason about the goodness of di#erent individual and composite phenotypic behaviors. We then show that whether the solutions reported by the coevolutionary algorithm improve monotonically with respect to this preference relation depends upon the solution concept implemented by the algorithm. We show that the solution concept implemented by the conventional coevolutionary algorithm does not guarantee monotonic improvement; in contrast, the game-theoretic solution concept of Nash equilibrium does guarantee monotonic improvement. Thus, this paper considers 1) whether global and objective metrics of goodness can be applied to coevolutionary problem domains (possibly with open-ended search spaces), and 2) whether coevolutionary algorithms can, in principle, optimize with respect to such metrics and find solutions to games of strategy.

Adaptive representations allow evolution to explore the space of phenotypes by choosing the most suitable set of genotypic parameters. Although such an approach is believed to be efficient on complex problems, few empirical studies have been conducted in such domains. In this paper, three neural network representations, a direct encoding, a complexifying encoding, and an implicit encoding capable of adapting the genotype-phenotype mapping are compared on Nothello, a complex game playing domain from the AAAI General Game Playing Competition. Implicit encoding makes the search more efficient and uses several times fewer parameters. Random mutation leads to highly structured phenotypic variation that is acquired during the course of evolution rather than built into the representation itself. Thus, adaptive representations learn to become evolvable, and furthermore do so in a way that makes search efficient on difficult coevolutionary problems.

Coevolution has often been based on averaged outcomes, resulting in unstable evaluation. Several theoretical approaches have used archives to provide stable evaluation. However, the number of tests required by some of these approaches can be prohibitive of practical applications. Recent work has shown the existence of a set of underlying objectives which compress evaluation information into a potentially small set of dimensions. We consider whether these underlying objectives can be approximated online, and used for evaluation in a coevolution algorithm. The Dimension Extracting Coevolutionary Algorithm (DECA) is compared to several recent reliable coevolution algorithms on a Numbers game problem, and found to perform efficiently. Application to the more realistic Tartarus problem is shown to be feasible. Implications for current coevolution research are discussed.

Co-evolutionary learning involves a training process where training samples are instances of solutions that interact strategically to guide the evolutionary (learning) process. One main research issue is with the generalization performance, i.e., the search for solutions (e.g., input-output mappings) that best predict the required output for any new input that has not been seen during the evolutionary process. However, there is currently no such framework for determining the generalization performance in co-evolutionary learning even though the notion of generalization is well-understood in machine learning. In this paper, we introduce a theoretical framework to address this research issue. We present the framework in terms of game-playing although our results are more general. Here, a strategy’s generalization performance is its average performance against all test strategies. Given that the true value may not be determined by solving analytically a closed-form formula and is computationally prohibitive, we propose an estimation procedure that computes the average performance against a small sample of random test strategies instead. We perform a mathematical analysis to provide a statistical claim on the accuracy of our estimation procedure, which can be further improved by performing a second estimation on the variance of the random variable. For game-playing, it is well-known that one is more interested in the generalization

To my parents, who gifted me with curiosity and the stubbornness to follow where it leads. Acknowledgments It almost goes without saying that a piece of work this size could not have been finished without the help of innumerable people. I wanted to extend my gratitude to those who had the greatest impact on my thinking and writing over the past eight years. First, to my advisor Jordan Pollack. Jordan’s visionary quest for “mindless intelligence, ” a search for artificial intelligence without modeling the human brain or mind, kept me engaged with a set of ideas and a set of people I would otherwise never have encountered. Jordan’s enthusiasm and deep understand-ing, not to mention his uncanny ability to direct attention to fertile areas of research, are truly contagious and inspiring. To Timothy Hickey and Marc Toussaint, who sat on my dissertation ex-amining committee. Tim’s careful scrutiny uncovered what would have been an embarrassing error. Marc, who has been interested in my work for several years, painstakingly scoured the entirety of this dissertation, offering a myriad small and large improvements along with suggestive interpretations of these ideas from perspectives I had not considered. v

Abstract. Coevolution often gives rise to counter-intuitive dynamics that defy our expectations. Here we suggest that much of the confusion surrounding coevolution results from imprecise notions of superiority and progress. In particular, we note that in the literature, three distinct notions of progress are implicitly lumped together: local progress (superior performance against current opponents), historical progress (superior performance against previous opponents) and global progress (superior performance against the entire opponent space). As a result, valid conditions for one type of progress are unduly assumed to lead to another. In particular, the confusion between historical and global progress is a case of a common error, namely using the training set as a test set. This error is prevalent among standard methods for coevolutionary analysis (CIAO, Master Tournament, Dominance Tournament, etc.) By clearly defining and distinguishing between different types of progress, we identify limitations with existing techniques and algorithms, address them, and generally facilitate discussion and understanding of coevolution. We conclude that the concepts proposed in this paper correspond to important aspects of the coevolutionary process. 1

We mowen nat, although we hadden it sworn, It overtake, it slit awey so faste. It wole us maken beggers atte laste! Evolution is notorious for its creative power, but also for giving rise to complex, unpredictable dynamics. As a result, practitioners of artificial evolution have encountered difficulties in predicting, analysing, or even understanding the outcome of their experiments. In particular, the concept of evolutionary “progress ” (whether in the sense of performance increase or complexity growth) has given rise to much debate and confusion. After a careful description of the mechanisms of evolution and natural selection, we provide usable concepts of performance and progress in coevolution. In particular, we introduce a distinction between three types of progress: local, historical, and global, which we suggest underlies much of the confusion that surrounds coevolutionary dynamics. Similarly, we provide a comprehensive answer to the question of whether an “arrow of complexity ” exists in evolution. We introduce several methods to detect and analyse performance and progress in coevolutionary experiments. We propose a statistical

Abstract — Xpilot, a network game where agents engage in space combat, has been shown to be a good test bed for controller learning systems. In this paper, we introduce the Core, an Xpilot learning environment where a population of learning agents interact locally through tournament selection, crossover, and mutation to produce offspring in the evolution of controllers. The system does not require the researcher to develop a fitness function or suitable agents to engage with the evolving agent. Instead, it employs a form of co-evolution where the environment, made up of the population of agents, evolves to continually challenge individual agents evolving within it. Tests show its successful use in evolving controllers for combat agents in Xpilot. L I.