21 We analyze the problem of learning about other agents in a class of dynamic multiagent systems, where performance of 22 the primary agent depends on behavior of the others. We consider an online version of the problem, where agents must learn 23 models of the others in the course of continual interactions. Various levels of recursive models are implemented in a 24 simulated double auction market. Our experiments show learning agents on average outperform non-learning agents who do 25 not use information about others. Among learning agents, those with minimum recursion assumption generally perform 26 better than the agents with more complicated, though often wrong assumptions. 2001 Published by Elsevier Science B.V. 27 Keywords: Multiagent learning; Multiagent systems; Computational market 28 29 1.

A new reinforcement learning (RL) methodology is proposed to design multi-agent systems. In the realistic setting of situated agents with local perception, the task of automatically building a coordinated system is of crucial importance. We use simple reactive agents which learn their own behavior in a decentralized way. To cope with the difficulties inherent to RL used in that framework, we have developed an incremental learning algorithm where agents face more and more complex tasks. We illustrate this general framework on a computer experiment where agents have to coordinate to reach a global goal. 1

. Learning in a multiagent environment is complicated by the fact that as other agents learn, the environment effectively changes. Moreover, other agents' actions are often not directly observable, and the actions taken by the learning agent can strongly bias which range of behaviors are encountered. We define the concept of a conjectural equilibrium, where all agents' expectations are realized, and each agent responds optimally to its expectations. We present a generic multiagent exchange situation, in which competitive behavior constitutes a conjectural equilibrium. We then introduce an agent that executes a more sophisticated strategic learning strategy, building a model of the response of other agents. We find that the system reliably converges to a conjectural equilibrium, but that the final result achieved is highly sensitive to initial belief. In essence, the strategic learner's actions tend to fulfill its expectations. Depending on the starting point, the agent may be better or...

When designing agent systems, it is often infeasible to foresee all the potential situations an agent may encounter and specify an agent behavior optimally in advance. In order to overcome these design problems, agents have to learn from and adapt to their environment.

An original Reinforcement Learning (RL) methodology is proposed for the design of multi-agent systems. In the realistic setting of situated agents with local perception, the task of automatically building a coordinated system is of crucial importance. To that end, we design simple reactive agents in a decentralized way as independent learners. But to cope with the difficulties inherent to RL used in that framework, we have developed an incremental learning algorithm where agents face a sequence of progressively more complex tasks. We illustrate this general framework by computer experiments where agents have to coordinate to reach a global goal.

Abstract not found

This paper examines the potential and the impact of introducing learning capabilities into autonomous agents that make decisions at run-time about which mechanism to exploit in order to coordinate their activities. Specifically, the efficacy of learning is evaluated for making the decisions that are involved in determining when and how to coordinate. Our motivating hypothesis is that to deal with dynamic and unpredictable environments it is important to have agents that can learn the right situations in which to attempt to coordinate and the right method to use in those situations. This hypothesis is evaluated empirically, using reinforcement based algorithms, in a grid-world scenario in which a) an agent's predictions about the other agents in the environment are approximately correct and b) an agent can not correctly predict the others' behaviour. The results presented show when, where and why learning is effective when it comes to making a decision about selecting a coordination mechanism.

Abstract. Open systems are becoming increasingly important in a variety of distributed, networked computer applications. Their characteristics, such as agent diversity, heterogeneity and fluctuation, confront multiagent learning with new challenges. This paper presents the interaction learning meta-architecture InFFrA as one possible answer to these challenges, and introduces the opponent classification heuristic ADHOC as a concrete multiagent learning method that has been designed on the basis of InFFrA. Extensive experimental validation proves the adequacy of ADHOC in a scenario of iterated multiagent games and underlines the usefulness of schemas such as InFFrA specifically tailored for open multiagent learning environments. At the same time, limitations in the performance of ADHOC suggest further improvements to the methods used here. Also, the results obtained from this study allow more general conclusions regarding the problems of learning in open systems to be drawn. 1

Abstract. In this paper we study Relational Reinforcement Learning in a multi-agent setting. There is growing evidence in the Reinforcement Learning research community that a relational representation of the state space has many benefits over a propositional one. Complex tasks as planning or information retrieval on the web can be represented more naturally in relational form. Yet, this relational structure has not been exploited for multi-agent reinforcement learning tasks and has only been studied in a single agent context so far. This paper is a first attempt in bridging the gap between Relation Reinforcement Learning (RRL) and Multi-agent Systems (MAS). More precisely, we will explore how a relational structure of the state space can be used in a Multi-Agent Reinforcement Learning context. 1

This paper presents a novel method for classifying adversaries that is designed to achieve social complexity reduction in large-scale, open multiagent systems. In contrast to previous work on opponent modelling, we seek to generalise from individuals and to identify suitable opponent classes.