21 We analyze the problem of learning about other agents in a class of dynamic multiagent systems, where performance of 22 the primary agent depends on behavior of the others. We consider an online version of the problem, where agents must learn 23 models of the others in the course of continual interactions. Various levels of recursive models are implemented in a 24 simulated double auction market. Our experiments show learning agents on average outperform non-learning agents who do 25 not use information about others. Among learning agents, those with minimum recursion assumption generally perform 26 better than the agents with more complicated, though often wrong assumptions. 2001 Published by Elsevier Science B.V. 27 Keywords: Multiagent learning; Multiagent systems; Computational market 28 29 1.

We present our approach to the problem of how an agent, within an economic Multi-Agent System, can determine when it should behave strategically (i.e. model the other agents), and when it should act as a simple price-taker. We provide a framework for the incremental implementation of modeling capabilities in agents. These agents were implemented and different populations simulated in order to learn more about their behavior and the merits of using agent models. Our results show, among other lessons, how savvy buyers can avoid being "cheated" by sellers, how price volatility can be used to quantitatively predict the benefits of deeper models, and how specific types of agent populations influence system behavior. Topic Areas: Agent Modeling/Learning, Economic Societies of Agents. Introduction When designing open multi-agent systems (i.e. those that allow anyone to add other agents to the system), one must consider how these agents will interact, and design protocols that discourage age...

. We use simulated soccer to study multiagent learning. Each team's players (agents) share action set and policy, but may behave differently due to position-dependent inputs. All agents making up a team are rewarded or punished collectively in case of goals. We conduct simulations with varying team sizes, and compare several learning algorithms: TD-Q learning with linear neural networks (TD-Q), Probabilistic Incremental Program Evolution (PIPE), and a PIPE version that learns by coevolution (CO-PIPE). TD-Q is based on learning evaluation functions (EFs) mapping input/action pairs to expected reward. PIPE and CO-PIPE search policy space directly. They use adaptive probability distributions to synthesize programs that calculate action probabilities from current inputs. Our results show that linear TD-Q encounters several difficulties in learning appropriate shared EFs. PIPE and CO-PIPE, however, do not depend on EFs and find good policies faster and more reliably. This suggests that in s...

. In this paper, we present some of our ongoing experimental research towards investigating advantages of modeling other agents in multiagent environments. We attempt to quantify the value or utility of building models about other agents using no more than the observation of others' behavior. We are interested in empirically showing that a modeler agent can take advantage of building and updating its beliefs about other agents. This advantage can make it perform better than an agent without modeling capabilities. We have been conducting a simulataionbased study using a competitive game called Meeting Scheduling Game as a testbed. First, we briefly describe our multiagent simultaion testbed. Then, we describe in detail our experimental study. We explore a range of strategies from least- to most-informed, and present some of our preliminary results on the relative performance of these strategies. Decreasing the a priori knowledge about the others and increasing the modeling ...

This paper presents some of our experimental work in quantifying the value of building models about other agents using no more than the observation of others' behavior. We view agent modeling as an iterative and gradual process, where every new piece of information about a particular agent is analyzed in such a way that the model of the agent is further refined. We present our bayesian-modeler agent which updates his models about the others using a bayesian updating mechanism. Then, he plays in a rational way using a decision-theoretic approach based on the probabilistic models that he is learning. We experimentally explore a range of strategies from least- to most-informed one in order to evaluate the lower- and upper-limits of the modeler agent performance. We have been running experiments in our test bed, the Meeting Scheduling Game, which resembles some characteristics of the distributed meeting scheduling problem. Keywords: Learning agent's models, bayesian updating, rational deci...

The work presented in this thesis deals with techniques to improve problem solving control skills of cooperative agents through machine learning. In a multi-agent system, the local problem solving control of an agent can interact in complex and intricate ways with the problem solving control of other agents. In such systems, an agent cannot make effective control decisions based purely on its local problem solving state. Effective cooperation requires that the global problem-solving state influence the local control decisions made by an agent. We call such an influence cooperative control. An agent with a purely local view of the problem solving situation cannot learn ...